High-Dimensional CVaR Optimization
This article presents a Python case study that optimizes the CVaR for a portfolio of 2000 stocks, illustrating that such problems are practically feasible.
Fast and stable Conditional Value-at-Risk (CVaR) optimization is much harder to implement in practice than variance optimization. This prevents many people from applying CVaR optimization, despite their desire to do so.
Some people even make up excuses such as “we don’t have enough data for CVaR optimization” or “CVaR optimization does not work for high-dimensional portfolios”. For a rejection of the the first excuse, see the case study in the Better Backtesting article.
The second excuse is addressed in this example, where we simulate 2000 stocks and show that even the open-source implementation from the fortitudo.tech Python package is capable of handling this optimization in a few seconds on a normal laptop.
Original CVaR Portfolio Optimization
Rockafellar and Uryasev (2000) first presented a method for optimizing the CVaR of fully general distributions and their associated joint probabilities, i.e., using the market representation from the next-generation investment framework.
In Chapter 6 of the Portfolio Construction and Risk Management book, it is explained how to integrate joint scenario probabilities. You can even test your understanding of the linear programming algorithm in this Python case study.
However, as illustrated in the above Python example and the cvar-optimization-benchmarks repository, the original formulation is unfortunately too slow and unstable for practical applications with high-dimensional markets and many joint scenarios. See also How Challenging is CVaR Optimization?
Speed and stability especially becomes crucial when we want to introduce parameter uncertainty into the portfolio optimization problem using the Resampled Portfolio Stacking approach.
Fortitudo Technologies CVaR Portfolio Optimization
As a consequence of the above issues, we have developed specialized algorithms for fast and stable CVaR optimization. You can see the performance of these algorithms in the cvar-optimization-benchmarks repository.
We also have an open-source implementation, which allows you to optimize CVaR efficient frontiers and portfolios with return targets. Although this implementation is significantly faster than the original, it is still not of production quality, and it does not allow you to have advanced constraints on risk, transaction costs, leverage, etc.
Hence, you should not expect that you can use the open-source implementation for advanced institutional portfolio construction and resampled portfolio optimization. For this, you need a subscription to the Investment Analysis module.
Python case study
You can find the Python case study where we optimize a portfolio of I=2000 stocks over S=10000 joint Monte Carlo scenarios here: https://github.com/fortitudo-tech/fortitudo.tech/blob/main/examples/13_HighDimensionalCVaR.ipynb
The case study uses the same data as Chapter 2 in the Portfolio Construction and Risk Management book and The Normal Distribution Myth SSRN article.
We use the Fully Flexible Resampling method to simulate 21 trading days (1 month) for the 10 equity indices.
We then use the simulation from the 10 equity indices with random factor loadings to simulate S=10000 joint P&L scenarios for I=2000 fictitious stocks.
Note that the particular stocks are not important in this case. We just want something that is non-normal and resembles stock-like P&L, which this simulation achieves.
The first optimization is performed with uniform scenario probabilities, while the second uses an Entropy Pooling posterior probability vector.
For both optimizations, even the open-source implementation solves the problem in 3-4 seconds, clearly illustrating that CVaR optimization is very feasible for high-dimensional markets.
For completeness, we also solve the mean-variance problem in both the prior and posterior case. As it turns out, both of these optimizations are slower than the CVaR optimizations. Hence, CVaR optimization allows us to work with realistic market simulations and elegantly avoids any potential issues related to high-dimensional covariance matrix estimation.
Conclusion
CVaR optimization with fully general distributions for high-dimensional markets is clearly feasible in practice. If someone tells you otherwise, they simply do not know what they are talking about, or do not know how to solve CVaR problems effectively.
Sophisticated institutional investors already have access to fast and stable CVaR optimization algorithms, allowing them to solve problems with complex constraints, derivatives, transaction costs, and CVaR risk budgets.
Right now, there is an opportunity to be ahead of the market as only few market participants have access to this technology. As the adoption increases and accelerates, you will likely fall further behind by continuing to use mean-variance.
In fact, CVaR becomes more natural for high-dimensional markets, because it avoids any issues with covariance matrix estimation.
So, even if we disregard all the conceptual issues with mean-variance, CVaR comes out on top, even in the normally distributed case: