Sequential Entropy Pooling

A high-level presentation of Sequential Entropy Pooling (SeqEP), which is a powerful method for views and stress-testing of fully general return distributions.

A comparison of results for the original Entropy Pooling heuristic and the H1 sequential heuristic.

Entropy Pooling is a core method of the next generation investment framework, thoroughly presented in the Portfolio Construction and Risk Management book¹.

As a very oversimplified introduction to Entropy Pooling, you can think about it as a generalization of the Black-Litterman model without all the oversimplifying normal distribution and CAPM assumptions.

Entropy Pooling works for fully general investment distributions and associated joint scenario probability vectors:

\(R=\left(\begin{array}{cccc} R_{1,1} & R_{1,2} & \cdots & R_{1,I}\\ R_{2,1} & R_{2,2} & \cdots & R_{2,I}\\ \vdots & \vdots & \ddots & \vdots\\ R_{S,1} & R_{S,2} & \cdots & R_{S,I} \end{array}\right)\quad p=\left(\begin{array}{c} p_{1}\\ p_{2}\\ \vdots\\ p_{S} \end{array}\right).\)

It is a theoretically sound generalization of Bayesian updating, in the sense that you have a prior model, input some information about the posterior, and get the posterior distribution through the Entropy Pooling update of the joint scenario probability vector.

The nice feature of this update is that it is consistent and predictive in the sense that it introduces the least amount of spurious structure, eliminating the need to reprice derivatives.

For example, it is just as easy to implement views on a portfolio consisting solely of the S&P 500 as on a portfolio that includes the S&P 500 and thousands of derivatives.

For an introduction to the Entropy Pooling intuition, watch this video:

Entropy Pooling views/stress-tests

Entropy Pooling is essentially a minimization of the relative entropy, also known as the Kullback–Leibler (KL) divergence

\(q=\underset{x}{\text{argmin}}\ x^{T}\left(\ln x-\ln p\right),\)

subject to linear constraints on the posterior probabilities

\(Gx\leq h \quad \text{and} \quad Ax=b.\)

The matrices G and A contain functions of the market simulation matrix R. For example, an equality view on the mean of column i is given by

\(R_{i}^{T}x=\tilde{\mu}_{i}.\)

Many different views/stress-tests can be easily implemented, for example, views on means, variances, skewness, kurtosis, correlation, and VaR. Fully general CVaR views are also possible, while they are quite hard to implement fast and stable algorithms for, but it is practically feasible.

See the Portfolio Construction and Risk Management book for all the details about how you implement the above views, how you solve the Entropy Pooling problem in a fast and stable way, and how you can incorporate view confidences.

Sequential Entropy Pooling

While Entropy Pooling, originally introduced by Meucci (2008)², is powerful, we can get even better results by using it in a clever sequential way as first presented in the Sequential Entropy Pooling Heuristics article³.

The main limitation of the original Entropy Pooling approach is the requirement that the constraints are linear in the posterior probabilities. This ensure that we can find the solution in a fast in stable way, even for very high-dimensional simulations.

We can overcome many of the issues imposed by the linear constraints by carefully partitioning our views and processing them sequentially.

Sequential Entropy Pooling typically gives much better results for the same views, that is, a lower relative entropy with the exact same constraints.

The cover image to this post illustrates an example where the joint distributions of developed market equities and emerging market equities are shown.

As we see from the image, the effective number of scenarios (ENS) is significantly higher for Sequential Entropy Pooling and, hence, the relative entropy is significantly lower.

See the Portfolio Construction and Risk Management book for more details about the effective number of scenarios and its relation to the relative entropy.

The video below gives you a quick introduction to Sequential Entropy Pooling, implementing mean and variance views on S&P 500 and STOXX 50.

You can find the accompanying Python code to the video here⁴.

Entropy Pooling resources in one place

I have shared many Entropy Pooling resources publicly. You can find a summary of all of them here⁵.

Chapter 5 in the Portfolio Construction and Risk Management book gives a complete mathematical presentation of all aspects of related to Entropy Pooling, so you are encouraged to study this chapter if you want to build a very deep understanding.

Exclusive case studies

Paid subscribers⁶ have access to walkthroughs of exclusive case studies⁷ that include several advanced Entropy Pooling applications. As a free subscriber, you can claim a one-time access to a paid post.

Future paid posts will contain many exclusive case studies using the next generation investment framework. Consider a paid subscription to lock in the current subscription price.

1. Portfolio Construction and Risk Management post: https://antonvorobets.substack.com/p/pcrm-book

2. Fully Flexible Views: Theory and Practice SSRN article: https://ssrn.com/abstract=1213325

3. Sequential Entropy Pooling Heuristics SSRN article: https://ssrn.com/abstract=3936392

4. Sequential Entropy Pooling Python example: https://github.com/fortitudo-tech/fortitudo.tech/blob/main/examples/2_SequentialEntropyPooling.ipynb

5. Entropy Pooling Collection post: https://antonvorobets.substack.com/p/entropy-pooling-collection

6. Introducing Paid Content post: https://antonvorobets.substack.com/p/introducing-paid-content

7. Quantamental Investing content for paid subscribers: https://antonvorobets.substack.com/t/paid-content