Academic Confirmation Bias
This article explains how finance and economics academia produces a genre of "confirmation bias research".
Maintaining the status quo and searching for information that confirms its sufficiency are fairly well established human biases. Hence, producing research that satisfies these biases is an easy way to make it popular among many, although it does not contribute anything new scientifically and is often directly anti-scientific.
I have seen many examples of this in finance and economics academia. In fact, it is rather the rule than the exception for articles published in academic journals, because they usually need to pass a status quo biased peer review.
In this article, I will present some common, scientifically-flawed cases that I see repeated for investment methods. Common for all of them is that they are used to justify mean-variance, utility theory and other old methods that have obvious real-world flaws.
The logical inconsistencies are typically obfuscated with an excessive amount of mathematical jargon, which is perhaps designed to discourage the reader from questioning the excessive number of modeling assumptions and their real-world validity.
Finally, the evaluation metrics are selected in a way that is almost guaranteed to produce the desired conclusion, which is naturally not questioned by other academics who themselves have used the particular metric or evaluation criterion.
The “mean-variance is sufficient” article
The number of articles with this perspective is enormous. It is typically framed as “you misunderstood something about mean-variance”. However, the reality is that you probably did not misunderstand anything, while some university academics keep trying to sell the claim that mean-variance produces valuable insights despite purposefully ignoring all the aspects that separates good portfolio construction from bad portfolio construction. The motivation behind such articles is mainly to maintain the relevance of previous academic work including articles, books and courses. The note below summarizes the undeniable features of mean-variance:
The “this new approach does not make a difference” article
This type of article is often used in combination with mean-variance justification. For example, where a risk model is intentionally made elliptical using a normal or t distribution and then various portfolio optimization methods are evaluated based on those market assumptions.
As explained in the Variance for Intuition, CVaR for Optimization article, CVaR optimization almost always coincide with mean-variance optimization when return distributions are elliptical, so the conclusion is simply an empirical validation of that theorem, like this Python example.
Another variant is to evaluate how well empirical Monte Carlo simulations perform for mean-variance optimization and then compare to methods that directly estimate the covariance matrix to draw the biased and illogical conclusion that “Monte Carlo simulations do not improve portfolio construction”. The issue here is simply that we go through the harder work of generating realistic Monte Carlo simulations exactly because we don’t want to rely on the mean-variance oversimplification, so it makes no sense to evaluate them based on what we are intentionally trying to avoid.
No matter which of the two perspectives the case study is approached from, it presents some clear logical inconsistencies, and the conclusion follows by construction, not from a properly designed experiment.
The “highly specific example generalized” article
A related common type of biased research is one that takes a very specific example or carefully constructed case and then quickly generalizes it to almost all other cases. This is especially common in backtesting, where a method is evaluated on one historical path combined with a carefully chosen train/test split of the data. This situation can actually go both ways, depending on whether you want to confirm the status quo or show superiority of a new method. Hence, this is why we always recommend validating the results on properly generated synthetic data.
The “trivial relation presented as something meaningful and insightful” article
An example of this kind of article is a ranking the standalone risk of various instruments, for example, computing the variance and CVaR and illustrating that they typically rank according to the same order and then proceed to make the illogical, status quo biased conclusion that “it does not matter which risk measure you use”, completely ignoring all other nuances.
It is almost trivial that there is a positive relation between variance and CVaR or any other risk measure like semi-variance and (lower semi)-absolute deviation for that matter. Below is a Python case study where I empirically verify this for 10 US equity indices using the fortitudo.tech Python package.
A logically-consistent alternative
The Fully General Investment Framework (FGIF) presents a logically-consistent alternative to the variance-biased articles above. FGIF is thoroughly described in the Portfolio Construction and Risk Management book, which is carefully presented in the Applied Quantitative Investment Management course.
The Applied Quantitative Investment Management course requires a paid subscription, which will also give you full access to the other paid content as well as asking me questions through the Substack chat.