What is Indexing?
Stock market indexing is a passive investment strategy that aims to replicate the performance of a specific group of stocks or the overall market. It involves creating a portfolio that closely mirrors the composition and weightings of a particular stock market index, such as the S&P 500 or the Dow Jones Industrial Average. By investing in this diversified basket of stocks, investors can achieve broad market exposure and potentially benefit from overall market growth without attempting to pick individual stocks. This strategy is popular due to its simplicity, low costs, and historical long-term returns.
What is MCAP indexing?
MCAP indexing, or market capitalization indexing, is a passive investment strategy that involves constructing a portfolio of stocks based on their market capitalization or market value. In this approach, companies with larger market capitalizations (the total value of all outstanding shares) have a higher weighting in the index, while companies with smaller market capitalizations have lower weights.
Why is it predominant?
MCAP became the predominant style because of its reliance on calculation convenience which was embedded in the Laspeyres method and the resulting bias inflated the basket value with rising prices. This made it a predominant index among its peers.
Why is it hard to beat?
Because of inflated value and probabilistic advantage over any other investing style (passive or active) created by overweighting winners and hence underweighting losers. An idiosyncratic selection when compared to MCAP weighting is at a probabilistic disadvantage to get both entry and exit of a selection right.
How much money is invested in MCAP indexing?
An estimated USD 15.6 trillion is indexed or benchmarked to the most popular MCAP-weighted index (S&P500) with indexed assets comprising approximately USD 7.1 trillion. The index includes 500 leading companies and covers approximately 80% of available market capitalization. The MCAP method is the most popular benchmarking method in USD 100 trillion investment management industry.
What is wrong with it?
There are many things wrong with it. Apart from extremely skewed concentration i.e. magnificent 7, there is concomitant poor diversification, higher risk, and hence mostly a slower capability to recover after a drawdown.
Who challenged it?
Smart Beta challenged it in 2002.
Why has Smart Beta failed?
Smart Beta failed because it uses various fundamental factors to beat the MCAP method.
What is wrong with factors?
There is a zoo of factors. Factors persist and fail. Factor timing is an impossibility.
Why is MCAP Indexing non-scientific?
Because it relies on one of the statistical laws called 80-20 i.e. bigger the size-bigger the weight winner’s bias. 80% of the value is indeed in 20% of the S&P500 stocks. 80-20 statistics is a behavior, it’s not science.
Why despite being non-scientific, the MCAP method is unchallenged?
The MCAP method is unchallenged because science today can’t explain why 80-20 statistical behavior works and what mechanism drives it. The explanation of the 80-20 law has been the biggest mystery since Vilfredo Pareto first illustrated it in Italian wealth distribution. Near 80% of Nobel prizes in economics reference or assume the 80-20 behavior in their Nobel prize speeches.
What do probabilities have to do with the problem?
This scientific process will rely on the big disadvantage of MCAP methodology, which is its assumption that the probability of today’s winner winning tomorrow is 1. The probability can never be 1 of a winner of today becoming a winner tomorrow because it’s probabilistically impossible that none of the 493 stocks that are not in the magnificent 7 lists in the 500 stocks of S&P won’t grow more than the magnificent 7 stocks. The reason this probabilistic match does not prevail i.e. today’s losers' portfolio doesn’t beat today’s winner’s portfolio because it’s hard to identify which of the losers will beat the magnificent 7, when they will start beating and when will they stop beating.
Can a scientific process challenge the MCAP method and redefine investing?
Yes, only a scientific process can challenge the MCAP method and redefine investing because only science can solve the probabilistic puzzle. How to probabilistically weight a set of components passively, with low tracking error and low turnover without being more winner’s biased like S&P 500 and still beat it.
What will this scientific process look like?
Any scientific process that has to beat the MCAP method has to be able to systematically understand how to bias in and out of the winner’s bias, probabilistically. The only way to probabilistically do this is by using a non-linear mechanism that combines the statistical law of 80-20 with the statistical law of mean reversion. The mechanism has to empirically demonstrate that despite different regimes, different starting points, and the compounding of the winner’s bias, the mechanism is capable to serve a better probabilistic hand than the winner’s bias. If a probabilistic mechanism can do that, it has not only illustrated the functioning of the 80-20 rule but also how a combination of statistical laws is better than one working alone. This would be a true scientific innovation or a step towards an innovative science, which would solve two problems while attempting to solve one.
How can one trust this process?
S&P 500 is a process too that has been trusted by investors globally. When it became the leading benchmark in 1950, it was still a benchmark, which means it was an empirical process that had no real money tracking it. If an open empirical process can illustrate that the MCAP method can be beaten over 20, 30, or 50 years that it’s possible to take 500 S&P 500 stocks and weigh them differently to beat the S&P 500, the proof is conclusive.
What’s Universal Indexing?
Such scientific proof won’t rely on MCAP but on probabilities and hence would be region, asset, and domain agnostic. It would be a universal index that would use just statistical laws to enhance probabilistically advantage overcoming the probabilistic disadvantage of MCAP winner’s bias.
How can AI enhance Universal Indexing?
A pure probabilistic process lends itself to AI well and hence AI can take a systematic empirical process and enhance it. This AI process would study the empirical behavior and detect patterns to improve the method, its rebalancing, its structure, and its complexity.
Why did no one come up with the idea earlier?
The father of statistics wrote about mean reversion in 1886 much after the first Index was created in 1854. The 80-20 law was discovered in 1900. The 80-20 law has seen more powerful expression of nature than mean reversion as late as 2004 when Mandelbrot called mean reversion a great intellectual fraud. The idea of reconciliation of the two laws needed interdisciplinary thinking and the luxury of time to understand statistics, physics, maths, psychology, finance, economics, information, computation, etc. Somebody had to eventually take the step of demystifying the winner’s bias.
What is the universal indexing talk about the future?
Universal Indexing would disrupt not only finance but also non-financial domains because it will create a superior index above everything data. This will not only beat the market consistently with a significant margin on a risk weighted basis but also allow non-financial domains to extract more predictive intelligence from raw data and consistently enhance the process using AI while using a fraction of data, hence low computation a new generation for AI processes.
Bibliography
[1] Matia, Kaushik and Pal, Mukul and Stanley, H. Eugene and Salunkay, H., Scale-Dependent Price Fluctuations for the Indian Stock Market. EuroPhysics Letters, Aug 2003 [2] M. Pal, M. Shah, A. Mitroi, Temporal Changes in Shiller’s Exuberance Data, SSRN, Feb 2011
[3] M. Pal, Mean Reversion Framework, SSRN, May 2015
[4] M. Pal, Markov and the Mean Reversion Framework, SSRN, May 2015
[5] M. Pal, Momentum and Reversion, Aug 2015
[6] M. Pal, What is Value, SSRN, Sep 2015
[7] M. Pal, M. Ferent, Stock Market Stationarity, SSRN, Sep 2015
[8] M. Pal, Reversion Diversion Hypothesis, SSRN, Nov 2015
[9] M. Pal, How Physics Solved your wealth problem, SSRN, Oct 2016
[10] M. Pal, Human AI, SSRN, Jul 2017
[11] M. Pal, The Size Proxy, Aug 2017
[12] M. Pal, The Beta Maths, SSRN, Mar 2017
[13] Maureen, O. Bhattacharya, A. ETFs and Systematic Risk. CFA Research Institute, Jan 2020 [14] M. Pal, [3N] model of life, SSRN, Apr 2021
[15] M. Pal, The S&P 500 Myth, SSRN, Jul 2022
[16] M. Pal, The Snowball Effect, SSRN, Jul 2022
[17] M. Pal, Mechanisms of Psychology, SSRN, Jun 2022
[18] M. Pal, The [3N] model of life, SSRN, Feb 2023