As the expectations for AI to change finance forever are increasing, a look back would help Investors understand the limitation of information to forecast. Information may carry all the elements to see the future because it comes from the past, but old information creates new information, old drift creates new drift, so much so that the more we touch the system the harder it becomes to separate the signal from the noise. Markets are a complex system and you can see the strange attractor but forecasting where the butterfly’s flapping wings will land the tornado is an impossibility. Till we grasp this constraint, AI is yet another piece of information, which like many other things, will create the illusion of a forecast, and eventually fail. This is the curse of information, you can mount it like a horse, but the probability that it will always take you to your desired destination is much lower than 50%. AI needs to be trained to understand mechanisms if it has to be of any use in markets.

History is a good way to understand mechanisms. One of the greatest bytes of information available out there is the S&P 500. It has history, it has data, it has complexity but it has limited use as a forecasting indicator. Now, many of us may say, "who uses S&P 500 as a forecasting indicator?" Well, there are a million technical analysts out there who look at every tweak and turn of the popular benchmark to actively trade it and understand market trends. There are economists and central bankers who use S&P 500 as a lead indicator for recession. You can’t get rid of a popular benchmark, it will always be at the top list, even higher than the yield curves to be seen as a forecasting measure.

What’s good for IBM is good for America

This phrase was popularized in the mid-twentieth century and is attributed to Thomas J. Watson, Sr., the founder of IBM. The phrase suggests that the success of a large and influential company like IBM can have a positive impact on the broader economy and society. However, the phrase has also been criticized for promoting a narrow view of corporate interests and ignoring the potential negative effects of a company's actions on society and the environment.

There is another saying, "What's good for Walmart is good for America." However, this phrase is not as well-known or widely used as the phrase "What's good for IBM is good for America." The phrase is sometimes used by supporters of Walmart to suggest that the company's low prices and wide availability of goods benefit the broader economy and society. However, like the IBM phrase, this saying has also been criticized for promoting a narrow view of corporate interests and ignoring the potential negative effects of Walmart's business practices on workers, small businesses, and the environment.

While there isn't a well-known saying that specifically states "What's good for Apple is good for America," some people have used similar phrases to suggest that the success of companies like Apple can have positive effects on the broader economy and society. For example, former President Barack Obama once said, "I think what's good for Apple is good for the United States." However, like the other phrases mentioned, this saying has also been criticized for promoting a narrow view of corporate interests and ignoring the potential negative effects of a company's actions on society and the environment.

Mass psychology seeks confirmation and hence the need for such news to act. However, this leadership of news is always fleeting and difficult to quantify. Hence looking at leading indicators, and winners, and seeking forecasts for them only works till it fails. The leadership of a signal is fleeting. Something I have written about in my earlier features.

What’s good for sunspots is good for economic cycles

Stanley Jevons the great economist of the 1780’s found out that Sunspot cycles lead the economic cycles. But later it was found out that it is the other way around, as economic cycles lead the sunspot cycles

Forecasters can’t forecast

The paper you are referring to is "Can Stock Market Forecasters Forecast?" written by Alfred Cowles in 1933. In the paper, Cowles analyzed the ability of financial forecasters to accurately predict stock prices and found that they were not able to consistently outperform the market. This led him to conclude that "stock market forecasters are not only frequently wrong but that there is no basis for believing that they are likely to be right in the future." The paper is widely regarded as a seminal work in the field of financial forecasting.

What’s good for S&P is good for America

There is a similar lead and lag problem with the S&P 500. Does it lead to a recession? Or does it lag the recession? The answer is quite obvious. When an Index is skewed in a few stocks, the indicator is not the index but the few stocks that decide the direction. S&P 500 is a weighted average, and as John Bogle said, invariably the world will oscillate around this mediocrity. Hence mediocrity will always prevail. Beating mediocrity is the Scientific challenge of modern times.

The S&P 500 forecaster

If 50 stocks own more than 50% of the weight of 500 stocks of the S&P 500, what should it do to its forecasting capability? When few stocks, and three sectors own more than 53% of the S&P 500, the index is a skewed representation of the market.

To expect a skew, a tail, to forecast the body of the distribution, only happens in science fiction or with madness of crowds, where tails can wag the dog for a long time. Skews limit the index's ability to accurately reflect the performance of the broader market. The concentration in a relatively small number of stocks can make the S&P 500 more vulnerable to the performance of these particular companies, and less representative of the overall market. As a result, it is designed to fail as a forecasting tool. And relying on AI that uses S&P 500 as an input to forecast, is garbage in garbage out.