I was at the AI conference in the top AI city, Toronto. There is a question I ask the panelists at the various AI conferences. I have not got an answer yet.
The Question
The industry is excited about jumping cats, kittens and felines because they are the most searched and consequently important for AI to be able to identify. But how with all this brilliance around identifying cats, AI can not solve the USD 100 trillion investment management problem of not beating the benchmark?
I got the following answers. First; cats are more important than the benchmark. Second; AI needs to take small steps. Third; solving cancer is more important than beating the benchmark. Fourth; driverless cars is where the industry is focussed as traffic is a bigger problem than beating the benchmark.
With the world under debt, student loans assumed to be bigger than the subprime problem, ignoring finance is like a blindfolded trapeze act. AI fledglings are presumptuous. They don't understand the market risk but Poker accuracy. They don't understand fixed income but lofty valuations. They don't understand computational linguistics but chat bots. Unfortunately, the industry does not have time to be so ignorant about the need for AI in finance. I don't think the AI community really knows how much money BlackRock or Vanguard manages, why it's important to build solutions that could save them or whether they need to be saved in the first place.
Jumping Cat Illusion
The latest AI techniques have emerged from improved statistical procedures and better data accessibility. The two could give confidence but can not impart vision. AI does not understand finance, the data the financial industry generates and its similarity to other kind of datasets. This is why AI's ambition to replace humans is misplaced. If AI can not understand human emotion's messy case study i.e. stock markets, AI is living a jumping cat illusion.
About Benchmarks
Benchmarks are not just for the stock markets, they are for everything data. Benchmarks are metrics with a predictive capability. It's important to understand that indexing the accuracy of a prediction is at the heart of science. Even if it may mean doing something as inane as creating a jumping cats AI benchmark to track the accuracy of the cat detecting AI algos on a daily basis. If detecting cats is easy, one can add the 'jumping' variable to it, or for that matter a fence. There may be only 0.049% images of cat jumping over a fence on the web.
Real accuracy is real dollar
We are heading into a digitized world. A predictive value can be unitized and can be given a dollar value. This means accuracy outside finance can be looked at as a financial portfolio, which means AI accuracy can be benchmarked. And when AI accuracy can be benchmarked then you can look at AI models as a set of portfolios, which can outperform or underperform a composite benchmark.
Domain Agnostic AI
AI needs to mature and become domain agnostic. Today it is domain specific. Till we cross this threshold, we will consider financial underperformance as a crisis happening next door, a flawed assumption with fatal consequences.