Machine learning models aren’t likely to replace human stock-market analysts any time soon, a new study concluded.
What’s new: Wojtek Buczynski at University of Cambridge and colleagues at Cambridge and Oxford Brookes University pinpointed key flaws in prior research into models that predict stock-market trends. Neither the algorithms nor the regulators who oversee the market are ready for automated trading, they said.
How it works: The authors surveyed 27 peer-reviewed studies published between 2000 and 2018 that used machine learning to forecast the market. They found patterns that rendered these approaches inadequate as guides to real-world investment.
- Prior studies often trained up to hundreds of models based on a single architecture and dataset. Then they tested the models and presented the best results. A real-world investment fund that tried the same thing wouldn’t earn the optimal return. Moreover, if it advertised its best return, likely it would run afoul of the law.
- Where real-world investors can provide a rationale for any trade, many of the proposed models were black boxes that shed little light on how they made a given decision. This lack of transparency also raises regulatory concerns, the authors said.
- The best-performing models predicted correctly whether a stock’s price would rise or fall over 95 percent of the time. That may be a high percentage, but for an investor who holds a high stake, an incorrect prediction can be a huge risk.
- Most of the studies didn’t account for trading costs, which can cut substantially into an investor’s profits.
Behind the news: Although investment funds that claim to use AI have garnered attention, so far they’ve generated mixed results.
- Sentient Investment Management, a hedge fund that used algorithms to control its trading strategies, started in 2016 and gained 4 percent the following year. It failed to make money in 2018 and promptly shut down.
- Rogers AI Global Macro ETF, an AI-driven international fund, launched in 2018 and liquidated its holdings the following year.
- EquBot’s AI Equity ETF, powered by IBM’s Watson, is “the closest we have come across to an AI fund success story to date,” the authors said. It has consistently underperformed the Standard & Poor’s 500, an index of the most valuable U.S. companies.
Why it matters: If machine learning can make predictions, why can’t it predict market activity? A couple of reasons stand out. This paper examines the misalignment between AI research and the likely challenges of real-world deployment. Moreover, even if an algorithm predicts market dynamics accurately within the short term, it will lose accuracy as its own predictions come to influence sales and purchases.
We’re thinking: Studying algorithms that make trading decisions has always been a challenge, since traders tend to keep information about successful algorithms to themselves lest competitors replicate them and dull their edge. Hedge funds that have access to non-public data (for example, specific online chats) have used machine learning with apparent success over years. But those funds haven’t published papers that describe their models!