Dear friends,
On the LMSYS Chatbot Arena Leaderboard, which pits chatbots against each other anonymously and prompts users to judge which one generated a better answer, Google’s Bard (Gemini Pro) recently leaped to third place, within striking distance of the latest version of OpenAI’s GPT-4, which tops the list. At the time of this writing, the open source Mixtral-8x7b-Instruct is competitive with GPT-3.5-Turbo, which holds 11th place. Meanwhile, I'm hearing about many small, capable teams that, like Mistral, seem to have the technical capability to train foundation models. I think 2024 will see a lot of new teams enter the field with strong offerings.
The barriers to building foundation large language models (LLMs) seem to be falling as the know-how to train them diffuses. In the past year, a lot of LLM technology has taken steps toward becoming commoditized. If it does become commoditized, who will be the winners and losers?
Meta has played a major role in shaping the strategic landscape by emphasizing open source. Unlike its big-tech peers, it makes money by showing ads to users, and does not operate a cloud business that sells LLM API calls. Meta has been badly bitten by its dependence on iOS and Android, which has left it vulnerable to Apple and Google hurting its business by imposing privacy controls that limit its ability to target ads precisely. Consequently, Meta has a strong incentive to support relatively open platforms that it can build upon and aren’t controlled by any one party. This is why releasing Llama as open source makes a lot of sense for its business (as does its strong support for PyTorch as a counterweight to Google’s TensorFlow). The resulting open source offerings are great for the AI community and diffusion of knowledge!
In contrast, Google Cloud and Microsoft Azure stand to benefit more if they manage to offer dominant, closed source LLMs that are closely tied to their cloud offerings. This would help them to grow their cloud businesses. Both Google Cloud and Microsoft Azure, as well as Amazon AWS, are in a good position to build meaningful businesses by offering LLM API calls as part of their broader cloud offerings. However, I expect their cloud businesses to do okay even if they don’t manage to offer an exclusive, clearly dominant LLM (such as Gemini, GPT-4, or their successors). If LLMs become commoditized, they should do fine simply by integrating any new LLMs that gain traction into their API offerings.
Open or closed, LLMs also offer these companies different opportunities for integration into their existing product lines. For example, Microsoft has a huge sales force for selling its software to businesses. These sales reps are a powerful force for selling its Copilot offerings, which complement the company’s existing office productivity tools. In contrast, Google faces greater risk of disruption to its core business, since some users see asking an LLM questions as a replacement for, rather than a complement to, web search. Nonetheless, it’s making a strong showing with Bard/Gemini. Meta also stands to benefit from LLMs becoming more widely available. Indeed, LLMs are already useful in online advertising, for example, by helping write ad copy to drives more clicks.
Tech giants can afford to invest hundreds of millions or even billions of dollars in building LLM technology only to see it become commoditized shortly afterward. Startups would have a harder time surviving after burning this much cash with little to show for it. However, well funded startups will have some time to explore other paths to growing revenue and building a moat.
Finally, competition among companies that offer LLMs is great for everyone who builds applications! With so much investment, by both big companies and startups, in improving LLMs and offering them as open source or API calls, I believe — as I described in this talk on “Opportunities in AI” — that many of the best business opportunities continue to lie in building applications on top of LLMs.
Keep learning!
Andrew