Quick Buzz Feed
Investors should know the difference between AI training and AI inference.
The AI market is dividing into two distinct segments: training and inference. The AI training market involves 'teaching' AI algorithms by processing vast amounts of data using large GPU clusters in data centers. This process is expensive, can last for weeks or months per model, and is cyclical. Leading companies in this space, such as OpenAI, Meta, and Alphabet, invest heavily in training their large language models (LLMs). In contrast, the AI inference market focuses on the practical application of these trained AI models, such as answering questions with ChatGPT or generating content. Inference is seen as a more stable and recurring revenue stream, with many AI-oriented companies increasing their spending in this area.
Nvidia has significantly benefited from the initial AI training boom, dominating over 90% of this market with its GPUs and proprietary software. Its strong market position and ecosystem make it a leader in AI training. However, Nvidia's general-purpose GPUs are less optimized for inference-oriented software, leaving it vulnerable to competitors like Broadcom. Broadcom develops application-specific integrated circuits (ASICs) that accelerate AI inference for hyperscalers at a lower cost. Major Nvidia customers, including Google and Meta, are investing in Broadcom's accelerators and developing their own custom chips. To adapt to this shift, Nvidia has partnered with AI inference start-up Groq, developing a new language processing unit (LPU) to compete in the inference market.
Many prominent AI companies, such as Nvidia, Google, and Meta, are involved in both the AI training and inference markets. However, the emerging split suggests that the inference market could offer more stable growth compared to the volatile training market. Nvidia's strategic investment in Groq highlights the increasing importance of inference. While Nvidia remains a key player in the broader AI landscape, investors should closely monitor the growth of inference-focused companies like Broadcom, as they may achieve greater gains than those primarily focused on AI training in the coming years.