SambaNova CEO: Inference Is the Biggest Cost in Enterprise AI

SambaNova CEO Rodrigo Liang said inference has become the biggest cost challenge in enterprise AI, as the company announced a new $3.5 billion AI cloud backed by Vista Equity Partners and Cambium Capital.

Liang, speaking on Bloomberg Open Interest, explained that inference—the process of running trained AI models to generate outputs—now accounts for the majority of AI computing costs for enterprises. He noted that as AI models grow larger and are deployed more widely, inference costs can quickly outpace training costs. SambaNova's new cloud offering uses a disaggregated architecture that combines GPUs, CPUs, and specialized AI processors, which Liang says can dramatically lower costs and accelerate the adoption of AI agents. The architecture allows each component to be scaled independently, optimizing for the specific demands of inference workloads rather than forcing a one-size-fits-all approach. For stock market investors, this development highlights the shifting cost dynamics in the AI value chain. Companies that can reduce inference costs may capture significant market share as enterprises seek to deploy AI at scale. Live stock prices and charts on NowPrice show how the market is reacting to these trends, with AI-related stocks experiencing increased volatility.

Looking ahead, investors will watch for adoption rates of SambaNova's cloud platform and similar offerings from competitors. The success of disaggregated architectures could reshape the competitive landscape for AI hardware and cloud services, potentially benefiting companies that specialize in efficient inference solutions. Key metrics to monitor include enterprise customer growth, cost-per-inference benchmarks, and the pace of AI agent deployment across industries.