OpenAI engineers found new optimizations that more than halve the cost of inference, or running its existing models, according to The Information. When applied to power ChatGPT for logged-out users, the techniques cut the Nvidia GPUs needed at one point to just a couple hundred, though the company limits how much those users can access. The savings sharpen a live question of whether OpenAI passes them to customers through higher usage limits or cheaper API pricing, or banks them to improve margins, its gross margin hit 39% last quarter, up from 33% a year earlier but short of the 52% it wants by year-end. Rival Anthropic, which calls these gains compute multipliers, has drawn flack for pricier models even as they produce stronger results.
