Nvidia licensed Groq’s inference technology for $20B in December 2025 and unveiled the resulting Groq 3 language processing unit at GTC 2026 in San Jose, pairing it with its Rubin GPUs in a heterogeneous architecture where each chip handles a distinct phase of the AI inference pipeline. The Groq 3 LPU delivers 150 terabytes per second of memory bandwidth compared to 22 terabytes per second on Nvidia’s Rubin GPU, making it purpose-built for the token-generation phase of inference where memory speed, not raw compute power, is the bottleneck. Nvidia CEO Jensen Huang projected $1 trillion in orders for Blackwell and Vera Rubin systems through 2027 and claims the combined GPU-plus-LPU architecture delivers up to 35x higher inference throughput per megawatt compared to GPU-only deployments, though the company notes those gains apply to decode-heavy workloads and recommends adding LPUs to only about 25% of a data center’s total capacity.
