Product Lineup

Specifications

Overview

NVIDIA introduced the Groq 3 LPU (Language Processing Unit) at GTC 2026, following a $20 billion non-exclusive licensing deal with AI chip startup Groq — the largest NVIDIA has ever paid for technology and personnel. Unlike a GPU, the LPU is not a general-purpose accelerator: it does one thing — autoregressive token generation (the decode phase of LLM inference) — and it does it by replacing HBM with a massive on-chip SRAM pool.

Each Groq 3 LPU packs 500 MB of SRAM at 150 TB/s bandwidth — roughly 45× the bandwidth of an H100's HBM3 (3.35 TB/s) and 7× faster than a Rubin GPU's HBM4. Because decode is fundamentally memory-bandwidth-bound (every token reads the full weight matrix and KV cache), that SRAM wall-breaking bandwidth translates directly into tokens. NVIDIA cites ~150 tokens/watt for 70B FP8 models and 35× throughput improvement versus a Blackwell NVL72 for a 1-trillion-parameter model when paired with Vera Rubin NVL72.

The LPX rack packages 256 LPUs (LP30 chips) with 128 GB of aggregate SRAM and 40 PB/s of aggregate bandwidth, holding a full 70B FP8 model plus KV cache in a single rack with deterministic, flat decode latency. Deployment targets dense LLM serving (7B–70B) using a ~25% LPU / 75% GPU hybrid — GPUs handle prefill and attention math while LPUs offload feed-forward network (FFN) layers to their high-bandwidth SRAM.