Mixture-of-Experts (MoE) LLM GPU Guide 2026 — Memory Bandwidth, VRAM & Multi-GPU Requirements for DeepSeek-V3, Mixtral & Qwen3-MoE

In 2026 the best open-weight models — DeepSeek-V3 and R1, Mixtral 8x7B and 8x22B, Qwen3-MoE, and Llama 4 Maverick — share one architectural trait: they are sparse, built as a Mixture of Experts (MoE) rather than a single dense network. That single design choice quietly inverts the GPU buying calculus. A dense model pays for its intelligence with compute; an MoE model pays with memory.

For procurement teams this matters because the vendor spec that sells dense-model servers — TFLOPS — is close to irrelevant for MoE. The binding constraints are memory capacity (can the GPU hold every expert?) and memory bandwidth (how fast can the active experts stream through?). Get those two numbers right and a mid-range H200 can serve a 46B-parameter MoE model faster than a higher-TFLOPS GPU with less VRAM.

What Makes a Model a "Mixture of Experts"

A dense 70B model activates all 70 billion parameters on every token. An MoE model keeps far more parameters in total but routes each token through only a small subset. A lightweight router scores the token and dispatches it to the top-k of N parallel "expert" feed-forward networks. The result: a model with frontier-scale capacity at a fraction of the per-token compute.

Why MoE Changes the GPU Buying Calculus

Token generation is memory-bound, and MoE splits that memory problem into two distinct constraints that point at different hardware:

• Capacity is a hard floor. Every expert must be resident somewhere, so the GPU (or GPU cluster) must hold the total parameter count plus the KV cache. DeepSeek-V3 in FP8 needs ~671 GB before you add KV cache — a single H200 (141 GB) cannot hold it, regardless of how few experts activate per token.
• Bandwidth is the throughput ceiling. Only the active experts stream through the compute units each token, so tokens/sec ≈ bandwidth ÷ active-weights-per-token. A Mixtral 8x7B streams ~26 GB of active weights per token (FP16); on a 4.8 TB/s H200 that is a theoretical ~185 tok/s, while the same model on a 1.79 TB/s RTX PRO 6000 tops out near 70 tok/s — even though both GPUs "run" the model.

The practical consequence: for MoE, buy capacity first (enough VRAM to hold all experts without spilling), then bandwidth second (to raise the throughput ceiling). Compute is a distant third. This is the opposite ranking from dense-model server shopping.

GPU Selection for MoE Inference

Match the GPU to the model's total-parameter footprint, not its active count. Mid-size MoE models (Mixtral 8x7B and 8x22B) are single-GPU feasible on HBM parts; frontier MoE models (Qwen3-235B, DeepSeek-V3) are multi-GPU or B200-class decisions.

Multi-GPU & Expert Parallelism for Large MoE

Frontier MoE models exceed any single GPU. DeepSeek-V3's 671B parameters in FP8 require roughly eight H200s (8 × 141 GB = 1.13 TB) once KV cache and activation buffers are accounted for. Two deployment patterns dominate:

• Expert parallelism (EP). Experts are sharded across GPUs, and each token triggers an all-to-all exchange as the router hands work to whichever GPU owns the selected experts. This makes MoE training and serving interconnect-bound — the NVLink/NVSwitch fabric (or InfiniBand/RoCEv2 for multi-node) determines whether expert routing becomes the bottleneck. High-bandwidth GPU-to-GPU links are not optional for EP at scale.
• CPU offload. For budget or single-node deployments, frameworks like ktransformers and llama.cpp keep the frequently-used experts in GPU VRAM and spill the rest to system DRAM. A single RTX PRO 6000 plus a large DDR5 host can run a 671B MoE at a few tokens/sec — viable for evaluation, not for production serving.

Buying checklist for MoE workloads:

• Compute the FP8/INT8 total footprint first (total params × bytes-per-param), then add KV cache for your longest context — this sets your minimum VRAM, single- or multi-GPU.
• Verify bandwidth, not TFLOPS, when estimating tokens/sec — MoE throughput tracks the active-weights-per-token ÷ bandwidth ratio.
• For multi-GPU, budget the interconnect — NVLink/NVSwitch for single-node EP, high-radix InfiniBand or RoCEv2 for multi-node, because expert all-to-all traffic can exceed the compute cost.
• Quantize with care — FP8 cuts the MoE footprint roughly in half and is well-tested on DeepSeek and Mixtral, but verify task accuracy before committing a whole fleet to INT4.