NVIDIA Vera Rubin: Why the Rack Is the New GPU

NVIDIA's next data center platform, Vera Rubin, is entering production in 2026, and the most important thing about it isn't on the spec sheet. It's that NVIDIA has stopped selling you a chip and started selling you a rack. Vera Rubin is designed, built, and sold as a rack-scale system, the NVL72, and understanding that shift matters more than any single performance number, because it changes what you're actually buying. This is the reference architecture story, and it's the lens this piece uses to explain what Vera Rubin is and how it breaks from the hardware that came before it.

TLDR

• Vera Rubin is NVIDIA's successor to Hopper and Blackwell: the Vera CPU paired with the Rubin GPU, delivered as a rack-scale system (the NVL72) rather than as standalone cards.
• Its reference architecture (the third generation of NVIDIA's MGX design) treats the entire rack as a single unit of compute, a distributed accelerator that NVIDIA designs end to end.
• The clean break from Blackwell: a single high-density SKU, cable-free modular trays, mandatory direct-to-chip liquid cooling, and roughly 190–230 kW per rack.
• Inference now splits across two rack types: Rubin GPUs handle prefill, and Groq 3 LPX racks handle decode. NVIDIA's earlier Rubin CPX chip has been shelved.
• This is Part 1: what Vera Rubin is and why its architecture is different. Part 2 covers what it means for your buildout.

What NVIDIA Vera Rubin Actually Is

Vera Rubin is a platform, not a single chip. The name pairs two components: Vera, an NVIDIA-designed CPU built on 88 custom Arm cores, and Rubin, the GPU that succeeds Blackwell. NVIDIA names each generation after a scientist and pairs a CPU with a GPU. The prior generation was Grace Blackwell (the Grace CPU alongside the Blackwell GPU, shorthanded "GB," as in GB200), and Vera Rubin is the next turn of that pattern. In its flagship configuration they ship together as the Vera Rubin NVL72, a rack holding 72 Rubin GPU packages and 36 Vera CPUs, wired together to act as one machine.

The Rubin GPU is where the raw generational jump lives. Each one carries 288 GB of HBM4 (the high-bandwidth memory stacked directly on the GPU package, which is what lets a chip keep large models and long context resident without constantly reaching out to slower memory), running at up to ~22 TB/s, and delivers roughly 50 petaFLOPS of NVFP4 compute. New this generation, the platform also folds in a third processor type: a Groq 3 LPU, an inference accelerator that sits alongside the Rubin GPUs and handles a specific slice of the workload (more on that below), joined by NVLink 6, ConnectX-9 networking, and BlueField-4 DPUs.

One point of confusion worth clearing up: you'll see both "NVL144" and "NVL72" attached to this rack. They describe the same system. The original NVL144 figure counted GPU dies; the official NVL72 name counts the 72 GPU packages, each of which contains two dies. You may also see the rack referred to as the VR200. Production reached its first milestones in 2026 with initial racks going live at hyperscalers, and broad availability across the major clouds is expected through the second half of 2026.

What a "Reference Architecture" Means Here

A reference architecture is the standardized, modular blueprint that server makers and cloud builders build to. Instead of NVIDIA shipping a chip and leaving each customer to design a server around it, NVIDIA specifies the whole rack (the trays, the interconnect, the power and cooling design) so that systems from different vendors are interoperable, validated, and fast to deploy. For Vera Rubin, that blueprint is the third generation of NVIDIA's MGX design, with more than 80 ecosystem partners building hardware to it.

The consequence is the through-line of this entire platform: the rack becomes the unit of compute. A Vera Rubin NVL72 is engineered to behave as a single distributed accelerator, and NVIDIA designs the system, not just the silicon inside it. NVIDIA frames the whole platform as a move away from discrete chips and standalone servers toward integrated rack-scale systems assembled from specialized racks — a framing that becomes concrete once you look at how the generation actually differs from Blackwell.

How Vera Rubin's Reference Architecture Differs From Blackwell

The clearest way to see the shift is to line up three generations. Hopper (the H100 era) was a server story: eight GPUs in an air-cooled HGX box, drawing on the order of 40 kW per rack. Blackwell introduced the rack as the product with the GB200 NVL72, but it hedged: NVIDIA also offered a lower-density GB200 NVL36×2 configuration as an off-ramp for data centers that weren't ready for 100kW-plus racks.

Vera Rubin removes the hedge and tightens the specification. The flagship ships as a single NVL72 SKU with no lower-density fallback, uses cable-free modular trays, and requires 100% direct-to-chip liquid cooling. It is no longer an option. Rack power climbs to roughly 190–230 kW.

The per-GPU numbers matter, but they're in service of the rack, not the other way around: more memory, more bandwidth, and a faster interconnect are what make it possible to treat 72 packages as one accelerator. What's genuinely new this generation isn't a faster chip — it's that the data center itself is disaggregating into function-specific racks. NVIDIA's original plan for this was a single monolithic GPU variant called Rubin CPX; that plan was set aside in favor of the split covered next. The trajectory continues from there: NVIDIA's 2027 roadmap points to 800 VDC power and roughly 600 kW Kyber racks for Rubin Ultra, scaling the NVLink domain to 576 GPUs.

The Inference Split: Rubin GPUs and Groq LPUs

Inference runs in two phases with opposite hardware appetites. Prefill (digesting the prompt and context) is compute-bound, and GPUs are excellent at it. Decode (generating output tokens one at a time) is latency- and bandwidth-bound, and it's the phase where a general-purpose GPU leaves performance on the table.

NVIDIA's answer arrived through its roughly $20 billion licensing deal for Groq's technology, announced in December 2025, and took product form at GTC 2026 as the Groq 3 LPX, a decode accelerator that plugs into the Vera Rubin platform. An LPU is a purpose-built inference ASIC: an application-specific chip built to do one job, generating tokens, far more efficiently than a general-purpose GPU can. Its arrival inside Vera Rubin is NVIDIA folding that class of specialized silicon into its own platform, a shift we've been tracking in our AI ASIC series. The result is what NVIDIA calls Attention–FFN disaggregation: Rubin GPUs handle prefill and attention, Groq LPUs handle decode, and NVIDIA's Dynamo software routes work between them token by token. No CUDA changes are required; the LPU operates as an accelerator to the existing stack. An LPX rack packs 256 Groq LPUs and leans on large on-chip SRAM rather than HBM, trading memory capacity for the ultra-low-latency bandwidth that sequential decode rewards. The earlier Rubin CPX, a GDDR7 GPU variant NVIDIA had sketched for long-context inference, has been shelved in favor of this LPU path.

Vera Rubin FAQ

Vera Rubin and the Rack as the New Unit of Compute

The generational story of Vera Rubin isn't a faster GPU; it's a change in what NVIDIA sells. Compute now arrives as a facility-scale, multi-tier reference architecture: a high-density GPU rack for prefill, a specialized LPU rack for decode, and a single blueprint tying them together. The rack is the new GPU — and with the inference split, sometimes it's two racks.

That reframing is exactly why the buying decision looks different this generation, and it's where we'll pick up in Part 2: what the reference architecture means for your buildout, and what we're seeing in the market. If Vera Rubin is already on your radar, start a conversation with our GPU hardware team.