Hopper vs Blackwell: notes from running both generations

Our fleet straddles the generation gap: H100 NVL pair in the production DL380, Blackwell RTX cards on the test bench. That means I do not have to speculate about Hopper versus Blackwell from launch-keynote slides. I ssh into both every week. This is the comparison I actually use when someone asks which generation their money should go to in 2026.

One framing note before the details: "Hopper vs Blackwell" is really two comparisons wearing one name. There is the datacenter fight (H100/H200 versus B200-class parts) and the workstation fight (nothing, versus RTX PRO 6000 Blackwell, since Hopper never had a workstation SKU). Most of the confusion I see comes from mixing the two. I will try to keep them separate.

the one-sentence version of each architecture

Hopper, 2022-2024's king: sm_90, HBM3/HBM3e, the first Transformer Engine with native FP8, fourth-generation NVLink, TMA for async memory movement. The architecture that made FP8 inference a boring production default, which is the highest compliment infrastructure can earn.

Blackwell: the second-generation Transformer Engine adds native FP4 with microscaling, memory moves to HBM3e on the datacenter parts and GDDR7 on the workstation cards, NVLink jumps another generation, and the big datacenter die is actually two dies pretending very convincingly to be one. The headline is FP4; the substance, as usual, is memory bandwidth.

decode is a memory business, so bandwidth decides

The single most important number for LLM serving is not TFLOPS. Token-by-token decode is memory-bandwidth-bound: every generated token requires streaming the active weights and the KV cache through the chip. Whoever moves bytes faster generates tokens faster, almost linearly, until batching changes the regime.

That lens sorts the whole product stack honestly. Our H100 NVL cards push roughly 3.9 TB/s of HBM3. A B200-class Blackwell part roughly doubles that. The RTX PRO 6000 Blackwell, for all its architectural novelty, pushes about 1.8 TB/s of GDDR7. That is why our two-generations-newer test cards decode at roughly half the speed of our two-generations-older production cards, a fact that reliably surprises visitors. Architecture generation is a weak predictor of serving speed. Memory technology is a strong one.

A newer architecture with slower memory loses the decode race to an older architecture with faster memory. Buy the byte pump, not the model year.

Prefill is the counterweight: it is compute-bound, batches well, and is where Blackwell's fatter tensor cores genuinely stretch out. If your workload is heavy on long prompts and light on long generations (RAG with short answers, classification, embedding-adjacent work), Blackwell's advantage is bigger than the decode numbers suggest. If your workload is chat and code generation, bandwidth still rules. Know which business you are in before reading any benchmark. I went through our own measurement discipline in measuring local tokens.

FP4 is the real generational divide

Feature-wise, one line matters more than everything else combined: Blackwell has FP4 tensor cores and Hopper does not. Not "slower FP4." None. Hopper stops at FP8.

FP8 on Hopper is, in 2026, gloriously mature. Kernels are tuned, quantized checkpoints ship day-one from model vendors, quality loss versus bf16 is within eval noise for most models we serve. It is the format our whole production serving stack stands on, and it halves both the memory footprint and the bandwidth cost of every weight touched, which, as noted above, is the same thing as generating tokens faster.

NVFP4 does it again: another halving, with two-level microscaling keeping the quality damage mostly acceptable. On a bandwidth-limited card, weights at 4.5 effective bits are not just smaller, they are proportionally faster to stream. That compounding (half the footprint, nearly double the effective decode ceiling) is why I think FP4 is the actual generational moat, not the TFLOPS chart. Our Blackwell test bench runs a whole NVFP4 quantization pipeline precisely because this is where the next few years get decided, and per-model quality verdicts there are genuinely mixed: dense models mostly shrug, some MoEs lose real capability. The format is a moat; it is not yet a default.

The strategic consequence for anyone owning Hopper: your hardware's terminal format is FP8. Every year the ecosystem leans harder into 4-bit-native serving, and Hopper will watch that from the sidelines. Not a 2026 problem. A 2027-2028 problem, and worth pricing in.

the interconnect story, and who actually needs it

NVLink is the other genuine architectural gap. Hopper's fourth-generation NVLink gives our NVL pair its 600 GB/s bridge, which makes tensor-parallel serving of big models across two cards nearly free. Datacenter Blackwell moves to fifth-generation NVLink at roughly triple the per-GPU bandwidth and, in the GB200-class racks, extends coherent NVLink domains across dozens of GPUs. That is the thing that makes trillion-parameter serving a product instead of a research demo.

Here is the part that matters for teams our size: none of that trickles down. The Blackwell RTX workstation cards have no NVLink at all: PCIe only, same as consumer cards. Which means a pair of 96 GB Blackwell RTX cards is two islands, while our elderly Hopper pair is one continuous 192 GB lake. For multi-GPU inference boxes below datacenter budgets, Hopper NVL remains weirdly unmatched; our test bench post covers living with the island model. If your model needs TP=2 to exist, that ancient bridge beats the shiny new architecture that lacks one.

software maturity: sm_90 is pavement, sm_120 is gravel

Eighteen months of production taught me to weight this more heavily than any spec. Hopper's sm_90 is the most-paved target in the CUDA ecosystem: every attention kernel, every quantization scheme, every vLLM release is tested there first, because that is what the labs and clouds run. Things just work, and when they do not, someone else hit the bug a week before you and the issue thread already has a workaround.

Blackwell RTX's sm_120 is noticeably rougher. We have hit kernels that silently fall back to slower paths, a FlashAttention variant that needed a version bump to stop crashing, and prebuilt wheels that shipped without sm_120 cubins entirely, gifting us a two-minute JIT compile on first load. All survivable (the test bench exists to absorb exactly this, per our lab-to-prod pipeline), but if someone put sm_120-only hardware in my production path today, I would decline. Datacenter Blackwell is better paved than the RTX side, but Hopper is still where the ecosystem's reflexes live. Give it another year.

so which one do you buy in 2026

My honest decision tree, with the usual caveat that our numbers are our numbers and /local-cost exists for yours.

• Serving big models to a team today, budget in the tens of thousands: used or discounted Hopper. H100 NVL pairs are coming off cloud rotations at prices that were fantasy in 2024, FP8 serving on them is a solved problem, and the NVLink bridge buys you a 192 GB pool no sub-datacenter Blackwell config can match. This is the boring, correct answer and it is what our production box will remain until at least next year.
• Dev, triage, quantization work, single-card experiments: Blackwell RTX. Same 96 GB as an H100 NVL at a fraction of the price, plus the FP4 hardware Hopper will never have. Capacity and future-format access matter more than bandwidth on a bench.
• Building fresh at datacenter scale with datacenter money: B200-class Blackwell, obviously, with double the bandwidth, FP4, and an interconnect roadmap. The calculus only gets interesting below that budget line, which is where most of us live.
• Sitting on working Hopper and wondering about upgrading: do not. The upgrade buys you FP4 readiness and bandwidth you can likely not saturate with your team's actual load. Spend the money on RAM, NVMe, or a test bench instead. The host around the GPUs is usually the cheaper speedup.

The limitation of everything above: I am comparing from the cheap seats. We run workstation Blackwell, not B200s, so my Blackwell datacenter judgments lean on the same public numbers everyone else reads, filtered through what the RTX cards let me verify locally. The architecture is real to me; the flagship parts are not. Season accordingly.

The meta-lesson after running both generations side by side for half a year: architecture generations matter less than three numbers nobody prints on the box, namely memory bandwidth, memory capacity, and whether your quantization format has native hardware. Get those three right for your actual workload and the generation name on the invoice is mostly branding. Get them wrong and no amount of newness saves you.