B200 is the GPU I've spent the most time with, and the one that broke the mold. It's not a bigger Hopper. It's a different kind of object: two dies, connected by a high-speed bridge, acting as one GPU. That one fact explains everything about it.
What changed from H100
- Two dies. B200 is two Blackwell dies joined by a 10TB/s die-to-die interconnect (NV-HBI / NVLink-C2C). The software sees one GPU; the silicon is two. This breaks the reticle-size wall.
- FP4. 5th-gen tensor cores support FP4 (E4M3) and 2:4 structured sparsity, doubling the FP8 throughput of Hopper.
- 2nd-gen Transformer Engine. Micro-tensor scaling: per-block (32-element) scale factors instead of per-tensor. This is why FP4 quality is much closer to FP8 than naive 4-bit quantization.
- NVLink 5. 1.8TB/s per GPU, up from Hopper's 900GB/s. NVSwitch 4 supports 576-GPU domains (vs 256 on Hopper).
- 192GB HBM3e. At 8TB/s, 2.4x H100's bandwidth.
The numbers
- FP4: 9000 TFLOPS dense. This is the headline number.
- FP8: 4500 TFLOPS dense.
- FP16: 2250 TFLOPS dense.
- Memory: 192GB HBM3e at 8TB/s.
- NVLink 5: 1.8TB/s per GPU, 18 ports.
- Power: 1000W TDP (liquid-cooled GB200).
Why the fabric decides the cluster
At 4000 B200s, the architecture stops being about one GPU and becomes about the fabric. The shape of that fabric depends entirely on which Blackwell system you bought. On GB200 NVL72, NVLink joins 72 GPUs into what behaves like one enormous GPU. On an HGX B200 baseboard, the NVSwitch domain is eight, and the ninth GPU is across a 400 Gb/s NIC. Same silicon, a thirty-six-fold difference in what it costs two GPUs to talk, and every placement decision follows from which one you have. The KV cache pool, the disaggregation, the routing, all of it is built on these two facts: the GPU is fast, and the fabric is faster.
For inference specifically, B200's FP4 is the unlock. A 405B model that needs 8 H100s in FP8 can run on 4 B200s in FP4 with higher throughput. A 671B MoE model like DeepSeek V4 fits in FP4 at ~336GB, which is under the 384GB of a dual-B200 node. That means you can serve a frontier model on two GPUs, not a rack.
Sources
- NVIDIA B200 product page: official specs.
- NVIDIA Blackwell overview: the architecture story.
- NVIDIA Blackwell platform blog: FP4, NV-HBI, NVLink 5 details.