B200 is the GPU I've spent the most time with, and the one that broke the mold. It's not a bigger Hopper. It's a different kind of object: two dies, connected by a high-speed bridge, acting as one GPU. That one fact explains everything about it.
What changed from H100
- Two dies. B200 is two Blackwell dies joined by a 10TB/s die-to-die interconnect (NV-HBI / NVLink-C2C). The software sees one GPU; the silicon is two. This breaks the reticle-size wall.
- FP4. 5th-gen tensor cores support FP4 (E4M3) and 2:4 structured sparsity, doubling the FP8 throughput of Hopper.
- 2nd-gen Transformer Engine. Micro-tensor scaling: per-block (32-element) scale factors instead of per-tensor. This is why FP4 quality is much closer to FP8 than naive 4-bit quantization.
- NVLink 5. 1.8TB/s per GPU, up from Hopper's 900GB/s. NVSwitch 4 supports 576-GPU domains (vs 256 on Hopper).
- 192GB HBM3e. At 8TB/s, 2.4x H100's bandwidth.
The numbers
- FP4: 9000 TFLOPS dense. This is the headline number.
- FP8: 4500 TFLOPS dense.
- FP16: 2250 TFLOPS dense.
- Memory: 192GB HBM3e at 8TB/s.
- NVLink 5: 1.8TB/s per GPU, 18 ports.
- Power: 1000W TDP (liquid-cooled GB200).
Why the 4000-GPU cluster works
At 4000 B200s, the architecture stops being about one GPU and becomes about the fabric. NVLink 5 connects 72 GPUs in a rack (NVL72) into what is effectively one giant GPU, and NVSwitch 4 extends that to 576-GPU domains. At 4000 GPUs, that's ~7 domains, with only inter-domain traffic going over InfiniBand or Ethernet with RDMA. The KV cache pool, the disaggregation, the routing, all of it is built on these two facts: the GPU is fast, and the fabric is faster.
For inference specifically, B200's FP4 is the unlock. A 405B model that needs 8 H100s in FP8 can run on 4 B200s in FP4 with higher throughput. A 671B MoE model like DeepSeek V4 fits in FP4 at ~336GB, which is under the 384GB of a dual-B200 node. That means you can serve a frontier model on two GPUs, not a rack.
Sources
- NVIDIA B200 product page: official specs.
- NVIDIA Blackwell overview: the architecture story.
- NVIDIA Blackwell platform blog: FP4, NV-HBI, NVLink 5 details.