DeepSeek V4 Pro: 1.6 trillion parameters on one node
A 1.6T MoE that reaches 27% of V3.2's per-token inference FLOPs and 10% of its KV cache at a million tokens. It got five times bigger and cheaper to run at the same time.
The architecture
how it is builtHybrid CSA and HCA attention
V4 pairs two attention mechanisms rather than picking one. The combination is what gets the model to 27% of V3.2's per-token inference FLOPs and, more importantly for anyone holding the memory budget, 10% of its KV cache at 1M context.
Manifold-constrained hyper-connections
mHC replaces the plain residual stream with a constrained formulation the paper argues scales better with depth. It is also why I could not point a V3 config at it and expect anything: the serving stack needed a new tokenizer mode and new reasoning and tool-call parsers rather than reusing V3's.
A mixed-precision checkpoint
MoE expert weights are stored in FP4; attention, normalisation and router weights stay FP8. Experts are the overwhelming majority of a 1.6T model, so the checkpoint lands near 0.8 TB rather than the 1.6 TB a uniform FP8 release would cost. That single decision is the difference between a model I can host and a model I can only read about.
49B active
Three percent of the model works on any given token. The other 97% is sitting in HBM waiting to be routed to, which is exactly why total parameters decide whether you can host it and active parameters decide what it costs you.
Serving it
the playbookAn 8-GPU Blackwell node minimum, or a multi-node rack. NVIDIA's own testing on HGX B300 used MXFP4, with NVFP4 named as the planned optimisation.
On a 4,000-GPU HGX B200 fleet
500 nodes, eight B200s each, 180 GB per GPU
Weights are 800 GB (FP4 experts + FP8 attention). Usable memory is 180 GB x 0.92 per GPU,
the gpu_memory_utilization from the published recipe. A placement only counts
here if the cache gets at least 30% of what is left, because a config that fits the weights
and nothing else cannot serve a request.
Tensor parallel stays inside the NVSwitch, so no TP traffic ever crosses a NIC.
- It uses the whole node. 800 GB of weights against 1,325 GB of usable memory across eight B200s leaves roughly 525 GB for KV. One replica per HGX node, 500 across the fleet, and tensor parallel spans exactly the NVLink domain and never crosses a NIC.
- Speculation ships in the recipe. The published configs include DSpark at 7 speculative tokens and MTP at 3. Speculative decoding is not an experiment on this model, it is part of the serving configuration.
- The headline throughput is not your topology. The 150 tokens/sec/user figure is measured on GB200 NVL72, where 72 GPUs share one NVLink domain. On 8-GPU HGX nodes the model still fits, but you do not get NVL72's cross-rack cache behaviour.
What I keep coming back to
my readThe number that stays with me is not 1.6 trillion, it is 27%. A model got five times larger and simultaneously cheaper per token than the thing it replaced, which is not how this usually goes. What I would want to know before planning capacity around it is how much of that holds up outside the vendor's own harness, because the headline throughput comes from NVL72, and on eight-GPU nodes the model fits fine but the cross-replica cache story does not transfer. Fitting is the easy half.
The numbers
measured elsewhere, not here- Total
- 1.6T
- Active
- 49B
- Per user
- 150 tok/s
- Perf/watt
- 30x H200
Measured by NVIDIA on GB200 NVL72 against an H200 NVL8 baseline at similar interactivity. Different topology from an HGX B200 fleet.
Sources: NVIDIA Technical Blog · DeepSeek-V4 paper · Hugging Face: DeepSeek-V4-Pro