Kimi K3: the largest open-weight model anyone has shipped
2.8 trillion parameters, 896 experts with 16 firing per token, a million tokens of context and a native vision encoder. The checkpoint is 1.56 TB, which is the number that decides everything about serving it.
The architecture
how it is built896 experts, 16 firing
The sparsest routing on this site by a wide margin. 2.8 trillion parameters resident, roughly 104 billion doing work per token. Note that a widely repeated figure of 32B active is wrong; the expert configuration gives 104B, and it is worth checking that number wherever you see it quoted.
Trained in MXFP4, not converted to it
The quantisation is in the training, so the released weights are the intended artifact rather than a lossy copy of a BF16 original. Same instinct as GLM-5.3-Flash shipping native FP8, and for the same reason: at this scale a post-hoc conversion is a quality argument nobody wants to have.
Native vision
A vision encoder in the base model rather than bolted on, which puts it in the same category as GLM-5.3-Flash and MiniMax M3. Multimodality has stopped being a variant and started being the default.
A million tokens
Same ceiling as most of the frontier now. The interesting question at 2.8T is not whether the window is long but whether the KV cache at that window leaves you any GPUs to serve with.
Serving it
the playbookThis is a multi-node model. At 1.56 TB the weights alone exceed a single HGX B200 node's 1.44 TB of HBM before any cache, so tensor or expert parallelism has to span nodes and the collectives leave the NVSwitch.
On a 4,000-GPU HGX B200 fleet
500 nodes, eight B200s eachThis one has no weight footprint of its own. It runs beside whichever target model you are already serving, adds a small resident drafter, and spends spare compute to buy tokens. On a fleet that is memory-bound during decode, which is every fleet, that is close to free.
- It does not fit on one node. 1,560 GB of weights against 1,325 GB of usable memory across eight B200s. Two nodes is the minimum and that is before the cache, which means every all-reduce crosses a 400 Gb/s NIC rather than 1.8 TB/s of NVLink.
- The licence has thresholds. Free for commercial use with attribution, with additional conditions above roughly twenty million dollars of annual revenue on model-as-a-service, or a hundred million monthly users. Not MIT, and not a detail to discover after you have built on it.
- Sparsity is the whole design. 16 of 896 experts is under two percent of the model firing per token. That is what makes 2.8T tractable at all, and it also makes expert placement across nodes the dominant performance question rather than an afterthought.
What 2.8 trillion actually means for a fleet
my readThe number that matters to me is not 2.8T, it is 1.56 TB. That is the MXFP4 checkpoint, and it does not fit on an eight-GPU node no matter how the memory is arranged, so this is the first model on the site that forces multi-node tensor parallelism and therefore forces the all-reduce out onto the NIC. Everything I have written about the 36x gap between NVLink and the fabric becomes load-bearing at exactly this size. The other thing I would read carefully is the licence, which is free commercially with attribution but has real conditions above twenty million in revenue, and that is a sentence for a lawyer rather than an engineer.
The numbers
measured elsewhere, not here- Total
- 2.8T
- Active
- 104B
- Experts
- 16 of 896
- Checkpoint
- 1.56 TB
Architecture and checkpoint figures from the model release. Nothing here was measured on my hardware, and at this size I have not been able to run it.
Sources: Hugging Face: moonshotai/Kimi-K3 · Kimi K3 specs and access