Frontier model

GLM-5.3-Flash: Half the depth, a quarter of the cache

320B total, 18B active, natively multimodal, and a context ceiling of a million tokens that is only affordable because 34 of its 45 layers refuse to do full attention.

MIT 320B total 18B active 1M context

The architecture

how it is built

The layer stack

45 layers, against 92 in the GLM-4.5 family. Three dense MLP layers form the stem, then 11 repeating blocks of three KDA linear-attention layers followed by one sparse MLA layer with a MoE feed-forward, and a single trailing KDA layer closes it out.

Two kinds of attention

34 of the 45 layers are KDA linear attention: 64 heads at 128 head dimension, a short convolution, and O(L) cost in sequence length. The other 11 are NoPE sparse MLA layers, DeepSeek-style, 64 heads at 256 QK/V dimension, which use a lightning indexer to select the top 2,048 tokens. Full attention runs on roughly one layer in four.

Routing

Every MoE layer routes each token to 8 of 288 experts plus one shared expert, top-K softmax with load balancing, 2,048 intermediate dimension per expert. 320B parameters sit in memory; 18B of them do work on any given token.

IndexPool key compression

The indexer's key vectors are pooled by weighted average rather than stored per token. This is the line I would underline: it cuts the KV cache 4.44x and attention compute 3.01x against GLM-5.3, and it is the only reason a million-token context is something you can pay for.

Native FP8

The released checkpoint is FP8, not a post-hoc conversion of a BF16 original. There is no quantization quality tax for me to argue about, and the BF16 path at TP=8 costs you 772 GB to buy nothing.

It is also multimodal, which I nearly missed

This is the first natively multimodal model in the GLM-5 line. Text and image on the model card, video on the API side. That changes the serving problem more than the parameter count does, because a vision encoder is a compute-bound stage bolted onto the front of a prefill that was already compute-bound.

mHC, borrowed

It adopts Manifold-Constrained Hyper-Connections, the residual-stream reformulation DeepSeek published. Seeing one lab's architectural result land in another lab's checkpoint this quickly is the part of the open-weights era I find most interesting.

Serving it

the playbook

Hopper or newer, so every B200 in the fleet qualifies. AMD MI300X, MI325X and MI355X are supported through gfx950.

On a 4,000-GPU HGX B200 fleet

500 nodes, eight B200s each, 180 GB per GPU

TP=4smallest placement that leaves a real cache budget

331 GBKV cache per replica, 50% of usable memory

2replicas per HGX node

1,000replicas across the fleet

Weights are 331 GB (FP8, native). Usable memory is 180 GB x 0.92 per GPU, the gpu_memory_utilization from the published recipe. A placement only counts here if the cache gets at least 30% of what is left, because a config that fits the weights and nothing else cannot serve a request. Tensor parallel stays inside the NVSwitch, so no TP traffic ever crosses a NIC.

Where the fleet math lands. 331 GB of FP8 weights against 165.6 GB of usable memory per B200 means four GPUs is the smallest placement that leaves a real cache budget. That is half a node, so two independent replicas per HGX B200 and no tensor parallel traffic ever leaving the NVSwitch.
What the cache buys. At TP=4 the published figure is a 14.92M-token KV pool at 128K context, which is roughly 113x concurrency on one replica. The derived headroom below agrees with that order of magnitude.
reasoning_effort changes what you are measuring. Like the flagship, this takes a thinking-budget setting. A throughput number captured at one effort level and one captured at another are not comparable, and most published figures do not say which they used.

What I would actually do with it

my read

This is the one I would reach for first, and the reason is the licence more than the architecture. MIT means I can put it behind a product without a lawyer in the loop, and at TP=4 it takes half a node, which means two independent replicas per box and no tensor-parallel traffic on a NIC. The thing I would want to measure before believing any of it is what the IndexPool compression does to quality at the long end of that million-token window, because a 4.44x cache saving is exactly the kind of number that is true on average and painful at the tail.

The numbers

measured elsewhere, not here

Output: 47.6 tok/s
TTFT: 1.57 s
Input: $0.15 / 1M
Output: $0.50 / 1M

Measured on Z.ai's hosted API by Artificial Analysis, not on this hardware. Treat it as the model's shape, not as your number.

Sources: Z.AI developer docs · GLM-5.3-Flash architecture deep dive · Artificial Analysis