Move a number, break something.

A GPU serving requests in fixed batches has to wait for the slowest member before it can start the next batch. Continuous batching lets a finished slot pick up new work immediately. Both lanes below get the identical thirty requests and the identical six slots, so the gap is the scheduler and nothing else.

Request lengths are drawn from a fixed seed so the comparison is reproducible. Real traffic is burstier than this, which makes the gap wider, not narrower. Background: simulating continuous batching and how vLLM does it.

Weights are a fixed cost you pay once. The KV cache grows with every token and every concurrent request, and past a certain context length you are running a cache server that happens to do matrix multiplication. This finds that crossing point for a given model.

KV size is 2 x layers x kv_heads x head_dim x bytes x tokens x batch, using published config values. For the MoE entries the parameter count is total, not active, which is the number that decides whether it fits. Background: the KV cache.

Splitting a model across GPUs divides the compute by N. It does not divide the communication, and every layer pays two all-reduces. On a fast NVLink domain the curve stays near linear for a while. On PCIe or Ethernet it bends early, and past the bend each extra GPU is making things worse.

A first-order model: compute from a 12 x hidden^2 per-layer parameter estimate against 400 TFLOP/s per GPU, communication from ring all-reduce volume plus a fixed per-collective latency. It gets the shape and the location of the knee right. It will not predict your p99. Background: TP, EP and PP and measured scaling curves.

Most of these live inside the page that explains them, because a calculator is worth more next to the argument it settles than in a drawer of calculators. This is the index. Every entry shows the expression it evaluates, so you can decide whether it is answering your question before you open it.