Time to first token is the metric I have argued for hardest. It is the one that correlates with whether an interface feels alive, it is cheap to measure, and it is comparable across engines. Reasoning models have quietly made it dishonest, and I do not think the tooling has caught up.
The problem in one sentence
A reasoning model emits its chain of thought before its answer. Those are real tokens: generated one at a time, occupying the decode loop, filling the KV cache, and billed. They are also, in most product surfaces, hidden. So TTFT fires on a token the user will never read, and the number you report describes a moment that does not exist from the outside.
The metric that survives this is time to first visible token. Same clock start, but it stops when something the user is actually shown appears. On a request that thinks for a few thousand tokens the two can differ by an order of magnitude, and only one of them predicts whether the interface feels broken.
What else this breaks
Once thinking tokens are in the picture, several things I had treated as stable stop being stable.
- The bill moved to the output side. The same question can cost tens of tokens or thousands depending on how long the model deliberates, with no change in the prompt. Cost per request stops being predictable from the input, which is what most capacity models assume.
- The cache grows in a way you did not plan for. A long reasoning chain is a long sequence, and it occupies KV like any other. The memory arithmetic on the rest of this site still holds; the input to it is now a number you cannot see in advance.
- Batch starvation. One request thinking hard holds memory that would otherwise serve a large number of ordinary queries. The scheduler cannot tell them apart at admission time, because at admission they look identical.
- Effort is a dial nobody reports. The current generation of models take a thinking-budget setting. Two throughput numbers captured at different settings are not comparable, and almost no published figure says which was used.
What I would measure instead
I would keep TTFT, because it still tells you whether prefill and the scheduler are healthy, and it is the right metric for a non-reasoning deployment. But I would report it alongside three others:
- TTFVT, because it is what the user experiences.
- Thinking tokens per request, as a distribution rather than a mean. The tail is the capacity problem.
- The ratio of thinking to visible output. This is the one I would put on a dashboard. If it moves after a model or prompt change, the cost per answered question moved with it, and nothing else on the dashboard would have told you.
The fixes people reach for are the familiar ones pointed at a new target: speculative decoding with a draft distilled on reasoning traces, and cache compression that evicts thinking tokens more aggressively than answer tokens on the grounds that they matter less once the conclusion is reached. Both are reasonable. Neither removes the measurement problem, which is that the number on the dashboard stopped describing the thing you care about.
A metric that no longer measures the user's experience is worse than no metric, because you will keep trusting it.
Sources
- How I would measure it: the harness this belongs in, and why an interactivity target has to be pinned before any throughput number means anything.
- GLM-5.3 and GLM-5.3-Flash: both expose a thinking-budget setting, and most published comparisons do not say which level they used.