Skand Tandon
Draft model

DFlash 2: Block diffusion, wearing a drafter's coat

Not a model you host. A speculative-decoding drafter that predicts a whole block of tokens in one forward pass, and gets 2.7x to 4.6x at what the rejection sampler guarantees is the target's own distribution.

open weights drafter total n/a active target's context

The architecture

how it is built

Parallel block drafting

Every other entry here generates one token per forward pass. DFlash 2 predicts every position in a block independently, in a single pass. Block sizes run from 4 to 16; SGLang defaults to 8.

Sixteen candidates per position

The drafter keeps its top 16 candidates at each slot. The correct token is in that set 99.5% of the time at position 1. By position 7 the recall has fallen to 87.8%, and that decay is the problem the rest of the architecture exists to solve.

A path selector

Low-rank bilinear attention scores adjacent candidate pairs and walks one coherent sequence through the lattice. It costs 2.0M parameters and 0.6% latency, and it buys 0.34 tokens of mean acceptance length.

Suffix-decay convolution

A two-tap dynamic depthwise convolution repairs the accuracy fall-off toward the end of the block, where attention has started neglecting local dependencies. 16.5M parameters, about 3% of the drafter, 0.7% latency.

Lossless, with conditions I did not used to state

The rejection sampler accepts a draft token with probability min(1, p/q) and resamples from the residual on rejection, which preserves the target distribution exactly. That guarantee is real, and it is narrower than I once wrote here. It holds only under strict verification at threshold 1.0 and non-anticipating scheduling, and it is a statement about distributions rather than about bytes: even greedy decoding against the same model can diverge from the unspeculated path on floating-point ordering alone. The distribution is preserved. The exact token sequence is not promised.

Serving it

the playbook

Runs natively in SGLang, vLLM, llama.cpp, Ollama and oMLX. Released drafters target Qwen3.8-27B and Meta Muse Glimmer.

On a 4,000-GPU HGX B200 fleet

500 nodes, eight B200s each

This one has no weight footprint of its own. It runs beside whichever target model you are already serving, adds a small resident drafter, and spends spare compute to buy tokens. On a fleet that is memory-bound during decode, which is every fleet, that is close to free.

Why this one interests me most

my read

Everything else on this site is a model you host. This is a scheduling trick that gives you a 3x and takes nothing but a little memory, and the word doing the work is lossless. I have learned to distrust speedup claims that quietly move the quality bar, and I have since had to distrust the word lossless as well. The rejection sampler preserves the distribution the target would have produced, which is a real guarantee and a weaker one than the byte-identical output I originally took it to mean. What I would actually verify is not that the tokens match but that acceptance length holds up on my own traffic, because that is where it quietly does not. On a fleet that is memory-bound during decode, spare compute is the cheapest thing I own, and this spends it.

The numbers

measured elsewhere, not here
Qwen3.8-27B
2.7-3.4x
Muse Glimmer
3.1-4.6x
Acceptance
+21% vs DFlash
Quality
distribution-preserving

Speedups measured by Inco AI against plain autoregressive decoding for each target model. Note that published acceptance lengths for this method disagree depending on target and benchmark, from roughly 3.8 to 4.8, which is the clearest available evidence that acceptance length is a property of the traffic rather than of the method.

Sources: Inco AI: DFlash 2 · Hugging Face: Qwen3.8-27B-DFlash2

Read alongside this

where it comes up elsewhere