DFlash 2: Block diffusion, wearing a drafter's coat
Not a model you host. A speculative-decoding drafter that predicts a whole block of tokens in one forward pass, and gets 2.7x to 4.6x losslessly.
The architecture
how it is builtParallel block drafting
Every other entry here generates one token per forward pass. DFlash 2 predicts every position in a block independently, in a single pass. Block sizes run from 4 to 16; SGLang defaults to 8.
Sixteen candidates per position
The drafter keeps its top 16 candidates at each slot. The correct token is in that set 99.5% of the time at position 1. By position 7 the recall has fallen to 87.8%, and that decay is the problem the rest of the architecture exists to solve.
A path selector
Low-rank bilinear attention scores adjacent candidate pairs and walks one coherent sequence through the lattice. It costs 2.0M parameters and 0.6% latency, and it buys 0.34 tokens of mean acceptance length.
Suffix-decay convolution
A two-tap dynamic depthwise convolution repairs the accuracy fall-off toward the end of the block, where attention has started neglecting local dependencies. 16.5M parameters, about 3% of the drafter, 0.7% latency.
Lossless, and that word is load-bearing
Greedy output matches the target model exactly. Sampled output preserves the target's distribution. The speedup is not a quality trade, which is what separates this from most things that claim a 3x.
Serving it
the playbookRuns natively in SGLang, vLLM, llama.cpp, Ollama and oMLX. Released drafters target Qwen3.8-27B and Meta Muse Glimmer.
On a 4,000-GPU HGX B200 fleet
500 nodes, eight B200s eachThis one has no weight footprint of its own. It runs beside whichever target model you are already serving, adds a small resident drafter, and spends spare compute to buy tokens. On a fleet that is memory-bound during decode, which is every fleet, that is close to free.
- It changes the fleet arithmetic, not the memory budget. A drafter adds a small resident model beside the target and turns spare compute into tokens. On a fleet that is memory-bound during decode, which is every fleet, that is close to free throughput.
- Against the alternatives. 21% better mean acceptance length than the original DFlash, and more than a full token better than DSpark.
Why this one interests me most
my readEverything else on this site is a model you host. This is a scheduling trick that gives you a 3x and takes nothing but a little memory, and the word doing the work is lossless. I have learned to distrust speedup claims that quietly move the quality bar, so the fact that greedy output matches the target exactly is the part I would verify first and the part that would make me adopt it. On a fleet that is memory-bound during decode, spare compute is the cheapest thing I own, and this spends it.
The numbers
measured elsewhere, not here- Qwen3.8-27B
- 2.7-3.4x
- Muse Glimmer
- 3.1-4.6x
- Acceptance
- +21% vs DFlash
- Quality
- lossless
Speedups measured by Inco AI against plain autoregressive decoding for each target model.
Sources: Inco AI: DFlash 2 · Hugging Face: Qwen3.8-27B-DFlash2