Now

What I'm working on right now.

A snapshot, not a resume. This changes whenever the work changes.

Hosting the open frontier. GLM-5.3-Flash at TP=4 and DeepSeek V4 Flash on the published vLLM recipe, against our own traffic rather than a benchmark harness. The interesting question is not which model wins a leaderboard, it is which one holds its p99 when the batch is real.
Speculative decoding as default, not experiment. DFlash 2 and MTP both ship in serving configs now. Measuring what the acceptance rate does under batch interference, because that is the number the papers never quote.
Disaggregated serving at fleet scale. Splitting prefill and decode across the GPU pool so the KV cache handoff is invisible to users. The tradeoff between cache locality and utilization is the thing I keep coming back to.
Prefix caching for agent workloads. A thousand agents sharing a system prompt means cache hits are free money. Building the eviction policy and the shared cache pool that makes the hit rate honest.
Speculative decoding in production. The paper numbers are beautiful; the reality of draft-model variance is not. Measuring acceptance rate under real batch interference, not just in a notebook.

The GLM-5.3-Flash architecture: 34 of 45 layers on linear attention, and the IndexPool trick that cuts the KV cache 4.44x. My notes.
DeepSeek V4 technical report: 1.6T total, 49B active, 27% of V3.2's per-token FLOPs. My notes.
DFlash 2: block diffusion as a drafter, and the suffix-decay problem it had to solve. My notes.
Mooncake's cache pool paper: the KVCache-centric architecture that separates cache from compute.

What "reliable token" means. Not average cost, but the 99th percentile under bursty, correlated load. When agents arrive in waves, the autoscaler has to predict, not react.
Why every model release changes the optimal serving config. The answer to "disaggregate or not" is different for a 1.6T MoE than a 70B dense. Keeping the playbook honest.
Observability that explains, not just displays. Correlating TTFT spikes with scheduler decisions, cache evictions, and network jitter. The dashboard is the hypothesis, not the conclusion.

Built the open-frontier page: what GLM-5.3, DeepSeek V4 and DFlash 2 actually cost to serve on Blackwell, every number sourced.
Corrected DeepSeek V4 Pro from 1.7T to 1.6T across the site after checking the model card against the paper.
Shipped the Rust inference engine to B200, measured it, wrote it up.
Collapsed ten nav items into seven and put all 166 pieces behind one search box.