Now · updated August 2026

What I'm working on right now.

A snapshot, not a resume. This changes whenever the work changes.

Disaggregated serving at fleet scale. Splitting prefill and decode across the GPU pool so the KV cache handoff is invisible to users. The tradeoff between cache locality and utilization is the thing I keep coming back to.
Prefix caching for agent workloads. A thousand agents sharing a system prompt means cache hits are free money. Building the eviction policy and the shared cache pool that makes the hit rate honest.
Speculative decoding in production. The paper numbers are beautiful; the reality of draft-model variance is not. Measuring acceptance rate under real batch interference, not just in a notebook.

Dynamo's disaggregation design notes - how NVIDIA thinks about the prefill/decode split in the serving layer.
Mooncake's cache pool paper - the KVCache-centric architecture that separates cache from compute.
DeepSeek V4 Pro technical report - the 1.7T model, MIT licensed, and what it means for serving economics.
KV-stationary sparse attention on Blackwell - keeping the cache resident and the compute moving.

What "reliable token" means. Not average cost, but the 99th percentile under bursty, correlated load. When agents arrive in waves, the autoscaler has to predict, not react.
Why every model release changes the optimal serving config. The answer to "disaggregate or not" is different for a 1.7T MoE than a 70B dense. Keeping the playbook honest.
Observability that explains, not just displays. Correlating TTFT spikes with scheduler decisions, cache evictions, and network jitter. The dashboard is the hypothesis, not the conclusion.