This is the manual I wish I'd had. It's organized as a roadmap: where to start, what to do next, and how to verify you actually know it. Each topic is a deep dive with its own page, grounded in real numbers and public sources.
The roadmap
Where to start
- Learn the fundamentals from the journey posts: attention, KV cache, roofline, kernels. These are the language everything else is spoken in.
- Pick a GPU and read its deep dive. Understand what changed from its predecessor, because that's what actually matters.
- Understand how GPUs talk: NVLink, NVSwitch, RDMA, GPUDirect Storage. This is the layer that makes or breaks a cluster.
- See the cluster at scale: 4000 B200s, disaggregated, with a Mooncake-style cache pool.
- Build your own engine in Rust, the way I did. This is the master class.
- Look ahead to Vera Rubin, so you're not caught flat-footed.
The topics
GPUs
- A100: the Ampere workhorse. TF32, MIG, 80GB HBM2e.
- T4: the tiny Turing that won the edge. 16GB, FP16, no NVLink.
- L4: Ada's answer to T4. 24GB, AV1, RTX IO.
- L40S: Ada for AI. 48GB, FP8, the no-NVLink tradeoff.
- H100: Hopper. FP8, Transformer Engine, NVLink 4.
- H200: Hopper with 141GB HBM3e. The memory play.
- B200: Blackwell. FP4, 2 dies, NVLink 5, the 4000-GPU story.
Playground & workstation
- RTX PRO 6000: Blackwell for the desk. 96GB GDDR7, FP4, the workstation B200.
- DGX Spark: GB10 Grace Blackwell. 128GB unified, NVFP4, vLLM. A data center in a box.
- Playground machines: Apple Silicon and RTX, measured side by side. What actually runs on your desk.
Next gen
- Vera Rubin: what's coming after Blackwell.
Networking
- RDMA and GPUDirect Storage: moving data without the CPU.
- NVLink and NVSwitch: the GPU-to-GPU fabric.
- EFA and InfiniBand: the cluster-wide network.
- NVLink Sentinel: watching the fabric's health.
Custom engine
- Rust inference engine: built and running on B200.
How to use this manual
Each topic page stands alone: read it when you need it. Each is grounded in real numbers and public sources, so you can trace every claim. Start at the roadmap, or jump straight to whatever's biting you.
The rule
If you can't verify it, you don't know it. Every page in this manual is built on numbers and sources you can check.