Not everyone has a 4000-GPU cluster. The machines on this page are the ones you can actually own: Apple Silicon with unified memory, and NVIDIA RTX cards with real tensor cores. They're where you test quantization, run a small vLLM, and prototype kernels. All numbers below are measured by local.ai, not spec-sheet claims.
Apple Silicon: unified memory is the superpower
Apple's M-series chips have one structural advantage for local inference: the GPU and CPU share one pool of memory, so a model's weights live in the same memory the GPU reads. No PCIe transfer, no copy. The downside is bandwidth: unified memory tops out around 500GB/s, far below an H100's 3.35TB/s. So Apple machines are memory-bound for decode, exactly like a data center GPU, just at a smaller scale.
MacBook Pro M5 Max (128GB, 40-core GPU, $6,699)
- LFM2.5 8B A1B (Q8_0): 200 tok/s, τ² 94.8%
- Gemma 4 E2B It (UD-Q4_K_XL): 152 tok/s
- Laguna XS 2.1 (Q4_K_M): 145 tok/s, GAIA 64.0%
- Gemma 4 E2B It (UD-Q8_K_XL): 121 tok/s
- Agents A1 (q4km): 113 tok/s, τ² 95.0%
- Holo 3.1 35B A3B (q4_k_m): 113 tok/s, τ² 86.2%
- Ornith 1.0 35B (Q4_K_M): 113 tok/s, τ² 96.2%
- Qwen3.6 35B A3B (iq1m): 112 tok/s, τ² 98.6%
The 128GB is the point: a 70B at 4-bit fits with room for context. This is the "I want to run real models on a laptop" machine.
MacBook Pro M5 Pro (64GB, 20-core GPU, $3,699)
- LFM2.5 8B A1B (Q8_0): 122 tok/s
- Gemma 4 E2B It (UD-Q4_K_XL): 104 tok/s
- Laguna XS 2.1 (Q4_K_M): 78 tok/s, GAIA 64.0%
- Gemma 4 E2B It (UD-Q8_K_XL): 75 tok/s
- Qwen3.6 35B A3B (iq1m): 72 tok/s, τ² 98.6%
- Ornith 1.0 35B (Q4_K_M): 71 tok/s, τ² 96.2%
The 64GB is the sweet spot for a 35B A3B MoE at 4-bit (~20GB) with long context. Interactive, quiet, and it fits in a backpack.
MacBook Pro M4 Max (36GB, 32-core GPU, $3,229)
- LFM2.5 8B A1B (Q8_0): 143 tok/s
- Gemma 4 E2B It (UD-Q4_K_XL): 111 tok/s
- Qwen3.6 35B A3B (iq1m): 74 tok/s, τ² 98.6%
- Gemma 4 E4B It (Q4_K_M): 74 tok/s
- Qwen3.6 35B A3B (UD-Q2_K_XL): 70 tok/s
The 36GB is a real constraint: a 35B A3B at 4-bit fits, a 70B doesn't. This is the "small models, fast iteration" machine.
NVIDIA RTX: real tensor cores, no unified memory
RTX cards have the opposite tradeoff: real NVIDIA tensor cores and CUDA, but discrete VRAM (24-96GB) that you can't grow. The upside is that everything you learn transfers directly to a data center GPU. The downside is VRAM is the wall.
RTX 5090 (32GB GDDR7, $4,799)
- LFM2.5 8B A1B (Q8_0): 509 tok/s, τ² 94.8%
- Nemotron 3 Nano 30B A3B (nvfp4): 416 tok/s, GAIA 48.3%
- LFM2.5 8B A1B (default, vLLM): 398 tok/s
- Gemma 4 E2B It (UD-Q4_K_XL): 337 tok/s
- Gpt Oss 20B (mxfp4): 286 tok/s
- Qwen3.6 35B A3B (iq1m): 268 tok/s, τ² 98.6%
- Qwen3.6 35B A3B (iq2xxs): 267 tok/s, τ² 99.4%
- Gemma 4 E2B It (fp8): 263 tok/s
The 5090 is the fastest consumer card for inference, period. 509 tok/s on an 8B is absurd for a $4,799 card. The 32GB limits you to a 35B A3B at 4-bit, but for that class it's unbeatable.
RTX 4090 (24GB GDDR6X, $3,449)
- LFM2.5 8B A1B (Q8_0): 316 tok/s
- Gemma 4 E2B It (UD-Q4_K_XL): 227 tok/s
- Gpt Oss 20B (mxfp4): 192 tok/s
- Gemma 4 E2B It (fp8): 189 tok/s
- Qwen3.5 9B Base (Q4_K_M): 128 tok/s
- Gemma 4 12B It (q4km): 90 tok/s
The 4090 is the value king: 80% of a 5090's speed at 70% of the price, and a huge used market. 24GB is tight but fine for 8-20B models.
The takeaway
For a personal playground in 2026, the choice is really between two philosophies:
- Apple Silicon (M5 Max): more memory (128GB), less speed, unified memory means zero-copy weights. Best for running big models slowly.
- RTX 5090: less memory (32GB), way more speed, real CUDA. Best for fast iteration on small models and format parity with production.
- DGX Spark: the middle path. 128GB unified, NVFP4, vLLM, and the same weight format as a B200 cluster. If you can afford it, it's the best of both.
All numbers from local.ai's independent benchmark runs. The site is live, so check it for the latest data.
Sources
- local.ai hardware benchmarks: all measured tok/s on this page.
- Apple MacBook Pro: official M5/M4 specs.
- NVIDIA RTX 50 series: official specs.