There is no address

Suppose the two vendors agree on layout, dtype, block size, scale placement and every other axis in that list. The bytes still have to move. This is the part that looks like plumbing and is not, because a transfer library is not neutral about what kind of machine sits at the other end. Historically one program has always owned the whole memory hierarchy it was moving data through. Disaggregation is the first time we have split that ownership across two vendors.

Every machine is a memory hierarchy

Nothing about this problem is new to accelerators. It is the oldest structure in systems programming.

A CPU has registers, then L1, then L2, then L3, then DRAM, then storage. Each level is smaller, faster and closer than the one below it, and a program that ignores the hierarchy runs an order of magnitude slower than one that respects it. Two mechanisms hide this: the hardware cache controller fetches, evicts and prefetches without being asked, and the compiler reasons about locality on your behalf, tiling loops and reordering accesses so the working set fits a level that is actually fast.

A GPU has the same structure with the automation removed. On Blackwell, an SM has registers, then tensor memory, then a configurable unified L1 and shared memory of 128 KB per SM, then a monolithic L2 of 64 to 65 MB, then HBM. Shared memory access runs on the order of 20 to 30 cycles. There is no cache controller deciding what belongs in shared memory. You write __shared__, you stage tiles into it yourself, or you use the tensor memory accelerator to move a block from global memory into shared memory asynchronously while compute proceeds.

A Cerebras wafer is the same idea taken to its limit. Roughly 900,000 processing elements in a 2D mesh, each with 48 KB of local SRAM, each accessing only its own memory at sub-nanosecond latency. Everything else is a message. PEs communicate over a circuit-switched network-on-chip, sending packets called wavelets along configured routes, where each PE's router supports a limited number of concurrent circuits (24, plus 8 reserved) called colors, which are virtual channels bound to physical routing resources. Two streams that might collide must be assigned different colors.

That machine is programmable. CSL is a Zig-inspired dataflow language in which computation is triggered by the arrival of data, and a CSL program is not just kernel code: it includes a layout file that prescribes which code runs on which PEs and how data is routed between them. The cslc compiler maps that onto the physical fabric. Placement and routing are first-class parts of the source program.

What a transfer library can actually touch

Now put a network in the middle and ask which rung of those ladders a remote peer can write into. The answer is one rung, and it is always the same rung: the level that is globally addressable, physically stable, and reachable by a device that is not the compute engine. On a CPU that is DRAM. On a GPU that is HBM. It is never L1, never shared memory, never tensor memory, never a PE's scratchpad.

NIXL, NVIDIA's inference transfer library and the thing underneath vLLM's NixlConnector and Dynamo's disaggregated path, states this in its type system. Its memory spaces are enumerated exhaustively:

// nixl/src/api/cpp/nixl_types.h
enum nixl_mem_t {DRAM_SEG, VRAM_SEG, BLK_SEG, OBJ_SEG, FILE_SEG};

Host DRAM, GPU VRAM, block device, object store, file. There is no SMEM_SEG, no TMEM_SEG, no PE_SEG, and their absence is not an oversight. Those levels are not addressable from off-chip by anything.

The unit NIXL moves is correspondingly simple:

// nixl/src/api/cpp/nixl_descriptors.h
class nixlBasicDesc {
public:
    uintptr_t addr;
    size_t len;
    uint64_t devId;
};

A pointer, a length, and a device number. Single contiguous. This is why RDMA needs registration: pages pinned so the OS cannot move them, the virtual-to-physical mapping handed to the NIC, a key returned that the remote peer presents on every access. Registration is exactly the operation that makes a region of the public rung stable enough for a foreign device to write into. It has no meaning at any other level of the hierarchy.

The seam is already there on a GPU

You cannot RDMA into shared memory. If a remote machine sends you a KV cache, it lands in HBM. Getting it from HBM into the 128 KB of shared memory where the attention kernel actually wants it is a second movement, performed by the receiving side, using its own machinery: a TMA descriptor, an async copy, a tiling strategy chosen by whoever wrote that kernel.

So even in the ordinary all-NVIDIA case, the handoff has two halves. The network gets the bytes to the public rung. The consumer's own compiler and kernel take them the rest of the way down the hierarchy. Nobody experiences this as a boundary, because both halves are written by the same people against the same layout. The producer knows the consumer will want 128-element tiles in a particular order, so it writes the cache to HBM in the order that makes the consumer's TMA descriptor cheap. That agreement is real, load-bearing, and entirely undocumented, because it never had to leave the building.

That is the seam. It has always existed. Disaggregation is what happens when the two halves stop being written by the same organisation.

The wafer removes the rung

A WSE-3 has 44 GB of SRAM, and that number invites you to picture a pool the way you picture 80 GB of HBM. It is not a pool. It is 900,000 × 48 KB, private to each PE, with no shared address space between them.

Try to fill in the descriptor. A single KV block for one layer of a mid-sized model is on the order of a hundred kilobytes: not one PE's scratchpad, two or three of them, and not adjacent in any address sense because there is no address sense. A sequence's whole cache spans thousands of PEs. There is no uintptr_t addr that names it, no size_t len over which the bytes are contiguous, and no devId resolving to something a NIC can write into. VRAM_SEG is the nearest enum value and it is wrong: this is not memory behind a controller, it is the compute substrate.

There is also nothing to pin. Registration assumes memory you can hold still while a device writes into it. On a wafer, where a value lives is part of the schedule, and the schedule is what cslc emitted from the layout file.

Why this matters for inference

The public rung is missing on a wafer. Which means the bytes have to stage: land in host DRAM or MemoryX, the off-wafer store that already streams weights in, and then get pulled onto the fabric by the wafer's own dataflow, as scheduled work. So the handoff is at least two hops, and the second is not a DMA the sender controls.

Weigh that against the reason for disaggregating at all. The whole point was latency: separate the phases so prefill stops interrupting decode and time-to-first-token improves. Now the first token waits on a network crossing, a staging buffer, and a fabric distribution, all sitting on the TTFT path in front of the metric the split existed to protect.

Disaggregation is not a plumbing problem. It is the first time the memory hierarchy has two owners, and the transfer library is where that split becomes visible.

Back to the blog