Runtime

Embeddings: from integers to vectors

The lookup table that turns token IDs into the vectors the transformer actually thinks in. Smaller than you think, more important than you'd guess.

The last post ended with token IDs. Now: how those integers become vectors, and why the embedding table is one of the most underrated parts of a model.

The lookup

An embedding table is a matrix of shape (vocab_size, hidden_dim). For Llama 3 8B, that's (128256, 4096), about 2GB in FP16. To embed a token, you just look up its row. Token 42 becomes row 42.

That's it. No computation, no neural magic. A table lookup. The "learning" happened during training, when the model figured out which tokens should be near each other in vector space.

Why vectors

The transformer operates on vectors, not integers. Every layer is matrix math on the hidden state, and the hidden state is a sequence of vectors. Embeddings are the bridge between the discrete world of tokens and the continuous world of linear algebra.

The property that matters: similar tokens have similar vectors. "cat" and "kitten" land near each other. That's what lets the attention mechanism find relationships, and what makes the whole thing more than a lookup table.

Mental model

The embedding table is a dictionary where the definitions are vectors. "king" minus "man" plus "woman" equals "queen", and that arithmetic only works because the definitions live in a space where meaning is direction.

The inference angle

For inference, the embedding lookup is one of the cheapest operations in the whole forward pass. It's a single gather, not a matmul. But it's also one of the first memory touches, so it matters for cold-cache latency.

And it's a place where quantization shows up: the embedding table is often kept at higher precision than the rest of the model, because it's so sensitive. One more reason the table deserves respect.

The takeaway

An embedding is a token that learned where it lives. The table is the map, and the map is smaller than you think.

Next: attention, the thing everything hangs on.