Orpheus is the 2025 open-source TTS from Canopy Labs that made the speech community do a double take. Not because it's a brand-new architecture, but because it's the same architecture as an LLM, pointed at audio instead of text. If you can serve an LLM, you can serve Orpheus.
The trick: audio as tokens
The hard part of TTS is that audio is continuous, and transformers want discrete tokens. Orpheus solves this with Residual Vector Quantization (RVQ): a codec (based on a neural audio codec) quantizes the audio into a hierarchy of discrete codes. The first level captures the coarse structure, each subsequent level adds finer detail.
So speech becomes a sequence of codebook indices, which are just tokens. And once audio is tokens, a language model can generate it.
The backbone: Llama-3.2-3B
Orpheus is built on Llama-3.2-3B, fine-tuned to predict audio codes instead of (or in addition to) text. The model takes text plus a speaker tag and a style tag, and generates the RVQ codes that decode into speech.
This is the genius of it: you inherit all the LLM serving infrastructure. vLLM, SGLang, TensorRT-LLM, the whole ecosystem, can serve Orpheus with minor changes. The KV cache, the continuous batching, the quantization, all of it applies.
Streaming: the latency game
TTS has a hard real-time requirement: if the model takes longer to generate a second of audio than a second of wall-clock time, it's not real-time. Orpheus streams, generating audio codes in chunks and decoding them as it goes, so the first audio arrives long before the full utterance is done.
The serving implication: TTFT matters for TTS too. The time to first audio is the user-perceived latency, and it's dominated by the first few RVQ levels, which you can generate and decode before the rest are done.
Orpheus vs the old guard
Older TTS like XTTS and Bark use different approaches: XTTS uses a VITS-style flow + a vocoder, Bark uses a hierarchical autoregressive model with a separate audio codec. Orpheus collapses this into a single LLM, which means it inherits the LLM ecosystem's serving maturity. That's the real win.
Orpheus is a translator in reverse: it reads text and writes audio, one "word" (RVQ code) at a time, and the codec turns those words back into sound.
The production math
Orpheus generates roughly 7 tokens per audio frame (coarse + mid + fine levels), which works out to about 87 tokens per second of audio. A 10-second utterance is ~870 tokens, so the KV cache grows fast on long generations. But the win is that it's a standard Llama-3.2 architecture: vLLM, TensorRT-LLM, or any LLM stack serves it with just a vocabulary expansion. A single A10G/L4 handles the 3B model, and the SNAC decoder runs on CPU in a separate thread, so the GPU is never blocked on audio reconstruction.
Sources
- Orpheus TTS repo and model card: Canopy Labs' Llama-3.2-3B based TTS.
- SNAC: Multi-Scale Neural Audio Codec: the RVQ codec Orpheus generates tokens for.
- XTTS and Bark: the prior generation, for comparison.
The takeaway
TTS stopped being a special snowflake the day it became a language model. Orpheus is the proof: serve it like an LLM and you get frontier-quality speech for the cost of a 3B model.