Anatomy of a CUDA binary

When you compile a CUDA kernel, the final artifact is a cubin, a CUDA binary. It is a standard ELF64 file with NVIDIA-specific sections that encode everything the CUDA driver needs to load and launch a kernel: the machine code, the parameter layout, register allocation metadata, and a collection of attributes that have no public documentation.

A note on methodology: everything here is based on analysis of cubins produced by ptxas and validated on real hardware. NVIDIA does not publish a specification for the cubin section layout, the .nv.info EIATTR encoding, or the constant bank parameter conventions. The structures and semantics are reverse-engineered from compiled binaries.

The ELF container

A cubin is an ELF64 executable. The header identifies it:

e_ident[EI_OSABI]    = 0x41       (CUDA ABI, not the older 0x33)
e_ident[EI_ABIVERSION] = 8
e_type               = ET_EXEC    (loadable, not relocatable)
e_machine            = EM_CUDA    (0xBE)
e_flags              = 0x06006402 (for sm_100, 64-bit addressing)

The e_flags field encodes the SM architecture in bits 8-15. For sm_100 (B200), that is 0x64 = 100 decimal. Bit 1 marks 64-bit addressing. Bits 24-26 carry a format version that the driver checks.

Older cubins used ELFOSABI_CUDA = 0x33 with ABI version 0. The CUDA 13 driver rejects these: cuModuleGetFunction returns CUDA_ERROR_NOT_SUPPORTED. If you are emitting cubins from scratch, the ABI version matters.

Section layout

A single-kernel cubin contains roughly twelve sections. Using a minimal kernel, an integer add written in LLVM IR:

define i32 @add(i32 %a, i32 %b) {
  %c = add i32 %a, %b
  ret i32 %c
}

After compiling to a cubin, the sections are:

.text.add (SHT_PROGBITS): SASS machine code, 128-bit instructions.
.nv.constant0.add (SHT_PROGBITS): constant bank 0, the parameter region.
.nv.info (SHT_LOPROC): module-level metadata (EIATTR stream).
.nv.info.add (SHT_LOPROC): per-kernel metadata (EIATTR stream).
.note.nv.cuver (SHT_NOTE): CUDA version note.
.note.nv.tkinfo (SHT_NOTE): toolkit release note.
.nv.compat: SM compatibility descriptor.
.nv.callgraph: intra-module call graph.
.symtab, .strtab, .shstrtab: standard ELF bookkeeping.

This is the minimal set. A real cubin produced by ptxas for a production kernel is larger, typically ~22 sections, 8 symbols, and 5 program headers. One notable addition is .nv.shared.reserved.0, a SHT_NOBITS section (64 bytes) that reserves shared memory space. Other additions include .nv.shared. (shared memory allocation), .nv.global (global variable metadata), .nv.constant2. (compiler-generated constant data), .debug_* sections (DWARF debug info), and .rel.* relocation sections.

.text.: the machine code

The .text.add section contains the SASS instructions. Each instruction is a 128-bit (16-byte) word. The section flags are SHF_ALLOC | SHF_EXECINSTR: SHF_ALLOC means the section occupies memory at runtime (the driver must load it onto the GPU), and SHF_EXECINSTR marks it as containing executable machine instructions. The section is aligned to 128 bytes.

Bits   0..104:  instruction body (opcode, operands, modifiers)
Bits 105..121:  scheduling control (stall, yield, barriers)
Bits 122..125:  operand reuse flags

The kernel's entry symbol is a STT_FUNC in the symtab with st_other = 0x10 (STO_CUDA_ENTRY), pointing at the start of this section. The driver uses STO_CUDA_ENTRY to distinguish kernel entry points from device functions.

.nv.constant0.: the constant bank

Kernel parameters are passed through constant bank 0. The section .nv.constant0.add is a SHT_PROGBITS section sized to cover the parameter region. The param_base is set by the CUDA toolkit, not a fixed ISA constant. The values observed on current toolkits:

sm_70 to sm_89 (Volta through Ampere): param_base = 0x160 (352 bytes)
sm_90 (Hopper): param_base = 0x210 (528 bytes)
sm_100 to sm_12x (Blackwell): param_base = 0x380 (896 bytes)

The first param_base bytes are reserved for driver-managed metadata (grid dimensions, block dimensions, shared memory size, etc.). User-specified kernel parameters are laid out contiguously starting at param_base, each aligned to its ABI alignment. For a kernel add(int a, int b), the two 4-byte parameters occupy offsets 0x0 and 0x4 relative to param_base. The code reads parameters with LDC (load constant) instructions:

LDC R0, c[0x0][param_base + 0x0]   // load 'a'
LDC R1, c[0x0][param_base + 0x4]   // load 'b'

The .nv.info metadata, the constant bank section, and the LDC instructions must all agree on the base offset. If they disagree, the driver copies launch arguments to one offset and the kernel reads from another: wrong results, no crash, no diagnostic.

.nv.info: the EIATTR metadata format

The .nv.info sections are the most opaque part of a cubin. They use SHT_LOPROC (0x70000000), a processor-specific section type. The content is a flat stream of EIATTR (ELF Info ATTRibute) entries. There is no public documentation for this format.

Each EIATTR entry has a fixed 4-byte header: byte 0 is the format (EIFMT), byte 1 is the attribute code (EIATTR), bytes 2-3 are the value or size depending on format. This is a TLV (type-length-value) scheme, except the "length" is implicit for formats 1-3 (always 4 bytes total) and explicit only for format 4.

The module-level .nv.info section contains attributes keyed by symbol index. For each kernel it emits three entries: REGCOUNT (0x2f), the number of general-purpose registers the kernel uses; FRAME_SIZE (0x11), the stack frame size in bytes; and MIN_STACK_SIZE (0x12), the minimum stack size. The driver uses regcount to compute occupancy, how many thread blocks can run concurrently on one SM. Over-reporting wastes occupancy. Under-reporting causes the hardware to clobber live registers.

The per-kernel section carries the kernel's ABI contract with the driver. The entries, in order:

SW_INFO (0x37): software info, self-referential (contains its own byte size).
PARAM_CBANK (0x0a): constant bank descriptor. Tells the driver which symbol table entry identifies the constant bank section, how large the parameter region is, and where it starts within the bank.
CBANK_PARAM_SIZE (0x19): the parameter region size as a 16-bit value. Redundant with the high half of PARAM_CBANK, but ptxas always emits both.
KPARAM_INFO (0x17): one entry per kernel parameter, emitted in reverse ordinal order. Each carries the index, ordinal, offset, and a packed field encoding the parameter size and the parameter space/log-alignment.
MAXREG_COUNT (0x1b): the maximum register count hint (usually 0xff = no limit).
EXIT_INSTR_OFFSETS (0x1c): the byte offsets of every EXIT instruction in the .text section. The driver likely uses these for pre-emption, knowing where a kernel can cleanly yield control.

Mental model

A cubin is a self-describing artifact: the machine code, the ABI contract, and the driver compatibility notes all live in one ELF file. The .nv.info metadata is the kernel's ID card, and the driver trusts it blindly. Get the register count wrong and the hardware clobbers live registers.

Putting it together

The complete flow from source to loaded kernel: .cu source → ptxas/llc → cubin (ELF64) → cuModuleLoad → driver reads sections → kernel launches. Every section has a job, and the driver's trust in the metadata is total.

For inference engineers, this matters because the cubin is the final artifact your kernels become. Understanding its structure is the difference between debugging a kernel launch failure by reading the ELF and guessing. And when you are building from-scratch emitters or custom tooling, the ABI version, the param_base convention, and the EIATTR format are the load-bearing details.

The cubin is a standard ELF file with an undocumented soul. The driver trusts its metadata completely, and the metadata must be exactly right.

Back to the blog