When working with modern, deep-pipeline GPUs like the NVIDIA B200, static analysis is necessary but insufficient for validating instruction schedules. It is a humbling experience to see a scheduler report 100% test coverage on dependency tracking, only to watch the emitted code fail silently on actual silicon. Why does this happen? The hardware pipeline itself is the final arbiter of correctness.
When a scheduler under-stalls a dependency, it allows a consumer instruction to issue into the pipeline before the producer's result is firmly committed to the register file. The hardware does not raise an exception. Instead, it executes the schedule, reading stale state, and propagates incorrect values through the rest of the computation.
These are not defects in the silicon. They are schedule violations where the hardware exposes the compiler's incorrect assumptions. The rule: over-stalling is a performance bug, but under-stalling is a silent correctness bug.
Why GPUs put scheduling on the compiler
On CPUs, sophisticated out-of-order execution engines mask latencies dynamically. On GPUs, the philosophy is to maximize die area for ALUs. This pushes the complexity of instruction scheduling onto the compiler. It is reminiscent of VLIW architectures, which share the same philosophy of offloading scheduling decisions from hardware to the compiler. This architectural tradeoff means compiler engineers must be pedantic about low-level constraints like pipeline depths and barrier encodings.
The predicate-consumer under-stall
The most difficult bugs slip through rigorous static checks. On the B200, a critical bug involving predicate evaluation in an instruction scheduler surfaced despite static metrics claiming full RAW coverage. The pattern involves an integer set-predicate instruction (ISETP) that computes a condition, writes it to a predicate register, which is then read by a branch instruction:
// 1. Produce the predicate P1 based on some condition.
ISETP.GE.AND P1, PT, R0, R1, PT;
// 2. Consume P1 as the branch target condition.
@!P0 BRA P1, target;The compiler correctly recorded the guard predicate P0 as a use for the branch, but it missed the branch condition operand P1. Consequently, the ISETP to BRA RAW dependency was missed entirely, and the scheduler failed to insert the required predicate-latency stall. The branch issued roughly 4 cycles after the ISETP, well before the predicate's modeled latency of 13 cycles had elapsed. The branch instruction read a stale value, took the wrong execution edge, and resulted in a silent miscomputation.
The true defense is an on-silicon probe: sweep the stall cycles between the ISETP and the branch, verifying the minimum latency required for correct execution. On the B200, microbenchmarking probes confirmed the divergence between the modeled 13 cycles and the actual pipeline depths, where the physical predicate latency floor sits at approximately 4 cycles.
Fixed-latency RAW under-stalls
Fixed-latency arithmetic instructions form the backbone of matrix multiplication and tensor core workloads. They require precise, fixed cycle delays before their destination registers can be safely read. Examples include FFMA (single-precision fused multiply-add) and DFMA (double-precision fused multiply-add). If a scheduler emits a stall with a cycle count strictly below the hardware's fixed latency, the consumer reads the destination register early.
Through direct hardware probing on the B200, I measured the exact latency floors where execution transitions from incorrect (stale read) to correct (valid read):
- FFMA FP32: 4 cycles. Stall 3 yields WRONG result. Stall 4 yields CORRECT.
- DFMA FP64: 8 cycles. Stall 7 yields WRONG result. Stall 8 yields CORRECT.
Notice the tradeoff: higher precision arithmetic naturally requires deeper pipelines. The FP64 unit requires exactly twice the latency of the FP32 unit.
When building latency validation tests, it is critical to construct floating-point recurrence chains rather than integer linear chains. Integer chains can be folded or bypassed via pipeline forwarding networks in hardware, which masks under-stalls. Floating-point chains, due to strict execution pipeline stages and rounding, make dependency latencies visible.
To validate these latencies, I wrote a probe kernel that builds a long dependent FFMA chain: a = a*1 + 1, repeated 64 times, so the expected result is exactly seed + 64. Every FFMA reads the immediately preceding FFMA's result, creating a pure RAW hazard chain. A post-processing script then rewrites the stall field of every FFMA in the compiled SASS binary to a forced value and runs each variant on the B200:
# stall cycles result (chain=64)
0 2205 CORRECT
1 225 WRONG
2 227 WRONG
3 291 WRONG
4 355 CORRECT
5 419 CORRECT
...
15 1075 CORRECTThe boundary is unambiguous. Stall 3 yields WRONG (the consumer reads a stale register), stall 4 yields CORRECT. Stall 0 is a special encoding that defaults to a large wait, which is why it reports CORRECT but at a much higher cycle count.
The GPU scheduler is a traffic controller with a printed timetable. The timetable (static latency model) is mostly right, but the controller must verify it against real traffic (silicon) because the roads are deeper than the map says. Under-stalling is running a red light: no alarm, just a collision nobody notices until the results are wrong.
The "friendliest dependency" trap
A word of caution: the FFMA floor of 4 came from the friendliest possible dependency shape, an FFMA-to-FFMA chain where the consumer reads a single live register and two folded-immediate 1.0f operands. This is the easiest case for the hardware to service. Real kernels have harder dependency shapes, and the true latency floor can be higher. The probe establishes a lower bound, not the whole truth.
Over-stalling is a performance bug. Under-stalling is a silent correctness bug. The hardware pipeline is the final arbiter of correctness.