Finding codegen bugs for atomic operations is fun! I’ll note that I am not a Zig developer. I don’t write code in the language, and I don’t work on the language. I’m just some guy who decided to learn a bit about it.
The Zig programming language has two compilers that are widely used: one generates LLVM IR that libLLVM then builds into native code, and the other is a self-hosted compiler which eschews LLVM and does its own codegen.
It’s important to note that the self-hosted compiler is only currently (as of late October 2026) used on x86_64 Debug targets. All other targets use the LLVM backend by default.
_ = @atomicRmw(u32, p, .Xchg, 1, .seq_cst); is a sequentially consistent exchange. On x86 that is xchg with a memory operand, which is implicitly locked and is a full barrier. When the result is discarded, Zig’s self-hosted x86_64 backend emits a plain mov store instead. A plain store on x86 lets a later load be reordered before it (StoreLoad), so the operation loses its seq_cst ordering. Any build that uses the self-hosted x86_64 backend is affected. That backend is the default for -O Debug on x86_64-linux, and -fno-llvm selects it explicitly.
The same backend emits xchg when the result is used. So does a lock acquired by testing the exchange’s return value, which is not affected. Zig’s LLVM backend and gcc emit xchg in both cases.
This affects softwhare when: code discards the exchange’s result, and then relies on access of a different variable to be executed by the processor after that xchg. An example could be code sets its own flag, then reads another thread’s flag.
Affected: Zig 0.16.0 (Fedora 44 package), 0.17.0 and (current at the time) master 0.18.0-dev.367+187d0bf9e (commit 187d0bf9e483, 2026-10-19), all with the self-hosted x86_64 backend (the default for -O Debug -target x86_64-linux). The LLVM backend (-fllvm) is not affected.
Reproducing the codegen bug
In order to see the codegen bug, you can build a small source file and disassemble it:
export fn xchg_unused(p: *u32) void {
_ = @atomicRmw(u32, p, .Xchg, 1, .seq_cst); // BUG: result discarded -> plain mov
}
export fn xchg_used(p: *u32) u32 { // control
return @atomicRmw(u32, p, .Xchg, 1, .seq_cst);
}We can then compare the result with some C code built with a variety of different toolchains, but gcc -O0 gives us the closest result to the self-hosted Zig compiler’s output.
void xchg_unused(unsigned *p) { (void)__atomic_exchange_n(p, 1, __ATOMIC_SEQ_CST); }The (annotated) gcc output for the C code looks like this:
+ x86_64-linux-gnu-gcc -O0 -c /repro/src/xchg.c -o /tmp/xchg/gcc-O0/xchg.o
+ cd /tmp/xchg/gcc-O0 && x86_64-linux-gnu-objdump -d --no-addresses --disassemble=xchg_unused xchg.o > xchg_unused.objdump
<xchg_unused>:
55 push %rbp
48 89 e5 mov %rsp,%rbp
48 89 7d f8 mov %rdi,-0x8(%rbp)
48 8b 45 f8 mov -0x8(%rbp),%rax
ba 01 00 00 00 mov $0x1,%edx
87 10 xchg %edx,(%rax) # opcode 87: xchg, implicitly locked: correct
90 nop
5d pop %rbp
c3 ret
RESULT[gcc -O0 (16.2.1)]: OK, 1 locked/fencing instruction(s)While we can see the difference in the code generated by the Zig compiler below, annotated as to where the bug is:
+ /opt/zig-0.17.0/zig build-obj /repro/src/xchg.zig -target x86_64-linux -O Debug -femit-bin=/tmp/xchg/zig-0.17.0/xchg.o
+ cd /tmp/xchg/zig-0.17.0 && x86_64-linux-gnu-objdump -d --no-addresses --disassemble=xchg_unused xchg.o > xchg_unused.objdump
+ diff -u /tmp/xchg/gcc-O0/xchg_unused.objdump /tmp/xchg/zig-0.17.0/xchg_unused.objdump
@@ -7,10 +7,12 @@
<xchg_unused>:
55 push %rbp
48 89 e5 mov %rsp,%rbp
- 48 89 7d f8 mov %rdi,-0x8(%rbp)
- 48 8b 45 f8 mov -0x8(%rbp),%rax
- ba 01 00 00 00 mov $0x1,%edx
- 87 10 xchg %edx,(%rax) # gcc: the locked exchange…
- 90 nop
+ 53 push %rbx
+ 48 83 ec 08 sub $0x8,%rsp
+ 48 89 3c 24 mov %rdi,(%rsp)
+ be 01 00 00 00 mov $0x1,%esi
+ 89 37 mov %esi,(%rdi) # …Zig: opcode 89, a plain store, and no lock or fence anywhere
+ 48 8d 65 f8 lea -0x8(%rbp),%rsp
+ 5b pop %rbx
5d pop %rbp
c3 ret
RESULT[zig-0.17.0]: BUG, no xchg, lock or mfence: the exchange is a plain storeWhile there’s a bunch of boilerplate around function entry/exit that differs between the two, the xchg and mov are fairly visible in the middle. If we diff the output of the xchg_used and xchg_unused from the self-hosted Zig compiler, we remove the boiler plate delta:
+ diff -u /tmp/xchg/zig-0.17.0/xchg_unused.objdump /tmp/xchg/zig-0.17.0/xchg_used.objdump
-<xchg_unused>:
+<xchg_used>:
…
be 01 00 00 00 mov $0x1,%esi
- 89 37 mov %esi,(%rdi)
+ 87 37 xchg %esi,(%rdi)
+ 89 f0 mov %esi,%eaxReproducing at runtime
Since it is always a pile of fun to reproduce concurrency bugs, catching this at runtime means running a bunch of iterations and looking for a fail condition. The following does that, with two threads:
// thread A: exchange(x, 1) seq_cst; r1 = load(y) seq_cst
// thread B: exchange(y, 1) seq_cst; r2 = load(x) seq_cst
//
// seq_cst forbids r1 == 0 and r2 == 0 together: whichever exchange comes first in the
// single total order is visible to the other thread's load. The program counts how often
// both loads read 0 in three trials:
//
// relaxed monotonic store, then monotonic load. Both-zero is allowed here, and is what
// x86 does for a plain `mov` store followed by a load. Shows the test can see it.
// discarded seq_cst exchange, result discarded. The bug: the self-hosted x86_64 backend
// emits a plain `mov`, so this behaves like `relaxed`.
// used seq_cst exchange, result used. Control: emitted as `xchg`, never both-zero.
//
// Exits 1 if both-zero was seen in either seq_cst trial.
const std = @import("std");
const iterations = 1_000_000;
var x: u32 = 0;
var y: u32 = 0;
var go: u32 = 0; // iteration the threads should run next; set by main
var done: u32 = 0; // threads that finished the current iteration
const Mode = enum { relaxed, discarded, used };
fn Worker(comptime mode: Mode) type {
return struct {
fn run(mine: *u32, other: *u32, seen: *u32, old: *u32) void {
var i: u32 = 1;
while (i <= iterations) : (i += 1) {
while (@atomicLoad(u32, &go, .acquire) != i) std.atomic.spinLoopHint();
switch (mode) {
.relaxed => @atomicStore(u32, mine, 1, .monotonic),
.discarded => _ = @atomicRmw(u32, mine, .Xchg, 1, .seq_cst), // BUG: plain `mov`
.used => old.* = @atomicRmw(u32, mine, .Xchg, 1, .seq_cst), // control: `xchg`
}
seen.* = @atomicLoad(u32, other, if (mode == .relaxed) .monotonic else .seq_cst);
_ = @atomicRmw(u32, &done, .Add, 1, .acq_rel);
}
}
};
}
fn trial(comptime mode: Mode) !u32 {
const W = Worker(mode);
var r1: u32 = 0;
var r2: u32 = 0;
var old1: u32 = 0;
var old2: u32 = 0;
go = 0;
done = 0;
const a = try std.Thread.spawn(.{}, W.run, .{ &x, &y, &r1, &old1 });
const b = try std.Thread.spawn(.{}, W.run, .{ &y, &x, &r2, &old2 });
var both_zero: u32 = 0;
var i: u32 = 1;
while (i <= iterations) : (i += 1) {
@atomicStore(u32, &x, 0, .seq_cst);
@atomicStore(u32, &y, 0, .seq_cst);
@atomicStore(u32, &go, i, .release); // start both threads
while (@atomicLoad(u32, &done, .acquire) != 2 * i) std.atomic.spinLoopHint();
if (r1 == 0 and r2 == 0) both_zero += 1; // forbidden in the two seq_cst trials
}
a.join();
b.join();
return both_zero;
}
pub fn main() !void {
const relaxed = try trial(.relaxed);
const discarded = try trial(.discarded);
const used = try trial(.used);
std.debug.print("relaxed store+load (allowed): both loads read 0 in {d} of {d} iterations\n", .{ relaxed, iterations });
std.debug.print("seq_cst xchg, result discarded: both loads read 0 in {d} of {d} iterations\n", .{ discarded, iterations });
std.debug.print("seq_cst xchg, result used (control): both loads read 0 in {d} of {d} iterations\n", .{ used, iterations });
if (discarded != 0 or used != 0) std.process.exit(1);
}
This will reproduce on x86-64 systems fairly easily. I’ll note that I didn’t get qemu to reproduce when doing usermode emulation on an aarch64 host, so be sure to try on a real x86-64 system (and almost certainly a multi-core one). Below, we have the output of running the binary produced by the self hosted Zig compiler, and then the output of the binary built with the LLVM based one:
sb-0.17.0 Worker(.discarded): 89 3e mov %edi,(%rsi) # self-hosted: plain store
sb-0.17.0 Worker(.used): 87 3e xchg %edi,(%rsi)
sb-0.17.0-llvm Worker(.discarded): 87 16 xchg %edx,(%rsi) # LLVM: xchg either way
+ /tmp/sb/sb-0.17.0
relaxed store+load (allowed): both loads read 0 in 96990 of 1000000 iterations
seq_cst xchg, result discarded: both loads read 0 in 128430 of 1000000 iterations # forbidden, seen 13% of the time
seq_cst xchg, result used (control): both loads read 0 in 0 of 1000000 iterations
RESULT[sb-0.17.0]: BUG, both loads read 0 under seq_cst
+ /tmp/sb/sb-0.17.0-llvm
relaxed store+load (allowed): both loads read 0 in 41808 of 1000000 iterations
seq_cst xchg, result discarded: both loads read 0 in 0 of 1000000 iterations # same source, LLVM backend
seq_cst xchg, result used (control): both loads read 0 in 0 of 1000000 iterations
RESULT[sb-0.17.0-llvm]: OK, seq_cst ordering heldI’ve reproduced this with the 0.16 version shipped in Fedora 44, the upstream 0.17 binaries, and upstream nightly build 0.18.0-dev.367+187d0bf9e.
Okay, but get Volume 3A of the Intel® 64 and IA-32 Architectures Software Developer’s Manual nerdy with me
Specifically, “11.2.2 Memory Ordering in P6 and More Recent Processor Families” in the Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3A has two relevant rules:
- “Reads may be reordered with older writes to different locations but not with older writes to the same location.” This is what the plain
movallows: a later load can complete before the store is visible to other processors. - “Reads or writes cannot be reordered with I/O instructions, locked instructions, or serializing instructions.” This is what a locked instruction guarantees, and seq_cst requires it.
The Zig language appears to have a big TODO in their language documentation around Atomics, and the std documentation for what the enum is for AtomicOrder. The upstream LLVM docs are rather complete.
“Additionally, the code generator must enforce ordering between SequentiallyConsistent stores followed by SequentiallyConsistent loads” – https://llvm.org/docs/Atomics.html#sequentiallyconsistent
Did I report this upstream?
No, because doing so would be a violation of the Code of Conduct. Specifically, let’s look at the LLM/AI policy part:
Since I used an LLM to help me learn a bit about Zig and how it works, and then got it to help isolate this down, the “No LLMs for finding bugs” undoubtedly applies. Later on, there’s also the explicit example of unacceptable behavior being “Discussing this Code of Conduct“.
I’ve written this up because others may find it interesting, and maybe this helps someone debug their concurrent Zig application at some point.