Double-width atomics aren’t atomic if using “zig cc” to build C/C++ code

The Zig project heavily advocates the use case of using its toolchain to build C/C++ code, “Use Zig as a zero-dependency, drop-in C/C++ compiler that supports cross-compilation out-of-the-box.” (Zig home page https://ziglang.org/ fetched 2026-10-21). Back in 2020, the announcement from Zig’s creator was “zig cc: a Powerful Drop-In Replacement for GCC/Clang”. As far as I can work out, the vast majority of the usage of the Zig toolchain is as a C/C++ compiler, and since it is LLVM based, one may reasonably assume that it is pretty much the same as using clang.

Unfortunately, this is not true if you rely on some atomic operations being, well, atomic.

I think going through the details on this bug is a pretty interesting learning exercise (I certainly learned some things), so here goes….

A compiler (at least in the Linux world, I assume it’s similar on other platforms) implements an atomic operation by emitting a lock-free instruction, or by calling a library function (e.g. __atomic_load_16, __atomic_load_8, etc) if building for a target which lacks a suitable lock-free instruction. This enables a compiler to emit efficient native code for the target, as well as have a library that can be used in two cases: 1) The base processor being targeted doesn’t have an instruction for the operation, and 2) the implementation of the library calls can be optimal for the CPU the program is running on, thus a program can target an older CPU but get (most of) the benefit of improved atomic instructions on newer processors.

The lock-free instruction and the implementation of the equivalent library function needs to be compatible as either form can be used on the same object in the same program. This allows building software that works across a large range of processors but has the hot paths built for different micro-architectures and selected at runtime. This is effectively the same thing as supporting linking in objects built with a prior version of the compiler that didn’t know about the lock-free instruction. Thus, LLVM and GCC require that the library implementing the atomic functions to be lock-free for every size the compiler can emit inline, and both link to GNU libatomic for the library functions.

What the toolchains require

The LLVM Atomics guide, “Libcalls: __atomic_*“:

The set of atomic sizes supported lock-free must be a superset of the sizes any compiler can emit. That is: if a new compiler introduces support for inline-lock-free atomics of size N, the __atomic_* functions must also have a lock-free implementation for size N. This is a requirement so that code produced by an old compiler (which will have called the __atomic_* function) interoperates with code produced by the new compiler (which will use native the atomic instruction).

GCC’s design for its atomic library, Atomic/GCCMM/LIbrary on the GCC wiki (archived copy, 2024-12-30):

Implementation of the external routines must agree with the compiler for any given object size. If the compiler generates lock-free code for an object, the library must also implement lock free for that size as well. It is conforming for the compiler to simply always call the library, so both must agree on the implementation.

The same page gives the 32-bit test below as its example:

Compile a library for 386. The 386 does not support a lock-free atomic int, so the compiler generates function calls. Compile an application for 486. The 486 does support a lock-free atomic int, so the compiler generates machine instructions. Link those two and run them on a 486. The operating system was built knowing that it’s architectural level is 486, so its atomic ABI library implements the atomic int with lock-free machine instructions.

The problem with “zig cc”

However, when you build your C/C++ code using zig cc, it will emit lock-free instructions and calls to library functions, but it does not link to GNU libatomic, instead it links to Zig’s bundled compiler_rt library, which contains its own implementations.

In Zig’s compiler_rt library, which is linked to your C/C++ code, __atomic_load_16 on x86_64 and ppc64le, and __atomic_load_8 on 32-bit x86, are implemented by taking a spinlock and then doing two separate half-size loads. This obviously breaks if the compiler can emit a lock-free instruction for these operations, which it does. Thus, loads tear and your atomic operations are not atomic. While it is possible to pass -latomic to zig cc, this makes no difference.

Reproducing with a single C file (x86-64, 16-byte atomics)

We will use a small C program where a writer thread and a reader thread access one _Atomic unsigned __int128. We use the __attribute__((target("cx16"))) feature of the compiler as a shortcut for situations where you’d have different implementations of one function optimized for different CPUs or for linking in different builds.

If we build the source with the default options, since writer() has __attribute__((target("cx16"))), its atomic_store will be an inline lock cmpxchg16b, and since reader() has no attribute, its atomic_load is a call to __atomic_load_16.

(spoiler) if you use clang or gcc to compile and link, this will work perfectly.

here’s the code:

/*
 * Test: the writer only stores values whose two 8-byte halves are equal;
 *       a load with unequal halves is torn, and should NEVER happen.
 */
#include <stdatomic.h>
#include <stdio.h>
#include <pthread.h>
#include <time.h>

static _Atomic unsigned __int128 shared;
static atomic_int stop;

__attribute__((target("cx16"))) static void *writer(void *arg) {
    (void)arg;
    for (unsigned long long i = 1; !atomic_load(&stop); i++)
        atomic_store(&shared, ((unsigned __int128)i << 64) | i);       /* inline lock cmpxchg16b */
    return NULL;
}

static void *reader(void *arg) {
    unsigned long long *torn = arg, loads = 0;
    while (!atomic_load(&stop)) {
        unsigned __int128 v = atomic_load(&shared);                    /* call __atomic_load_16 */
        if ((unsigned long long)(v >> 64) != (unsigned long long)v) (*torn)++;
        loads++;
    }
    torn[1] = loads;
    return NULL;
}

int main(void) {
    unsigned long long res[2] = {0, 0};
    pthread_t w, r;
    pthread_create(&w, NULL, writer, NULL);
    pthread_create(&r, NULL, reader, res);
    struct timespec t = {2, 0}; nanosleep(&t, NULL);
    atomic_store(&stop, 1);
    pthread_join(w, NULL); pthread_join(r, NULL);
    printf("torn loads: %llu of %llu\n", res[0], res[1]);
    return res[0] != 0;
}

If we now build and run that code (run it 3x for 2s each time), we can see how the different compilers fare:


+ clang -O2 atomic16.c -lpthread -latomic -o out/atomic16-clang-latomic
#    writer: cmpxchg16b   reader: call <__atomic_load_16@plt>
#    Links to: [libatomic.so.1] [libc.so.6]
#      __atomic_load_16: U __atomic_load_16@LIBATOMIC_1.0
+ zig cc -target x86_64-linux-gnu -O2 atomic16.c -lpthread -o out/atomic16-zigcc
#    writer: cmpxchg16b   reader: call <__atomic_load_16>
#    Links to: [libc.so.6] [libpthread.so.0]
#     __atomic_load_16: t __atomic_load_16
+ zig cc -target x86_64-linux-gnu -O2 atomic16.c -lpthread -latomic -o out/atomic16-zigcc-latomic
#    writer: cmpxchg16b   reader: call <__atomic_load_16>
#    Links to: [libc.so.6] [libpthread.so.0]
#    __atomic_load_16: t __atomic_load_16

+ ./out/atomic16-clang-latomic
torn loads: 0 of 682030142
torn loads: 0 of 645687188
torn loads: 0 of 692854822
+ ./out/atomic16-zigcc
torn loads: 720 of 128380947
torn loads: 382 of 138503698
torn loads: 13108 of 131805641
+ ./out/atomic16-zigcc-latomic
torn loads: 455 of 137924614
torn loads: 20909 of 132735414
torn loads: 425 of 135945438

I used Zig 0.17.0 for these tests.

The inability to report upstream

I wrote about it in more detail in my first post on Zig, but the tl;dr is that since an LLM was involved in finding this bug, the “No LLMs for finding bugs” part of their Code of Conduct obviously applies.

I’ve written this up because others may find it interesting, and maybe this helps someone debug their concurrent Zig application at some point, or learn something about how compilers do (or do not) work.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.