Hi, Gentle ping.
Thanks, Soumya > On 13 Jul 2026, at 9:20 PM, Soumya AR <[email protected]> wrote: > > Hi, > > Sending a couple of updated patches with the following fixes/additions: > > ---- > > [1] middle-end: Add support for atomic fetch min/max builtins via CAS lowering > > I had missed adding support for atomic fetch min/max on pointer types. The > updated patch enables it in the frontend. > > For incompatible types, eg: > > - char* <-> int* > - int* <-> int > > GCC does a silent conversion without throwing a warning, so the same is > followed for fetch min/max builtins. > > ---- > > [4] aarch64: Add backend support for atomic fetch min/max operations > > For the fallback to LL/SC atomics (in case of no LSE), the older > implementation > had the following bug: > > - Consider the call __atomic_fetch_min (&ptr, val, __ATOMIC_RELAXED). > - In the LL/SC fallback for QI/HI modes, LDXR loads the byte or halfword from > memory (&ptr) and zero-extends it into a full W register. > - For UMIN / UMAX, we also need to zero-extend val since CMP operates on > the full W register and the high bits of val can affect the comparison. > - Have added this in the updated patch. > - (The sign-extend for SMIN/SMAX was already handled.) > > Thanks, > Soumya > > On on Wed, 10 Jun 2026 19:38:11 +0530, <[email protected]> wrote: >> From: Soumya AR <[email protected]> >> >> Hi, >> >> This series implements support for integer atomic fetch min/max in GCC, >> backing the C++26 std::atomic<T>::fetch_min and std::atomic<T>::fetch_max >> operations. This is v3 of previous patch series posted here, but has had >> significant rework that pretty much renders the previous patches obsolete. >> >> v1: https://gcc.gnu.org/pipermail/gcc-patches/2025-September/693926.html >> v2: https://gcc.gnu.org/pipermail/gcc-patches/2026-January/706239.html >> >> For some context, v1 initially took the approach of implementing everything >> through builtins, resulting in a significant builtin explosion. v2 improved >> upon this by only emitting generic builtins and lowering those to an IFN >> instead. >> >> v3 improves further upon this design, makes the infrastructure more generic >> for future atomic operations, and has had wider testing. >> >> ---- >> >> This series introduces the following patches: >> >> [1] middle-end: Add support for atomic fetch min/max builtins via CAS >> lowering >> [2] expand: Expand IFN_ATOMIC_FETCH_MINMAX to target-specific optabs >> [3] middle-end: Pattern match CAS loops to IFN_ATOMIC_FETCH_MINMAX >> [4] aarch64: Add backend support for atomic fetch min/max operations >> [5] experimental: Move atomic-op-lowering pass to gimple-isel >> >> The infrastrucure these patches aim to provide looks something like: >> >> __atomic_fetch_{min,max} [1] user-written CAS loop >> | | >> | pass_cas_to_atomic_op [3] >> | | >> +-----------------+-----------------+ >> v >> IFN_ATOMIC_FETCH_MINMAX [1] >> | >> --- IPA passes --- >> | >> +-----------------+-----------------+ >> | | >> can expand cannot expand >> | | >> backend-specific optab [2] CAS-loop lowering >> | pass_lower_atomic_ifn [1] >> (for aarch64) >> appropriate expansion [4] >> >> [5] is an experimental attempt at moving the lowering pass into gimple-isel, >> in an effort to not create a separate standalone pass (more on that below). >> >> ---- >> >> [1] middle-end: Add support for atomic fetch min/max builtins via CAS >> lowering >> >> This pass provides support to lower atomic ops to CAS loops in gimple. >> Earlier >> versions of this patch were min/max specific. Since we expect future work >> (FP fetch_add/sub, atomic reductions + -fno-inline-atomics) to require the >> same >> plumbing, the pass has been rewritten to be generic. Adding support for a >> new >> atomic fetch IFN now involves creating two op-specific callbacks here that >> modify the CAS loop accordingly for that op. >> >> This pass runs after IPA so that the optab query sees the correct backend. >> That matters under offloading, where we can compile for either host or >> target, >> and querying optabs pre-IPA would not give us the right backend. See >> https://gcc.gnu.org/pipermail/gcc-patches/2025-November/699630.html >> for the relevant discussion. >> >> Lowering this in gimple rather than expand gives us the benefit of subsequent >> optimization passes now having access to the lowered CAS loop. >> >> We also lower when -fno-inline-atomics is true, since fetch min/max do not >> have >> corresponding libcalls in libatomic, the best we can do is emit a CAS >> libcall. >> >> ---- >> >> [2] expand: Expand IFN_ATOMIC_FETCH_MINMAX to target-specific optabs >> >> The IFN reaches expand only when the backend has a target-specific optab. >> Still, we choose to emit expand_atomic_fetch_op over the _no_fallback variant >> because having the optab doesn't guarantee that the insn's operand predicates >> will accept the rtx we hand it. >> >> Therefore, since the actual optab emission could fail, expand_atomic_fetch_op >> should fall through to a CAS loop or libcall. >> >> ---- >> >> [3] middle-end: Pattern match CAS loops to IFN_ATOMIC_FETCH_MINMAX >> >> The motivation for this patch comes from the discussion here: >> https://gcc.gnu.org/pipermail/gcc-patches/2025-November/699602.html >> >> Pattern-matching to an IFN allows us to emit fetch_min/max specific backend >> optabs (if they exist), in place of the user written CAS loop. >> >> That said, I have only found one CAS loop in the wild so far that implements >> fetch min/max, but from my reasoning, this is an instance we cannot match: >> >> static inline void >> update_earliest (p64_tick_t exp) >> { >> p64_tick_t old; >> do >> { >> old = atomic_load_n (&g_timer.earliest, __ATOMIC_RELAXED); >> if (exp >= old) >> return; /* <-- note the early exit */ >> } >> while (UNLIKELY (!atomic_compare_exchange_n (&g_timer.earliest, >> &old, exp, >> __ATOMIC_RELEASE, >> __ATOMIC_RELAXED))); >> } >> >> (from progress64, src/p64_timer.c, courtesy of Al Grant who guided me to >> this) >> >> This is technically a fetch-min on g_timer.earliest, but the fast-path return >> turns the loop into load -> maybe-write -> otherwise-noop. AIUI, fetch >> min/max >> is an RMW that always writes (even when the stored value would be unchanged), >> so collapsing this loop into IFN_ATOMIC_FETCH_MINMAX would change the >> observable behaviour. >> >> https://eel.is/c++draft/atomics#types.pointer-6 on fetch_add/sub/min/max: >> >>> Effects: Atomically replaces the value pointed to by this with the result of >>> the computation applied to the value pointed to by this and the given >>> operand. >>> Memory is affected according to the value of order. These operations are >>> atomic read-modify-write operations. >> >> If anyone is aware of other projects that pattern-match CAS loops into atomic >> fetch min/max, I would appreciate pointers. Especially because the pass for >> it is relatively complex. >> >> For most matched loops we do run the risk of rewriting CAS -> IFN only to >> have >> the gimple-isel lowering pass turn it right back into a CAS loop when the >> backend doesn't have an optab, simply because of the few backends that do >> support it. But as mentioned earlier, we accept that round-trip since we >> cannot >> query optabs before IPA. >> >> ---- >> >> [4] aarch64: Add backend support for atomic fetch min/max operations >> >> This patch plugs into the infrastructure from [1] and [2] to emit the >> relevant >> aarch64 instructions. >> >> Three execution paths depending on LSE availability: >> - LSE inline (-march=armv8.1-a or later): native LSE ldsmin/ldsmax/ >> ldumin/ldumax. >> - Outline atomics (default): libgcc dispatches to LSE or LL/SC at runtime. >> - Inline LL/SC (-mno-outline-atomics on non-LSE targets): LDXR/STXR with >> conditional select. >> >> ---- >> >> [5] experimental: Move atomic-op-lowering pass to gimple-isel >> >> This is a follow-up to feedback on the v2 thread: >> https://gcc.gnu.org/pipermail/gcc-patches/2026-March/710694.html >> >> Having a dedicated post-IPA pass just to walk the IR and lower a >> (usually-absent) IFN unconditionally is costly. >> >> This patch folds the lowering into gimple-isel, which already walks the >> IR post-IPA (at -O0 and above) and already lowers other atomic IFNs. >> >> gimple-isel runs very late, after most optimizations, which means the CAS >> loop >> we synthesise misses pretty much every gimple-level optimisation that could >> have cleaned it up. >> >> Functionally, this does prove that the lowering can fit inside a pre-existing >> pass, but in practice lowering this late is close to lowering at RTL expand >> AIUI. >> >> ---- >> >> We currently don't extend tsan / asan / analyzer to handle >> IFN_ATOMIC_FETCH_MINMAX. They all already understand the CAS the IFN lowers >> to, >> but they don't know the IFN itself. To extend these, we could do the >> following: >> >> - Teach each analyser to understand the IFN directly. Should be doable for >> asan but not entirely sure how it would work for the analyzer... >> For tsan, we would have to emit an "unsupported" warning. >> >> - Lower to CAS if any of the sanitizer flags are true so we can use the >> pre-existing CAS support. >> >> ---- >> >> Note on integrating atomic reductions in the future: >> >> Reductions are atomic store-ops (add/sub/and/or/xor/min/max) that do not >> produce a visible read. For the most trivial fallback, they can be >> lowered to a fetch_op with the result discarded. >> >> Because we already handle min/max separately from add/sub/and/or/xor, >> adding store-op reductions on top will give us two paths per family: >> >> -fno-inline-atomics path: >> - add/sub/and/or/xor: substitute the IFN with the corresponding >> __atomic_fetch_OP builtin and let expand_builtin_atomic_fetch_op >> emit the libcall. >> - min/max: lower to a CAS loop in gimple; under -fno-inline-atomics >> its __atomic_compare_exchange_N becomes a libcall. (libatomic >> doesn't have min/max support.) >> >> -finline-atomics path: >> - add/sub/and/or/xor: directly use expand_atomic_fetch_op, which >> already handles the appropriate fallbacks. >> - min/max: if the backend exposes >> atomic_fetch_{smin,smax,umin,umax}_optab, >> we can use it in the expander, otherwise lower to a CAS loop in gimple. >> >> Flagging this to highlight the complexity that follows once min/max diverges >> from the rest of the atomic op handling. >> >> ---- >> >> Bootstrapped and regression tested on aarch64-linux-gnu and x86_64-linux-gnu. >> Cross-compiled and regression tested for arm-linux-gnueabihf-armv7-a and >> aarch64-linux-gnu without LSE. >> >> I also tested OpenMP offloading to verify that the lowering pass picks the >> right backend for both host and target. >> >> Soumya AR (5): >> middle-end: Add support for atomic fetch min/max builtins via CAS >> lowering >> expand: Expand IFN_ATOMIC_FETCH_MINMAX to target-specific optabs >> middle-end: Pattern match CAS loops to IFN_ATOMIC_FETCH_MINMAX >> aarch64: Add backend support for atomic fetch min/max operations >> experimental: Move atomic-op-lowering pass to gimple-isel >> >> gcc/Makefile.in | 1 + >> gcc/builtins.cc | 82 + >> gcc/builtins.h | 1 + >> gcc/c-family/c-common.cc | 63 +- >> gcc/config/aarch64/aarch64-protos.h | 4 + >> gcc/config/aarch64/aarch64.cc | 51 + >> gcc/config/aarch64/atomics.md | 54 +- >> gcc/config/aarch64/iterators.md | 30 +- >> gcc/gimple-isel.cc | 365 +++++ >> gcc/internal-fn.cc | 8 + >> gcc/internal-fn.def | 1 + >> gcc/optabs.cc | 38 +- >> gcc/optabs.def | 24 + >> gcc/optabs.h | 2 + >> gcc/passes.def | 1 + >> gcc/sync-builtins.def | 7 + >> .../template/builtin-atomic-overloads6.C | 33 +- >> .../template/builtin-atomic-overloads7.C | 23 +- >> .../g++.dg/tree-ssa/cas-to-minmax-1.C | 36 + >> .../g++.dg/tree-ssa/cas-to-minmax-2.C | 35 + >> .../g++.dg/tree-ssa/cas-to-minmax-3.C | 36 + >> .../g++.dg/tree-ssa/cas-to-minmax-4.C | 36 + >> .../gcc.dg/atomic-fetch-minmax-bad-types.c | 17 + >> .../gcc.dg/atomic-fetch-minmax-ifn.c | 16 + >> .../gcc.dg/atomic-fetch-minmax-lower.c | 17 + >> .../gcc.dg/atomic-fetch-minmax-sign-cast.c | 38 + >> .../atomic-fetch-minmax-type-mismatch.c | 29 + >> gcc/testsuite/gcc.dg/atomic-op-1.c | 243 ++- >> gcc/testsuite/gcc.dg/atomic-op-2.c | 243 ++- >> gcc/testsuite/gcc.dg/atomic-op-3.c | 243 ++- >> gcc/testsuite/gcc.dg/atomic-op-4.c | 243 ++- >> gcc/testsuite/gcc.dg/atomic-op-5.c | 245 ++- >> .../gcc.dg/tree-ssa/cas-to-minmax-1.c | 29 + >> .../gcc.dg/tree-ssa/cas-to-minmax-2.c | 28 + >> .../gcc.dg/tree-ssa/cas-to-minmax-3.c | 25 + >> .../gcc.dg/tree-ssa/cas-to-minmax-4.c | 33 + >> .../gcc.dg/tree-ssa/cas-to-minmax-5.c | 29 + >> .../gcc.dg/tree-ssa/cas-to-minmax-6.c | 28 + >> .../gcc.dg/tree-ssa/cas-to-minmax-7.c | 34 + >> .../gcc.dg/tree-ssa/cas-to-minmax-run-1.c | 75 + >> .../gcc.dg/tree-ssa/cas-to-minmax-run-2.c | 55 + >> .../gcc.dg/tree-ssa/cas-to-minmax-run-3.c | 59 + >> .../gcc.target/aarch64/atomic-minmax-lse.c | 122 ++ >> .../gcc.target/aarch64/atomic-minmax-nolse.c | 196 +++ >> .../gcc.target/aarch64/atomic-minmax.c | 128 ++ >> .../gcc.target/aarch64/atomic-minmax.x | 172 ++ >> gcc/tree-cas-to-atomic-op.cc | 1430 +++++++++++++++++ >> gcc/tree-pass.h | 1 + >> libgcc/config/aarch64/lse.S | 62 +- >> libgcc/config/aarch64/t-lse | 3 +- >> 50 files changed, 4752 insertions(+), 22 deletions(-) >> create mode 100644 gcc/testsuite/g++.dg/tree-ssa/cas-to-minmax-1.C >> create mode 100644 gcc/testsuite/g++.dg/tree-ssa/cas-to-minmax-2.C >> create mode 100644 gcc/testsuite/g++.dg/tree-ssa/cas-to-minmax-3.C >> create mode 100644 gcc/testsuite/g++.dg/tree-ssa/cas-to-minmax-4.C >> create mode 100644 gcc/testsuite/gcc.dg/atomic-fetch-minmax-bad-types.c >> create mode 100644 gcc/testsuite/gcc.dg/atomic-fetch-minmax-ifn.c >> create mode 100644 gcc/testsuite/gcc.dg/atomic-fetch-minmax-lower.c >> create mode 100644 gcc/testsuite/gcc.dg/atomic-fetch-minmax-sign-cast.c >> create mode 100644 gcc/testsuite/gcc.dg/atomic-fetch-minmax-type-mismatch.c >> create mode 100644 gcc/testsuite/gcc.dg/tree-ssa/cas-to-minmax-1.c >> create mode 100644 gcc/testsuite/gcc.dg/tree-ssa/cas-to-minmax-2.c >> create mode 100644 gcc/testsuite/gcc.dg/tree-ssa/cas-to-minmax-3.c >> create mode 100644 gcc/testsuite/gcc.dg/tree-ssa/cas-to-minmax-4.c >> create mode 100644 gcc/testsuite/gcc.dg/tree-ssa/cas-to-minmax-5.c >> create mode 100644 gcc/testsuite/gcc.dg/tree-ssa/cas-to-minmax-6.c >> create mode 100644 gcc/testsuite/gcc.dg/tree-ssa/cas-to-minmax-7.c >> create mode 100644 gcc/testsuite/gcc.dg/tree-ssa/cas-to-minmax-run-1.c >> create mode 100644 gcc/testsuite/gcc.dg/tree-ssa/cas-to-minmax-run-2.c >> create mode 100644 gcc/testsuite/gcc.dg/tree-ssa/cas-to-minmax-run-3.c >> create mode 100644 gcc/testsuite/gcc.target/aarch64/atomic-minmax-lse.c >> create mode 100644 gcc/testsuite/gcc.target/aarch64/atomic-minmax-nolse.c >> create mode 100644 gcc/testsuite/gcc.target/aarch64/atomic-minmax.c >> create mode 100644 gcc/testsuite/gcc.target/aarch64/atomic-minmax.x >> create mode 100644 gcc/tree-cas-to-atomic-op.cc >> >> -- >> 2.43.0 > > <v2-0004-aarch64-Add-backend-support-for-atomic-fetch-min-max.patch><v2-0001-middle-end-Add-support-for-atomic-fetch-min-max-buil.patch>
