https://gcc.gnu.org/g:a33f26607eb4f3e3d68e575a0395401c940cf1c7
commit r17-2519-ga33f26607eb4f3e3d68e575a0395401c940cf1c7 Author: Jeff Law <[email protected]> Date: Sun Jul 19 08:11:23 2026 -0600 [committed] Improve select across A/A OP C where C is 2^n Testing for PR125731 exposed a bit of unexpected behavior on loongarch. Basically the PR125731 patch allows if-conversion to again handle generating conditional zero based sequences where one of the two operands in the original sequence was a constant integer. Generating a conditional zero based sequence usually results in better code than a generalized conditional move. Discovering those cases better led to a regression on loongarch which seems to want to generate even more specialized sequences than a conditional select across 0,2^n for things like conditional add. Consider this loongarch assembly: sltu $r12,$r0,$r12 slli.d $r12,$r12,16 add.d $r14,$r14,$r12 That's a conditional add by 65536. We use the sltu to generate 1/0, shift that by 16 generating 65536/0, then add that result to the other operand. With the work for PR125731 we get this instead: lu12i.w $r16,16 # 0x10 maskeqz $r12,$r16,$r12 add.d $r14,$r14,$r12 Normally I would prefer the 2nd sequence as the high part load has no dependencies and can issue whenever is convenient, but loongarch explicitly prefers the first sequence and I'm willing to assume that was done for a good reason. For RISC-V it's probably a toss-up. The first form likely compresses better and doesn't rely on zicond, but the second form has one less incoming dependency. Barring hard data, I'm going to declare them equivalent and target the sequence loongarch wants. Thankfully this is a class of problems that's been on my radar for a while. Given a select across A and A OP C where C is 2^N we can left shift the result of the SCC to give us a select across 0 and 2^n, then we emit A OP X (where X holds the result of that left shift). We can do this add, sub, shifts, rotates, ior, xor, basically anything where "0" is a neutral operand. That obviously excludes AND where -1 is the neutral. That's ultimately the same set of operators as the condzero arithmetic supports except we'd need to filter out AND. The implementation is structured similar to store_flag_constants, though simplified where obviously possible. If we look at a couple subtests within the loongarch conditional-move-opt-1.c testcase, but compiling for RISC-V: extern long lm, lr; void test_nez () { if (lm != 0) lr <<= (1 << 4); lr += lm; } void test_eqz () { if (lm == 0) lr >>= (1 << 2); lr += lm; } The relevant conditional move sequences look like this: slli a3,a5,16 czero.eqz t1,a3,t0 czero.nez t2,a5,t0 add a0,t2,t1 add a1,t0,a0 and: srai a3,a5,4 czero.nez t1,a3,t0 czero.eqz t2,a5,t0 add a0,t2,t1 add a1,t0,a0 Not bad, but with this patch we clearly do better: snez a3,t0 slli t1,a3,4 sll t2,a4,t1 add a0,t0,t2 and seqz a3,t0 slli t1,a3,2 sra t2,a4,t1 add a0,t0,t2 Probably the same performance as the czeros can execute in parallel, but it's smaller from an encoding standpoint and doesn't require zicond. Bootstrapped and regression tested on x86_64, alpha, armv7, loongarch64, riscv64 (k3, k1 and c920). Probably others as well, though I didn't check other natives explicitly to see if it'd picked up the latest version of the patch. Interestingly enough this does trigger meaningfully during bootstraps on various targets as I stumbled across multiple failures due to a couple logic errors in earlier versions. There's still things that could be improved in here. Most obviously AND handling, cases where STORE_FLAG_VALUE != 1 (which likely work due to normalization, but our ability to test is limited), exploiting negated logicals for things like conditional bit clear, etc. Even with the limitations, this seems worthwhile to go forward now. Pushing to the trunk. PR target/125731 gcc/ * ifcvt.cc (noce_cond_zero_binary_op_supported): Move earlier. (noce_try_shifted_store_flag): New function. (noce_process_if_block): Use it. gcc/testsuite * gcc.target/riscv/pr125731-1.c: New test. * gcc.target/riscv/rvv/vsetvl/vsetvl-15.c: Drop shift count test. Diff: --- gcc/ifcvt.cc | 161 ++++++++++++++++++--- gcc/testsuite/gcc.target/riscv/pr125731-1.c | 30 ++++ .../gcc.target/riscv/rvv/vsetvl/vsetvl-15.c | 1 - 3 files changed, 173 insertions(+), 19 deletions(-) diff --git a/gcc/ifcvt.cc b/gcc/ifcvt.cc index 1df5d54efa36..7c50ab06e2f7 100644 --- a/gcc/ifcvt.cc +++ b/gcc/ifcvt.cc @@ -781,6 +781,7 @@ static bool noce_try_ifelse_collapse (struct noce_if_info *); static bool noce_try_store_flag (struct noce_if_info *); static bool noce_try_addcc (struct noce_if_info *); static bool noce_try_store_flag_constants (struct noce_if_info *); +static bool noce_try_shifted_store_flag (struct noce_if_info *); static bool noce_try_store_flag_mask (struct noce_if_info *); static rtx noce_emit_cmove (struct noce_if_info *, rtx, enum rtx_code, rtx, rtx, rtx, rtx, rtx = NULL, rtx = NULL); @@ -1558,6 +1559,145 @@ noce_try_inverse_constants (struct noce_if_info *if_info) return false; } +/* Check if OP is supported by conditional zero based if conversion, + returning TRUE if satisfied otherwise FALSE. + + OP is the operation to check. */ + +static bool +noce_cond_zero_binary_op_supported (rtx op) +{ + enum rtx_code opcode = GET_CODE (op); + + if (opcode == PLUS || opcode == MINUS || opcode == IOR || opcode == XOR + || opcode == ASHIFT || opcode == ASHIFTRT || opcode == LSHIFTRT + || opcode == ROTATE || opcode == ROTATERT || opcode == AND) + return true; + + return false; +} + +/* Convert "if (test) x = a; else x = a OP c " where c is 2^n. + + We can shift the output of a set flag insn left to generate 2^n + cheaply, then apply OP unconditionally. This works even if the + target does not have a conditional zero idiom. On targets such + as RISC-V this sequence may also encode better. + + This is based on noce_try_store_flag_constants. */ + +static bool +noce_try_shifted_store_flag (struct noce_if_info *if_info) +{ + rtx target; + rtx_insn *seq; + machine_mode mode = GET_MODE (if_info->x); + rtx common = NULL_RTX; + rtx_code code = UNKNOWN; + + if (STORE_FLAG_VALUE != 1) + return false; + + if (!noce_simple_bbs (if_info)) + return false; + + rtx a = if_info->a; + rtx b = if_info->b; + + /* Most binary operators are allowed, essentially the same set + as the condzero arithmetic paths, with the exception of AND + which could be handled if we were inclined. */ + int shiftval; + bool reversed = false; + if (noce_cond_zero_binary_op_supported (a) + && GET_CODE (a) != AND + && GET_CODE (b) == REG + && rtx_equal_p (XEXP (a, 0), b) + && CONST_INT_P (XEXP (a, 1)) + && pow2p_hwi (INTVAL (XEXP (a, 1))) + && noce_reversed_cond_code (if_info) != UNKNOWN) + { + code = GET_CODE (a); + common = XEXP (a, 0); + shiftval = exact_log2 (INTVAL (XEXP (a, 1))); + + /* When the condition is true, we branch around A and thus we want + the condition reversed from what you might expect. */ + reversed = true; + } + else if (noce_cond_zero_binary_op_supported (b) + && GET_CODE (b) != AND + && GET_CODE (a) == REG + && rtx_equal_p (XEXP (b, 0), a) + && CONST_INT_P (XEXP (b, 1)) + && pow2p_hwi (INTVAL (XEXP (b, 1)))) + { + code = GET_CODE (b); + common = XEXP (b, 0); + shiftval = exact_log2 (INTVAL (XEXP (b, 1))); + } + + + /* If CODE hasn't been reset, then the basic expression form was wrong and + we can not optimize. */ + if (code == UNKNOWN) + return false; + + start_sequence (); + + /* If X and COMMON are the same, then we're going to need a temporary. */ + if (common && rtx_equal_p (common, if_info->x)) + { + common = gen_reg_rtx (mode); + noce_emit_move_insn (common, if_info->x); + } + + target = noce_emit_store_flag (if_info, if_info->x, reversed, 1); + if (!target) + { + end_sequence (); + return false; + } + + /* Right now TARGET has the value 1/0. A left shift will produce the + target value. Unnecessary if the shift value is zero. */ + if (shiftval != 0) + target = expand_simple_binop (mode, ASHIFT, target, + GEN_INT (shiftval), + NULL_RTX, 0, OPTAB_WIDEN); + + if (!target) + { + end_sequence (); + return false; + } + + /* Now we just need to apply the code to COMMON and TARGET. */ + target = expand_simple_binop (mode, code, + common, target, + NULL_RTX, 0, OPTAB_WIDEN); + + if (!target) + { + end_sequence (); + return false; + } + + /* If necessary, store the result into the proper destination. */ + if (target != if_info->x) + noce_emit_move_insn (if_info->x, target); + + seq = end_ifcvt_sequence (if_info); + if (!seq || !targetm.noce_conversion_profitable_p (seq, if_info)) + return false; + + emit_insn_before_setloc (seq, if_info->jump, + INSN_LOCATION (if_info->insn_a)); + if_info->transform_name = "noce_try_shifted_store_flag"; + + return true; +} + /* Convert "if (test) x = a; else x = b", for A and B constant. Also allow A = y + c1, B = y + c2, with a common y between A @@ -3167,24 +3307,6 @@ noce_try_sign_mask (struct noce_if_info *if_info) return true; } -/* Check if OP is supported by conditional zero based if conversion, - returning TRUE if satisfied otherwise FALSE. - - OP is the operation to check. */ - -static bool -noce_cond_zero_binary_op_supported (rtx op) -{ - enum rtx_code opcode = GET_CODE (op); - - if (opcode == PLUS || opcode == MINUS || opcode == IOR || opcode == XOR - || opcode == ASHIFT || opcode == ASHIFTRT || opcode == LSHIFTRT - || opcode == ROTATE || opcode == ROTATERT || opcode == AND) - return true; - - return false; -} - /* Helper function to return REG itself, otherwise NULL_RTX for other RTX_CODE. */ @@ -4550,6 +4672,9 @@ noce_process_if_block (struct noce_if_info *if_info) if (!targetm.have_conditional_execution () && noce_try_store_flag_constants (if_info)) goto success; + if (!targetm.have_conditional_execution () + && noce_try_shifted_store_flag (if_info)) + goto success; if (noce_try_sign_bit_splat (if_info)) goto success; if (!targetm.have_conditional_execution () diff --git a/gcc/testsuite/gcc.target/riscv/pr125731-1.c b/gcc/testsuite/gcc.target/riscv/pr125731-1.c new file mode 100644 index 000000000000..74e3f4b93f46 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/pr125731-1.c @@ -0,0 +1,30 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv64gcbv_zicond -mabi=lp64d -mbranch-cost=4" { target rv64 } } */ +/* { dg-additional-options "-march=rv32gcbv_zicond -mabi=ilp32 -mbranch-cost=4" { target rv32 } } */ +/* { dg-skip-if "" { *-*-* } { "-O0" "-O1" "-Os" "-Oz" "-Og" } } */ + + + +extern long lm, lr; + +void +test_nez () +{ + if (lm != 0) + lr <<= (1 << 4); + lr += lm; +} + +void +test_eqz () +{ + if (lm == 0) + lr >>= (1 << 2); + lr += lm; +} + +/* { dg-final { scan-assembler-not "\tczero.eqz\t" } } */ +/* { dg-final { scan-assembler-not "\tczero.nez\t" } } */ +/* { dg-final { scan-assembler-times "\tslli\t" 2 } } */ +/* { dg-final { scan-assembler-times "\tsll\t" 1 } } */ +/* { dg-final { scan-assembler-times "\tsra\t" 1 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/vsetvl/vsetvl-15.c b/gcc/testsuite/gcc.target/riscv/rvv/vsetvl/vsetvl-15.c index e42f77deaef9..2f193ef82993 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/vsetvl/vsetvl-15.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/vsetvl/vsetvl-15.c @@ -19,4 +19,3 @@ void foo(int32_t *in1, int32_t *in2, int32_t *in3, int32_t *out, size_t n, int c } /* { dg-final { scan-assembler-times {vsetvli} 2 { target { no-opts "-O0" no-opts "-O1" no-opts "-Os" no-opts "-g" no-opts "-funroll-loops" } } } } */ -/* { dg-final { scan-assembler-times {srli\s+[a-x0-9]+,\s*[a-x0-9]+,\s*4} 1 { target { no-opts "-O0" no-opts "-g" no-opts "-funroll-loops" } } } } */
