[PATCH] match.pd, v2: Avoid infinite recursion during bswap (int_cst_ovf1) cmp int_cst_ovf2 [PR104644]

[Jakub Jelinek via Gcc-patches]

On Wed, Feb 23, 2022 at 09:32:41AM +0100, Jakub Jelinek via Gcc-patches wrote:
> The following patch avoids infinite recursion during generic folding.
> The (cmp (bswap @0) INTEGER_CST@1) simplification relies on
> (bswap @1) actually being simplified, if it is not simplified, we just
> move the bswap from one operand to the other and if @0 is also INTEGER_CST,
> we apply the same rule next.
> 
> The reason why bswap @1 isn't folded to INTEGER_CST is that the INTEGER_CST
> has TREE_OVERFLOW set on it and fold-const-call.cc predicate punts in
> such cases:
> static inline bool
> integer_cst_p (tree t)
> {
>   return TREE_CODE (t) == INTEGER_CST && !TREE_OVERFLOW (t);
> }
> The patch uses ! modifier to ensure the bswap is simplified, but because
> ! is only supported in gimple-match, guards it also with #if GIMPLE.

Here is another variant, which just breaks the possible ping-pong.
If @0 is not INTEGER_CST, we still want to canonicalize to bswap on the
INTEGER_CST (e.g. in the hope that we throw away TREE_OVERFLOW during/after
gimplification), but if it is INTEGER_CST, we don't want to move bswap
to the operand with TREE_OVERFLOW on it.

Ok for trunk if this passes bootstrap/regtest (it fixes the testcase too)?

2022-02-23  Jakub Jelinek  <ja...@redhat.com>

        PR tree-optimization/104644
        * match.pd (cmp (bswap @0) INTEGER_CST@1): Don't simplify
        if TREE_OVERFLOW (@1) and @0 is INTEGER_CST.

        * gcc.dg/pr104644.c: New test.

--- gcc/match.pd.jj     2022-02-23 09:17:04.867124392 +0100
+++ gcc/match.pd        2022-02-23 10:31:05.417304115 +0100
@@ -3961,8 +3961,9 @@ (define_operator_list SYNC_FETCH_AND_AND
     (cmp (convert:ctype @0) (convert:ctype @1))))
   (simplify
    (cmp (bswap @0) INTEGER_CST@1)
-   (with { tree ctype = TREE_TYPE (@1); }
-    (cmp (convert:ctype @0) (bswap @1)))))
+   (if (TREE_CODE (@0) != INTEGER_CST || !TREE_OVERFLOW (@1))
+    (with { tree ctype = TREE_TYPE (@1); }
+     (cmp (convert:ctype @0) (bswap @1))))))
  /* (bswap(x) >> C1) & C2 can sometimes be simplified to (x >> C3) & C2.  */
  (simplify
   (bit_and (convert1? (rshift@0 (convert2? (bswap@4 @1)) INTEGER_CST@2))
--- gcc/testsuite/gcc.dg/pr104644.c.jj  2022-02-23 10:29:50.704341688 +0100
+++ gcc/testsuite/gcc.dg/pr104644.c     2022-02-23 10:29:50.704341688 +0100
@@ -0,0 +1,9 @@
+/* PR tree-optimization/104644 */
+/* { dg-do compile } */
+/* { dg-options "-Wno-overflow" } */
+
+int
+foo (void)
+{
+  return __builtin_bswap16 (1.31072e+5f) != (signed char) 1.31072e+5f;
+}


        Jakub