[PATCH] Fix UB in prepare_cmp_insn (PR middle-end/89281)

[Jakub Jelinek]

Hi!

The following hunk of code results in UB on the recently added testcase,
because if cmp_mode is SImode or DImode, then 1 << 32 or 1 << 64 is
undefined.  Fixed by using GET_MODE_MASK, plus UINTVAL because size is
really unsigned (code later on uses unsignedp=1 too).

Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk?

2019-02-12  Jakub Jelinek  <ja...@redhat.com>

        PR middle-end/89281
        * optabs.c (prepare_cmp_insn): Use UINTVAL (size) instead of
        INTVAL (size), compare it to GET_MODE_MASK instead of
        1 << GET_MODE_BITSIZE.

--- gcc/optabs.c.jj     2019-02-05 10:16:34.533743051 +0100
+++ gcc/optabs.c        2019-02-11 09:48:15.514432541 +0100
@@ -3898,7 +3898,7 @@ prepare_cmp_insn (rtx x, rtx y, enum rtx
 
          /* Must make sure the size fits the insn's mode.  */
          if (CONST_INT_P (size)
-             ? INTVAL (size) >= (1 << GET_MODE_BITSIZE (cmp_mode))
+             ? UINTVAL (size) > GET_MODE_MASK (cmp_mode)
              : (GET_MODE_BITSIZE (as_a <scalar_int_mode> (GET_MODE (size)))
                 > GET_MODE_BITSIZE (cmp_mode)))
            continue;

        Jakub