[Bug middle-end/78429] [5/6/7 Regression] ice on valid C code on x86_64-linux-gnu at -O3 in both 32-bit and 64-bit modes (internal compiler error: in set_value_range, at tree-vrp.c:361)

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78429

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P1
          Component|tree-optimization           |middle-end
   Target Milestone|7.0                         |5.5
            Summary|[7 Regression] ice on valid |[5/6/7 Regression] ice on
                   |C code on x86_64-linux-gnu  |valid C code on
                   |at -O3 in both 32-bit and   |x86_64-linux-gnu at -O3 in
                   |64-bit modes (internal      |both 32-bit and 64-bit
                   |compiler error: in          |modes (internal compiler
                   |set_value_range, at         |error: in set_value_range,
                   |tree-vrp.c:361)             |at tree-vrp.c:361)

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
Hmm, so

 <boolean_type 0x2aaaac16c888 public SI
    size <integer_cst 0x2aaaac025ee8 type <integer_type 0x2aaaac02a150
bitsizetype> constant 32>
    unit size <integer_cst 0x2aaaac025f00 type <integer_type 0x2aaaac02a0a8
sizetype> constant 4>
    align 32 symtab 0 alias set -1 canonical type 0x2aaaac16c888 precision 32
min <integer_cst 0x2aaaac176a20 -2147483648> max <integer_cst 0x2aaaac176bb8
2147483647>>

(look at the types min/max value).  For this rason 

(gdb) p debug_tree (min)
 <integer_cst 0x2aaaac176d98 type <boolean_type 0x2aaaac16c888> constant public
overflow -1>

is not considered is_overflow_infinity (it's not vrp_val_is_min/max).

So what is causing this is likely behavioral change of some constant folding
with respect to BOOLEAN_TYPEs.  Here

  _22 = BIT_FIELD_REF <mask__23.24_5, 32, 96>;
  _53 = (_Bool) _22;

changed behavior in force_fit_type which does:

tree
force_fit_type (tree type, const wide_int_ref &cst,
                int overflowable, bool overflowed)
{
  signop sign = TYPE_SIGN (type);

  /* If we need to set overflow flags, return a new unshared node.  */
  if (overflowed || !wi::fits_to_tree_p (cst, type))
    {
      if (overflowed
          || overflowable < 0
          || (overflowable > 0 && sign == SIGNED))
        {
          wide_int tmp = wide_int::from (cst, TYPE_PRECISION (type), sign);
          tree t = build_new_int_cst (type, tmp);
          TREE_OVERFLOW (t) = 1;
          return t;
        }

where -1 no longer "fits" bool even though it should.  So

  /* Short-circuit boolean types since various transformations assume that
     they can only take values 0 and 1.  */
  if (TREE_CODE (type) == BOOLEAN_TYPE)
    return eq_p (x, 0) || eq_p (x, 1);

is wrong and should use eq_p (x, -1) instead as bool is signed?  The
int_fits_type_p function is likely wrong as well (uses integer_onep).

The patch has been backported as well, adjusting regression marker.