[PATCH] Another extract_muldiv-induced overflow (PR sanitizer/80932)

[Marek Polacek]

Another case of extract_muldiv running off the rails.  Here it did a wrong
distribution; turning 

  ((A * x) - (B * x)) * -6

into

  (A' * x) - (B' * x)

incurred an overflow in the subtraction.  The fix is essentially the same
as what I did in sanitizer/80800.

Bootstrapped/regtested on x86_64-linux, ok for trunk/7/6?

2017-06-06  Marek Polacek  <pola...@redhat.com>

        PR sanitizer/80932
        * fold-const.c (extract_muldiv_1) <case MINUS_EXPR>: Add
        TYPE_OVERFLOW_WRAPS check. 

        * c-c++-common/ubsan/pr80932.c: New test.

diff --git gcc/fold-const.c gcc/fold-const.c
index b0d03c9..f3975c7 100644
--- gcc/fold-const.c
+++ gcc/fold-const.c
@@ -6178,6 +6178,7 @@ extract_muldiv_1 (tree t, tree c, enum tree_code code, 
tree wide_type,
       t1 = extract_muldiv (op0, c, code, wide_type, &sub_strict_overflow_p);
       t2 = extract_muldiv (op1, c, code, wide_type, &sub_strict_overflow_p);
       if (t1 != 0 && t2 != 0
+         && TYPE_OVERFLOW_WRAPS (ctype)
          && (code == MULT_EXPR
              /* If not multiplication, we can only do this if both operands
                 are divisible by c.  */
diff --git gcc/testsuite/c-c++-common/ubsan/pr80932.c 
gcc/testsuite/c-c++-common/ubsan/pr80932.c
index e69de29..a833712 100644
--- gcc/testsuite/c-c++-common/ubsan/pr80932.c
+++ gcc/testsuite/c-c++-common/ubsan/pr80932.c
@@ -0,0 +1,17 @@
+/* PR sanitizer/80932 */
+/* { dg-do run } */
+/* { dg-options "-fsanitize=undefined -fsanitize-undefined-trap-on-error" } */
+
+int x = 1;
+
+long int
+foo (void)
+{
+  return ((long) (13801962912760474560ULL * x) - (long) 
(15334142073106273231ULL * x)) * -6;
+}
+
+int
+main ()
+{
+  foo ();
+}

        Marek