https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91450
Bug ID: 91450
Summary: __builtin_mul_overflow(A,B,R) wrong code if product <
0, *R is unsigned, and !(A&B)
Product: gcc
Version: 9.2.0
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: c
Assignee: unassigned at gcc dot gnu.org
Reporter: eggert at cs dot ucla.edu
Target Milestone: ---
GCC generates wrong code for this program:
int a = -2;
int b = 1;
int
main (void)
{
unsigned long long r;
return __builtin_mul_overflow (a, b, &r);
}
The program should exit with status 1 because the mathematical product -2 does
not fit in unsigned long long. However, it exits with status 0, for both x86-64
and x86.
Looking at the generated code, the problem occurs when (a<0) != (b<0), i.e.,
the mathematical product is nonpositive. In this case, overflow should be
reported if the product is negative, i.e., if a!=0 & b!=0. However, the
generated code reports overflow if (a&b) != 0. That is an invalid optimization,
as illustrated in the sample program where a is -2 and b is 1 so overflow
occurs, but (a&b) == 0 so __builtin_mul_overflow incorrectly reports no
overflow.
