https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68065
--- Comment #3 from Alexander Cherepanov <ch3root at openwall dot com> ---
(In reply to jos...@codesourcery.com from comment #2)
> This seems like a matter for -fsanitize=undefined
UBSAN is intended to help with invalid programs but this code looks like valid.
Hence diagnostic for problems with such code seem to belong to core gcc.
The core issue is an overflow in size computations which is not limited to VLA.
You can as easily get a crash with non-VLA-array+sizeof+malloc:
#define N /* complex computation leading to.. */ (UINT_MAX / sizeof(int) + 2)
int (*p)[N];
printf("%zu\n", sizeof *p);
p = malloc(sizeof *p);
if (!p)
return 1;
for (unsigned i = 0; i < N; i++)
(*p)[i] = 1;
Please note:
- size in this examples is tuned so that it crashes on 32-bit platform only and
works fine on 64-bit one, i.e. the problem could arise in not quite evident
situations;
- the approach to dynanmic arrays a-la Pascal -- "int (*p)[n]; p =
malloc(sizeof *p);" -- is not common in C but could be expected to become more
popular now that UBSAN supports bounds-checking for VLAs.
