Hi!
As mentioned in the PR, on a huge function init_costs attempts to allocate
124 * 17380077 bytes, 124 is max_struct_costs_size and 17380077
is cost_elements_num. The problem is that the multiplication is done in
int type, so it overflows and we attempt to allocate 0xffffffff8074aacc
bytes. Other similar multiplications in ira-costs.c use size_t as type
of at least one operand, so it works properly. The following patch
fixes it by making sure struct_costs_size and max_struct_costs_size have
size_t type. The first hunk shouldn't result in more memory being used,
on 32-bit host it is not any change and the field is followed by
a pointer, so on 64-bit hosts there has been padding that we now use.
Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk?
2017-02-07 Jakub Jelinek <ja...@redhat.com>
Richard Biener <rguent...@suse.de>
PR middle-end/79399
* ira-int.h (struct target_ira_int): Change x_max_struct_costs_size
type from int to size_t.
* ira-costs.c (struct_costs_size): Change type from int to size_t.
--- gcc/ira-int.h.jj 2017-01-01 12:45:39.000000000 +0100
+++ gcc/ira-int.h 2017-02-07 09:29:32.694102809 +0100
@@ -782,7 +782,7 @@ struct target_ira_int {
/* Initialized once. It is a maximal possible size of the allocated
struct costs. */
- int x_max_struct_costs_size;
+ size_t x_max_struct_costs_size;
/* Allocated and initialized once, and used to initialize cost values
for each insn. */
--- gcc/ira-costs.c.jj 2017-01-16 12:28:35.000000000 +0100
+++ gcc/ira-costs.c 2017-02-07 11:44:26.534868418 +0100
@@ -74,7 +74,7 @@ static struct costs *costs;
static struct costs *total_allocno_costs;
/* It is the current size of struct costs. */
-static int struct_costs_size;
+static size_t struct_costs_size;
/* Return pointer to structure containing costs of allocno or pseudo
with given NUM in array ARR. */
Jakub
