Re: [PATCH] Avoid ggc_collect () after WPA forking

[Richard Biener]

On Wed, 19 Mar 2014, Martin Liška wrote:

> There are stats for Firefox with LTO and -O2. According to graphs it
> looks that memory consumption for parallel WPA phase is similar.
> When I disable parallel WPA, wpa footprint is ~4GB, but ltrans memory
> footprint is similar to parallel WPA that reduces libxul.so linking by ~10%.

Ok, so I suppose this tracks RSS, not virtual memory use (what is
"used" and what is "active")?

And it is WPA plus LTRANS stages, WPA ends where memory use first goes
down to zero?

I wonder if you can identify the point where parallel streaming
starts and where it ends ... ;)

Btw, I have another patch in my local tree, limiting the
exponential growth of blocks we allocate when outputting sections.
But it shouldn't be _that_ bad ... maybe you can try if it has
any effect?

Thanks,
Richard.

Index: gcc/lto-section-out.c
===================================================================
--- gcc/lto-section-out.c       (revision 208642)
+++ gcc/lto-section-out.c       (working copy)
@@ -99,13 +99,19 @@ lto_end_section (void)
 }
 
 
+/* We exponentially grow the size of the blocks as we need to make
+   room for more data to be written.  Start with a single page and go up
+   to 2MB pages for this.  */
+#define FIRST_BLOCK_SIZE 4096
+#define MAX_BLOCK_SIZE (2 * 1024 * 1024)
+
 /* Write all of the chars in OBS to the assembler.  Recycle the blocks
    in obs as this is being done.  */
 
 void
 lto_write_stream (struct lto_output_stream *obs)
 {
-  unsigned int block_size = 1024;
+  unsigned int block_size = FIRST_BLOCK_SIZE;
   struct lto_char_ptr_base *block;
   struct lto_char_ptr_base *next_block;
   if (!obs->first_block)
@@ -135,6 +141,7 @@ lto_write_stream (struct lto_output_stre
       else
        lang_hooks.lto.append_data (base, num_chars, block);
       block_size *= 2;
+      block_size = MIN (MAX_BLOCK_SIZE, block_size);
     }
 }
 
@@ -152,7 +159,7 @@ lto_append_block (struct lto_output_stre
     {
       /* This is the first time the stream has been written
         into.  */
-      obs->block_size = 1024;
+      obs->block_size = FIRST_BLOCK_SIZE;
       new_block = (struct lto_char_ptr_base*) xmalloc (obs->block_size);
       obs->first_block = new_block;
     }
@@ -162,6 +169,7 @@ lto_append_block (struct lto_output_stre
       /* Get a new block that is twice as big as the last block
         and link it into the list.  */
       obs->block_size *= 2;
+      obs->block_size = MIN (MAX_BLOCK_SIZE, obs->block_size);
       new_block = (struct lto_char_ptr_base*) xmalloc (obs->block_size);
       /* The first bytes of the block are reserved as a pointer to
         the next block.  Set the chain of the full block to the