On Thu, Jun 21, 2012, Nadav Har'El wrote about "Re: Python question - first call is slower?": > My example program actually *forked* right before that first call. > > It turns out that (at least, this is my understanding now) memory allocation - > in Python but also in C - is significantly slower during this first call, but >... > The short explanation is copy-on-write page faults.
I have since discovered that there's another reason for slow memory allocation after a fork, even if we avoid copy-on-write by filling the parent's malloc arena before the fork: When the child allocates *new* memory with sbrk(), and later uses it, "minor page faults" happen causing the kernel to zero the new pages before the program can using it. This zeroing takes time, but is important (for security reasons). What worries me more, though, are these minor page faults. These happen lazily - only when I access a page, a page fault happens, so I am getting one minor page fault per page, and these appear to have significant overhead (page table locks, TLB flushes, or I don't know what). What I'd rather happen in the child is that when sbrk() allocates new memory, the kernel immediately maps and zeros all these pages - and doesn't wait until they are actually used (which I know they will). Do anyone have any idea if this can be done, hopefully without modifying the kernel? Is there some madvise() or other trick which can be used to ensure that a large number of pages get mapped *now* instead of mapping them one after another? Nadav. -- Nadav Har'El | Tuesday, Jul 3 2012, 13 Tammuz 5772 n...@math.technion.ac.il |----------------------------------------- Phone +972-523-790466, ICQ 13349191 |The human mind is like a parachute - it http://nadav.harel.org.il |functions better when it is open. _______________________________________________ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
