Sean O'Rourke wrote: > >I took a look at what was going on, and found that the GC probably needs a > >good tuning. For the 20K file, parrot is doing 217 collections of the > >string pool, the last 102 of which reclaim less than 10% of the pool. > >Changing compact_string_pool() to increase the pool size by a factor of > >(0.5 - pct_freed_last_time) if it reclaimed less than 50% of memory > >reduced this to 21 collections, much fewer of which reclaimed abysmally > >small amounts of memory.
And Dan Sugalski replied: > Interesting. Could you work with Peter Gibbs and see what the two of > you can do to come up with an adaptive collection system? That's the > best way to go in the long term--the current system's functionally > complete, but far from well tuned. I am currently busy working on a variation of this theme - I was planning to get COW strings in before starting on the memory pool tuning phase, but I have put that on hold for a while to make sure I get it right next time. My current work is based on counting the size of freed buffers - yes, this adds some overhead in free_unused_buffers, but it seems like it may be worth it - and only doing compaction runs when a predefined fraction of the pool size is available for freeing. I am also looking at implementing an inflation factor similar to Sean's suggestion above. Another option is to allow the block size used to expand the pool to grow, so if the pool is growing rapidly, we will allocate less larger blocks; this keeps the linked list shorter, and means less calls to the operating system. Sean's program is ideal as a benchmark for this (after I removed all the shl's before the pack's, otherwise I just get files full of nulls). My latest test runs show an improvement on my system as follows: zip.pbc, input = string.c (23688 bytes) Current CVS version: 85.30 seconds (it's only an old 166MHz Pentium I) Latest test version: 1.77 seconds I have also been playing with variants of Mike Lambert's proposal for simplifying the resource pool tuning, so I don't have a patch ready yet; something will definitely be out sometime over the weekend. As usual, all suggestions are most welcome. -- Peter Gibbs EmKel Systems
