Mike Lambert wrote: >>The whole resource.c is replaced by calls to calloc/realloc (res.c), >>add_free_buffer does free(). >> > > I think that's the problem right there. What exactly are you changing > to use this new calloc/malloc implementation? One approach is to modify > memory.c to use the new versions, although since parrot does a lot of its > own memory management of large chunks, it's not likely to give any speed > improvement over the OS.
This is malloc is a very sophisticated non copying GC system, memory is not given back to OS by default. > If you're replacing resources.c and headers.c with this calloc/malloc, you > have to make sure to get every occurance, which can be difficult. What > level are you trying to hook in this new implementation? As I said, I replaced resources.c by res.c, which calls calloc/realloc directly for all buffers. headers.c is unchanged except for freeing buffers in add_free_buffers. But free means here actually they get reused sooner or later. > You can bind every buffer's contents to calloc/realloc calls, but then > there will be no copying or collection going on because we're not > allocating out of pools. No, the pools are in malloc. Please UTSL. And actually, yes there is (almost) no copying inside malloc, so it's faster. > You'll need to disable compact_pool to do > nothing, and update the checks in mem_allocate such that they aren't > dependant upon it being called. There is no more compact_pool, no mem_allocate, nothing. The buffer_pool is inside malloc now. Headers are the same. > If you want the actual headers to be allocated from calloc/realloc, you'll > need to change add_free_object and get_free_object in smallobject.c, and > (add|get)_free_(buffer|pmc) in headers.c. Then you'll need to disable DOD > because all of the headers will no longer be consecutive in large pools > like they were before. No headers are all the same like now, DOD runs (and works). > ... At this point, we're screwed because we will never > free any memory. You could reimplement DOD to use a pool of pointers to > headers, but that's just going to be diminishing returns, especially with > the random memory dereferences (we had relatively good cache coherency > before). DOD is unchanged. >>make test shows currently gc_2-4 broken (no statistics) and 2 tests from >>src/intlist.t, which is proably my fault. >> > > I'm not surprised, to be honest. :) Actually now _all_ tests (parrot + perl) run with success ;-) > gc tests some GC behaviors that aren't > tested through the normal code (ie, it will actually trigger a DOD and/or > collection run), so if they're broken, you've likely done something wrong. No the statistic counter (GC) are wrong, because there is no collect_run, that's all. > (changing just add_free_buffer without get_free_buffer or dod is one thing > that classifies as "something wrong" ;) I'll send tomorrow an update with some docu. Then we'll see, how wrong my hack is ;-) >>string_substr / unmake_COW currently does highly illegal (WRT malloc) >>things, which might or might not cause problems for the current GC >>implementation. s. patch. There are probably more things like this. > Those illegal things are likely illegal for malloc, but they work > perfectly fine for our pool-based system. Yes I said WRT malloc, or to be more exact, WRT realloc. Actually, when you look into the patch, it contains a fix. > ... Feel free to remove some of that > code (it's mostly optimizations) if you like. However, I sincerely doubt > that unmake_COW or string_substr is the cause of the GC bugs you > mentioned, ... This were the statistic counters. > ... since they shouldn't be used by those tests at all. (Those > tests merely verify that collections and dods are being run. They don't > even really check that the GC runs don't destroy important data. (So > gc_2 failing is basically saying that the pool compaction is failing.) "perl6 --test -r" says enough about destroyed data - and doesn't ;-) >>I didn't look further into memory usage or such, though top seems to >>show ~double the footprint of CVS. >> > > I think Dan would disallow such a patch for this reason alone. We're > already taking a 2x hit (peak) by using a copying collector. No need to > make it worse. :) Already solved to 99%. Footprint is like CVS, but parrot goes up and down with ever growing up. Please watch "perl6 examples/life.p6 1000" with top in a second window. > Mike Lambert Thanks for your comments, leo
