On 10/19/2016 02:51 PM, Tomas Vondra wrote:
...
>
Yeah. There are three contexts in reorder buffers:- changes (fixed size) - txns (fixed size) - tuples (variable size) The first two work perfectly fine with Slab. The last one (tuples) is used to allocate variable-sized bits, so I've tried to come up with something smart - a sequence of Slabs + overflow AllocSet. I agree that in hindsight it's a bit strange, and that the "generational" aspect is the key aspect here - i.e. it might be possible to implement a memory context that allocates variable-length chunks but still segregates them into generations. That is, don't build this on top of Slab. That would also fix the issue with two allocators in GenSlab. I'll think about this.
And here is a fairly complete prototype of this idea, adding "Gen" generational memory context based only on the "similar lifespan" assumption (and abandoning the fixed-size assumption). It's much simpler than GenSlab (which it's supposed to replace), and abandoning the idea of composing two memory contexts also fixed the warts with some of the API methods (e.g. repalloc).
I've also been thinking that perhaps "Gen" would be useful for all three contexts in ReorderBuffer - so I've done a quick test comparing the various combinations (using the test1() function used before).
master slabs slabs+genslab slabs+gen gens
----------------------------------------------------------------
50k 18700 210 220 190 190
100k 160000 380 470 350 350
200k N/A 750 920 740 679
500k N/A 2250 2240 1790 1740
1000k N/A 4600 5000 3910 3700
Where:
* master - 23ed2ba812117
* slabs - all three contexts use Slab (patch 0001)
* slabs+genslab - third context is GenSlab (patch 0002)
* slabs+gen - third context is the new Gen (patch 0003)
* gens - all three contexts use Gen
The results are a bit noisy, but I think it's clear the new Gen context
performs well - it actually seems a bit faster than GenSlab, and using
only Gen for all three contexts does not hurt peformance.
This is most likely due to the trivial (practically absent) freespace management in Gen context, compared to both Slab and GenSlab. So the speed is not the only criteria - I haven't measured memory consumption, but I'm pretty sure there are cases where Slab consumes much less memory than Gen, thanks to reusing free space.
I'd say throwing away GenSlab and keeping Slab+Gen is the way to go.There's still a fair bit of work on this, particularly implementing the missing API methods in Gen - GenCheck() and GenStats(). As Robert pointed out, there's also quite a bit of duplicated code between the different memory contexts (randomization and valgrind-related), so this needs to be moved to a shared place.
I'm also thinking that we need better names, particularly for the Gen allocator. It's supposed to mean Generational, although there are no explicit generations anymore. Slab is probably OK - it does not match any of the existing kernel slab allocators exactly, but it follows the same principles, which is what matters.
regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
slab-allocators-v4.tgz
Description: application/compressed-tar
-- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
