On Wed, Dec 23, 2015 at 7:48 PM, Peter Geoghegan <p...@heroku.com> wrote: > I wonder, what's the situation here like with the attached patch > applied on top of what you were testing? I think that we might be > better off with more merge steps when under enormous memory pressure > at the low end, in order to be able to store more tuples per tape (and > do more sorting using quicksort).
Actually, now that I look into it, I think your 64MB work_mem setting would have 234 tapes in total, so my patch won't do anything for your case. Maybe change MAXORDER to 100 within the patch, to see where that leaves things? I want to see if there is any improvement. 234 tapes means that approximately 5.7MB of memory would go to just using tapes (for accounting purposes, which is mostly my concern here). However, for a case like this, where you're well short of being able to do everything in one pass, there is no benefit to having more than about 6 tapes (I guess that's probably still true these days). That 5.7MB of tape space for accounting purposes (and also in reality) may not only increase the amount of random I/O required, and not only throw off the memtuples estimate within grow_memtuples() (its balance against everything else), but also decrease the cache efficiency in the final on-the-fly merge (the efficiency in accessing tuples). -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
