On Wed, Jun 24, 2020 at 07:18:10PM -0400, Bruce Momjian wrote: > On Wed, Jun 24, 2020 at 12:19:00PM -0700, Andres Freund wrote: > > Hi, > > > > On 2020-06-24 13:12:03 -0400, Bruce Momjian wrote: > > > Well, my point is that merge join works that way, and no one has needed > > > a knob to avoid mergejoin if it is going to spill to disk. If they are > > > adjusting work_mem to prevent spill of merge join, they can do the same > > > for hash agg. We just need to document this in the release notes. > > > > I don't think this is comparable. For starters, the IO indirectly > > triggered by mergejoin actually leads to plenty people just straight out > > disabling it. For lots of workloads there's never a valid reason to use > > a mergejoin (and often the planner will never choose one). Secondly, the > > planner has better information about estimating the memory usage for the > > to-be-sorted data than it has about the size of the transition > > values. And lastly, there's a difference between a long existing cause > > for bad IO behaviour and one that's suddenly kicks in after a major > > version upgrade, to which there's no escape hatch (it's rarely realistic > > to disable hash aggs, in contrast to merge joins). > > Well, this sounds like an issue of degree, rather than kind. It sure > sounds like "ignore work_mem for this join type, but not the other".
I think my main point is that work_mem was not being honored for hash-agg before, but now that PG 13 can do it, we are again allowing work_mem not to apply in certain cases. I am wondering if our hard limit for work_mem is the issue, and we should make that more flexible for all uses. -- Bruce Momjian <br...@momjian.us> https://momjian.us EnterpriseDB https://enterprisedb.com The usefulness of a cup is in its emptiness, Bruce Lee
