On Fri, 20 Sept 2024 at 17:46, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Fri, Sep 20, 2024 at 5:13 AM David Rowley <dgrowle...@gmail.com> wrote: > > In general, it's a bit annoying to have to code around this > > GenerationContext fragmentation issue. > > Right, and I am also slightly afraid that this may not cause some > regression in other cases where defrag wouldn't help.
Yeah, that's certainly a possibility. I was hoping that MemoryContextMemAllocated() being much larger than logical_work_mem could only happen when there is fragmentation, but certainly, you could be wasting effort trying to defrag transactions where the changes all arrive in WAL consecutively and there is no defragmentation. It might be some other large transaction that's causing the context's allocations to be fragmented. I don't have any good ideas on how to avoid wasting effort on non-problematic transactions. Maybe there's something that could be done if we knew the LSN of the first and last change and the gap between the LSNs was much larger than the WAL space used for this transaction. That would likely require tracking way more stuff than we do now, however. With the smaller blocks idea, I'm a bit concerned that using smaller blocks could cause regressions on systems that are better at releasing memory back to the OS after free() as no doubt malloc() would often be slower on those systems. There have been some complaints recently about glibc being a bit too happy to keep hold of memory after free() and I wondered if that was the reason why the small block test does not cause much of a performance regression. I wonder how the small block test would look on Mac, FreeBSD or Windows. I think it would be risky to assume that all is well with reducing the block size after testing on a single platform. David
