On Fri, Oct 24, 2025 at 8:38 AM Tomas Vondra <[email protected]> wrote:
> Hmmm, I wonder if the m_w_m is high enough to confuse the trimming logic > in some way. Can you try if using smaller m_w_m (maybe 128MB-256MB) > makes the issue go away? > The index builds at up to 4GB of m_w_m. 5GB and above crashes. Now that I know roughly where the limits are that de-escalates things a bit. The sort of customers deploying a month after release should be OK with just knowing to be careful about high m_w_m settings on PG18 until a fix is ready. Hope everyone is enjoying Latvia. My obscure music collection includes a band from there I used to see in the NYC area, The Quags; https://www.youtube.com/watch?v=Bg3P4736CxM Can you show the contents of buffer and tup? I'm especially interested > in these fields: > buffer->nitems > buffer->maxitems > buffer->nfrozen > tup->nitems > I'll see if I can grab that data at the crash point. FYI for anyone who wants to replicate this: if you have a system with 128GB+ of RAM you could probably recreate the test case. Just have to download the Planet file and run osm2pgsql with the overly tweaked settings I use. I've published all the details of how I run this regression test now. Settings: https://github.com/gregs1104/pgbent/tree/main/conf/18/conf.d Script setup: https://github.com/gregs1104/pgbent/blob/main/wl/osm-import Test runner: https://github.com/gregs1104/pgbent/blob/main/util/osm-importer Parse results: https://github.com/gregs1104/pgbent/blob/main/util/pgbench-init-parse > If I'm right, I think there are two ways to fix this: > (1) apply the trimming earlier, i.e. try to freeze + flush before > actually merging the data (essentially, update nfrozen earlier) > (2) use repalloc_huge (and palloc_huge) in GinBufferStoreTuple > Or we probably should do both. Sounds like (2) is probably mandatory and (1) is good hygiene. -- Greg Smith, Software engineering Snowflake - Where Data Does More [email protected]
