On Wed, Mar 6, 2019 at 1:37 PM Robert Haas <robertmh...@gmail.com> wrote: > I know I'm stating the obvious here, but we don't have many weeks left > at this point. I have not reviewed any code, but I have been > following this thread and I'd really like to see this work go into > PostgreSQL 12, assuming it's in good enough shape. It sounds like > really good stuff.
Thanks! Barring any objections, I plan to commit the first 3 patches (plus the amcheck "relocate" patch) within 7 - 10 days (that's almost everything). Heikki hasn't reviewed 'Add high key "continuescan" optimization' yet, and it seems like he should take a look at that before I proceed with it. But that seems like the least controversial enhancement within the entire patch series, so I'm not very worried about it. I'm currently working on v15, which has comment-only revisions requested by Heikki. I expect to continue to work with him to make sure that he is happy with the presentation. I'll also need to revalidate the performance of the patch series following recent minor changes to the logic for choosing a split point. That can take days. This is why I don't want to commit the first patch without committing at least the first three all at once -- it increases the amount of performance validation work I'll have to do considerably. (I have to consider both v4 and v3 indexes already, which seems like enough work.) Two of the later patches (one of which I plan to push as part of the first batch of commits) use heuristics to decide where to split the page. As a Postgres contributor, I have learned to avoid inventing heuristics, so this automatically makes me a bit uneasy. However, I don't feel so bad about it here, on reflection. The on-disk size of the TPC-C indexes are reduced by 35% with the 'Add "split after new tuple" optimization' patch (I think that the entire database is usually about 12% smaller). There simply isn't a fundamentally better way to get the same benefit, and I'm sure that nobody will argue that we should just accept the fact that the most influential database benchmark of all time has a big index bloat problem with Postgres. That would be crazy. That said, it's not impossible that somebody will shout at me because my heuristics made their index bloated. I can't see how that could happen, but I am prepared. I can always adjust the heuristics when new information comes to light. I have fairly thorough test cases that should allow me to do this without regressing anything else. This is a risk that can be managed sensibly. There is no gnawing ambiguity about the on-disk changes laid down in the second patch (nor the first patch), though. Making on-disk changes is always a bit scary, but making the keys unique is clearly a big improvement architecturally, as it brings nbtree closer to the Lehman & Yao design without breaking anything for v3 indexes (v3 indexes simply aren't allowed to use a heap TID in their scankey). Unique keys also allow amcheck to relocate every tuple in the index from the root page, using the same code path as regular index scans. We'll be relying on the uniqueness of keys within amcheck from the beginning, before anybody teaches nbtree to perform retail index tuple deletion. -- Peter Geoghegan
