Re: Improving btree performance through specializing by key shape, take 2

2024-11-16 Thread Dmitry Dolgov
> On Mon, Mar 04, 2024 at 09:39:37PM GMT, Matthias van de Meent wrote: > Performance > > > I haven't retested the results separately yet, but I assume the > performance results of [2] hold mostly true in comparing 0002 vs 0007. > I will do a performance (re-)evaluation of only this patch i

Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2024-11-13 Thread Peter Geoghegan
On Wed, Nov 13, 2024 at 3:30 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Tue, Aug 13, 2024 at 02:39:10PM GMT, Peter Geoghegan wrote: > > On Tue, Aug 6, 2024 at 5:42 PM Matthias van de Meent > > To be clear, this test involves bulk loading of an unlogged table (the > > land registry table

Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2024-11-13 Thread Dmitry Dolgov
> On Tue, Aug 13, 2024 at 02:39:10PM GMT, Peter Geoghegan wrote: > On Tue, Aug 6, 2024 at 5:42 PM Matthias van de Meent > wrote: > > Attached is version 16 now. > > I ran this with my old land registry benchmark, used for validating > the space utilization impact of nbtree deduplication (among oth

Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2024-08-13 Thread Peter Geoghegan
On Tue, Aug 6, 2024 at 5:42 PM Matthias van de Meent wrote: > Attached is version 16 now. I ran this with my old land registry benchmark, used for validating the space utilization impact of nbtree deduplication (among other things). This isn't obviously the best benchmark for this sort of thing,

Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2024-08-06 Thread Matthias van de Meent
On Fri, 1 Mar 2024 at 14:48, Matthias van de Meent wrote: > Attached is version 15 of this patch, with the above issues fixed. > It's also rebased on top of 655dc310 of this morning, so that should > keep good for some time again. Attached is version 16 now. Relevant changes from previous patch v

Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2024-03-01 Thread Matthias van de Meent
On Wed, 24 Jan 2024 at 13:02, Matthias van de Meent wrote: > > 1. > > Commit message refers to a non-existing reference '(see [0])'. > > Noted, I'll update that. > > > 2. > > +When we do a binary search on a sorted set (such as a BTree), we know that > > a > > +tuple will be smaller than its left

Re: Improving btree performance through specializing by key shape, take 2

2024-01-26 Thread vignesh C
On Mon, 30 Oct 2023 at 21:50, Matthias van de Meent wrote: > > On Thu, 26 Oct 2023 at 00:36, Peter Geoghegan wrote: > > Most of the work with something like > > this is the analysis of the trade-offs, not writing code. There are > > all kinds of trade-offs that you could make with something like

Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2024-01-24 Thread Matthias van de Meent
On Fri, 19 Jan 2024 at 05:55, Dilip Kumar wrote: > > On Wed, Nov 1, 2023 at 2:42 AM Matthias van de Meent > wrote: > > > > Hi, > > > > Currently, nbtree code compares each and every column of an index > > tuple during the binary search on the index page. With large indexes > > that have many dupl

Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2024-01-18 Thread Dilip Kumar
On Wed, Nov 1, 2023 at 2:42 AM Matthias van de Meent wrote: > > Hi, > > Currently, nbtree code compares each and every column of an index > tuple during the binary search on the index page. With large indexes > that have many duplicate prefix column values (e.g. an index on (bool, > bool, uuid) )

Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2023-11-01 Thread Pavel Stehule
st 1. 11. 2023 v 11:32 odesílatel Matthias van de Meent < boekewurm+postg...@gmail.com> napsal: > On Wed, 1 Nov 2023 at 07:47, Pavel Stehule > wrote: > > > > Hi > > > > út 31. 10. 2023 v 22:12 odesílatel Matthias van de Meent < > boekewurm+postg...@gmail.com> napsal: > >> This patch was originall

Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2023-11-01 Thread Matthias van de Meent
On Wed, 1 Nov 2023 at 07:47, Pavel Stehule wrote: > > Hi > > út 31. 10. 2023 v 22:12 odesílatel Matthias van de Meent > napsal: >> This patch was originally suggested at [0], but it was mentioned that >> they could be pulled out into it's own thread. Earlier, the >> performance gains were not cl

Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2023-10-31 Thread Pavel Stehule
Hi út 31. 10. 2023 v 22:12 odesílatel Matthias van de Meent < boekewurm+postg...@gmail.com> napsal: > Hi, > > Currently, nbtree code compares each and every column of an index > tuple during the binary search on the index page. With large indexes > that have many duplicate prefix column values (e

btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2023-10-31 Thread Matthias van de Meent
Hi, Currently, nbtree code compares each and every column of an index tuple during the binary search on the index page. With large indexes that have many duplicate prefix column values (e.g. an index on (bool, bool, uuid) ) that means a lot of wasted time getting to the right column. The attached

Re: Improving btree performance through specializing by key shape, take 2

2023-10-25 Thread Peter Geoghegan
On Mon, Sep 25, 2023 at 9:13 AM Matthias van de Meent wrote: > I think it's fairly complete, and mostly waiting for review. > > > I don't have time to do a comprehensive (or even a fairly > > cursory) analysis of which parts of the patch are helping, and which > > are marginal or even add no value

Re: Improving btree performance through specializing by key shape, take 2

2023-09-25 Thread Matthias van de Meent
On Tue, 19 Sept 2023 at 22:49, Peter Geoghegan wrote: > > On Tue, Sep 19, 2023 at 6:28 AM Matthias van de Meent > wrote: > > > To be clear, page deletion does what I described here (it does an > > > in-place update of the downlink to the deleted page, so the same pivot > > > tuple now points to i

Re: Improving btree performance through specializing by key shape, take 2

2023-09-19 Thread Peter Geoghegan
On Tue, Sep 19, 2023 at 6:28 AM Matthias van de Meent wrote: > > To be clear, page deletion does what I described here (it does an > > in-place update of the downlink to the deleted page, so the same pivot > > tuple now points to its right sibling, which is our page of concern), > > in addition to

Re: Improving btree performance through specializing by key shape, take 2

2023-09-19 Thread Matthias van de Meent
On Tue, 19 Sept 2023 at 03:56, Peter Geoghegan wrote: > > On Mon, Sep 18, 2023 at 6:29 PM Peter Geoghegan wrote: > > I also have significant doubts about your scheme for avoiding > > invalidating the bounds of the page based on its high key matching the > > parent's separator. The subtle dynamic

Re: Improving btree performance through specializing by key shape, take 2

2023-09-18 Thread Peter Geoghegan
On Mon, Sep 18, 2023 at 8:57 AM Matthias van de Meent wrote: > > Rebased again to v13 to account for API changes in 9f060253 "Remove > > some more "snapshot too old" vestiges." > > ... and now attached. I see that this revised version is approximately as invasive as earlier versions were - it has

Re: Improving btree performance through specializing by key shape, take 2

2023-09-18 Thread Peter Geoghegan
On Mon, Sep 18, 2023 at 6:29 PM Peter Geoghegan wrote: > I also have significant doubts about your scheme for avoiding > invalidating the bounds of the page based on its high key matching the > parent's separator. The subtle dynamic prefix compression race > condition that I was worried about was

Re: Improving btree performance through specializing by key shape, take 2

2023-09-18 Thread Matthias van de Meent
On Wed, 30 Aug 2023 at 21:50, Matthias van de Meent wrote: > > Updated in the attached version 12 of the patchset (which is also > rebased on HEAD @ 9c13b681). No changes apart from rebase fixes and > these added comments. Rebased again to v13 to account for API changes in 9f060253 "Remove some m

Re: Improving btree performance through specializing by key shape, take 2

2023-06-26 Thread Dilip Kumar
On Tue, Jun 27, 2023 at 9:42 AM Dilip Kumar wrote: > > On Fri, Jun 23, 2023 at 8:16 PM Matthias van de Meent > wrote: > > > > On Fri, 23 Jun 2023 at 11:26, Dilip Kumar wrote: > > > > > > and I have one confusion in the > > > below hunk in the _bt_moveright function, basically, if the parent > >

Re: Improving btree performance through specializing by key shape, take 2

2023-06-26 Thread Dilip Kumar
On Fri, Jun 23, 2023 at 8:16 PM Matthias van de Meent wrote: > > On Fri, 23 Jun 2023 at 11:26, Dilip Kumar wrote: > > > and I have one confusion in the > > below hunk in the _bt_moveright function, basically, if the parent > > page's right key is exactly matching the HIGH key of the child key >

Re: Improving btree performance through specializing by key shape, take 2

2023-06-23 Thread Matthias van de Meent
On Fri, 23 Jun 2023 at 11:26, Dilip Kumar wrote: > > On Fri, Jun 23, 2023 at 2:21 AM Matthias van de Meent > wrote: > > > > > == Dynamic prefix truncation (0001) > > The code now tracks how many prefix attributes of the scan key are > > already considered equal based on earlier binsrch results, a

Re: Improving btree performance through specializing by key shape, take 2

2023-06-23 Thread Dilip Kumar
On Fri, Jun 23, 2023 at 2:21 AM Matthias van de Meent wrote: > > == Dynamic prefix truncation (0001) > The code now tracks how many prefix attributes of the scan key are > already considered equal based on earlier binsrch results, and ignores > those prefix colums in further binsrch operations (s

Re: Improving btree performance through specializing by key shape, take 2

2023-04-04 Thread Gregory Stark (as CFM)
Hm. The cfbot has a fairly trivial issue with this with a unused variable: 18:36:17.405] In file included from ../../src/include/access/nbtree.h:1184, [18:36:17.405] from verify_nbtree.c:27: [18:36:17.405] verify_nbtree.c: In function ‘palloc_btree_page’: [18:36:17.405] ../../src/include/access/nb

Re: Improving btree performance through specializing by key shape, take 2

2023-01-12 Thread David Christensen
Hi Matthias, I'm going to look at this patch series if you're still interested. What was the status of your final performance testing for the 0008 patch alone vs the specialization series? Last I saw on the thread you were going to see if the specialization was required or not. Best, David

Re: Improving btree performance through specializing by key shape, take 2

2022-10-31 Thread Matthias van de Meent
On Wed, 27 Jul 2022 at 13:34, Matthias van de Meent wrote: > > On Wed, 27 Jul 2022 at 09:35, Matthias van de Meent > wrote: > > > > On Mon, 4 Jul 2022 at 16:18, Matthias van de Meent > > wrote: > > > > > > On Sun, 5 Jun 2022 at 21:12, Matthias van de Meent > > > wrote: > > > > While working on

Re: Improving btree performance through specializing by key shape, take 2

2022-04-10 Thread Peter Geoghegan
On Sun, Apr 10, 2022 at 4:08 PM Matthias van de Meent wrote: > I'll send an updated patchset soon (I'm planning on moving around when > what is changed/added); but before that the attached incremental patch > should help. FYI, the patchset has been tested on commit 05023a23, and > compiles (with u

Re: Improving btree performance through specializing by key shape, take 2

2022-04-10 Thread Matthias van de Meent
On Sun, 10 Apr 2022 at 23:45, Peter Geoghegan wrote: > > On Fri, Apr 8, 2022 at 9:55 AM Matthias van de Meent > wrote: > > Here's generation 2 of this effort. Instead of proceeding to trying to > > shoehorn all types of btree keys into one common code path, this > > patchset acknowledges that the

Re: Improving btree performance through specializing by key shape, take 2

2022-04-10 Thread Peter Geoghegan
On Sun, Apr 10, 2022 at 2:44 PM Peter Geoghegan wrote: > Can you post a version of this that compiles? I forgot to add: the patch also bitrot due to recent commit dbafe127. I didn't get stuck at this point (this is minor bitrot), but no reason not to rebase. -- Peter Geoghegan

Re: Improving btree performance through specializing by key shape, take 2

2022-04-10 Thread Peter Geoghegan
On Fri, Apr 8, 2022 at 9:55 AM Matthias van de Meent wrote: > Here's generation 2 of this effort. Instead of proceeding to trying to > shoehorn all types of btree keys into one common code path, this > patchset acknowledges that there exist different shapes of keys that > each have a different bes