Re: Index Skip Scan (new UniqueKeys)

2022-03-24 Thread Jesper Pedersen
On 6/9/20 06:22, Dmitry Dolgov wrote: Here is a new version of index skip scan patch, based on v8 patch for UniqueKeys implementation from [1]. I want to start a new thread to simplify navigation, hopefully I didn't forget anyone who actively participated in the discussion. This CommitFest ent

Re: Index Skip Scan (new UniqueKeys)

2022-01-24 Thread Zhihong Yu
On Mon, Jan 24, 2022 at 8:51 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Sun, Jan 23, 2022 at 04:25:04PM -0800, Zhihong Yu wrote: > > On Sat, Jan 22, 2022 at 1:32 PM Dmitry Dolgov <9erthali...@gmail.com> > wrote: > > > Besides that the new patch version contains some cleaning up and > >

Re: Index Skip Scan (new UniqueKeys)

2022-01-24 Thread Dmitry Dolgov
> On Sun, Jan 23, 2022 at 04:25:04PM -0800, Zhihong Yu wrote: > On Sat, Jan 22, 2022 at 1:32 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Besides that the new patch version contains some cleaning up and > > addresses commentaries around leaf page pinning from [1]. The idea > > behind the ser

Re: Index Skip Scan (new UniqueKeys)

2022-01-23 Thread Zhihong Yu
On Sat, Jan 22, 2022 at 1:32 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Fri, May 21, 2021 at 05:31:38PM +0200, Dmitry Dolgov wrote: > > Hi, > > > > Here is another take on the patch with a couple of changes: > > > > * I've removed for now UniqueKeys parts. The interaction of skip scan

Re: Index Skip Scan (new UniqueKeys)

2022-01-22 Thread Dmitry Dolgov
> On Fri, May 21, 2021 at 05:31:38PM +0200, Dmitry Dolgov wrote: > Hi, > > Here is another take on the patch with a couple of changes: > > * I've removed for now UniqueKeys parts. The interaction of skip scan & > unique keys patch was actually not that big, so the main difference is > that now

Re: Index Skip Scan (new UniqueKeys)

2021-05-21 Thread Dmitry Dolgov
Hi, Here is another take on the patch with a couple of changes: * I've removed for now UniqueKeys parts. The interaction of skip scan & unique keys patch was actually not that big, so the main difference is that now the structure itself went away, a list of unique expressions is used instea

Re: Index Skip Scan (new UniqueKeys)

2021-03-17 Thread Tomas Vondra
On 3/17/21 6:02 PM, Dmitry Dolgov wrote: >> On Wed, Mar 17, 2021 at 03:28:00AM +0100, Tomas Vondra wrote: >> Hi, >> >> I took a look at the new patch series, focusing mostly on the uniquekeys >> part. It'd be a bit tedious to explain all the review comments here, so >> attached is a patch series

Re: Index Skip Scan (new UniqueKeys)

2021-03-17 Thread Dmitry Dolgov
> On Wed, Mar 17, 2021 at 03:28:00AM +0100, Tomas Vondra wrote: > Hi, > > I took a look at the new patch series, focusing mostly on the uniquekeys > part. It'd be a bit tedious to explain all the review comments here, so > attached is a patch series with a "review" patch for some of the parts. Gre

Re: Index Skip Scan (new UniqueKeys)

2021-01-28 Thread Dmitry Dolgov
> On Thu, Jan 28, 2021 at 09:49:26PM +0900, Masahiko Sawada wrote: > Hi Dmitry, > > Status update for a commitfest entry. > > This patch entry has been "Waiting on Author" on CF app and the > discussion seems inactive from the last CF. Could you share the > current status of this patch? Heikki alre

Re: Index Skip Scan (new UniqueKeys)

2021-01-28 Thread Masahiko Sawada
Hi Dmitry, On Sun, Oct 25, 2020 at 1:45 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Tue, Oct 06, 2020 at 05:20:39PM +0200, Dmitry Dolgov wrote: > > > On Mon, Sep 21, 2020 at 05:59:32PM -0700, Peter Geoghegan wrote: > > > > > > * I see the following compiler warning: > > > > > > /code/

Re: Index Skip Scan (new UniqueKeys)

2020-12-05 Thread Thomas Munro
On Wed, Dec 2, 2020 at 9:59 AM Heikki Linnakangas wrote: > On 01/12/2020 22:21, Dmitry Dolgov wrote: > >> On Mon, Nov 30, 2020 at 04:42:20PM +0200, Heikki Linnakangas wrote: > >> - I'm surprised you need a new index AM function (amskip) for this. Can't > >> you just restart the scan with index_res

Re: Index Skip Scan (new UniqueKeys)

2020-12-05 Thread Dmitry Dolgov
> On Tue, Dec 01, 2020 at 10:59:22PM +0200, Heikki Linnakangas wrote: > > > > - Does this optimization apply to bitmap index scans? > > > > No, from what I understand it doesn't. > > Would it be hard to add? Don't need to solve everything in the first > version of this, but I think in principle you

Re: Index Skip Scan (new UniqueKeys)

2020-12-01 Thread Heikki Linnakangas
On 01/12/2020 22:21, Dmitry Dolgov wrote: On Mon, Nov 30, 2020 at 04:42:20PM +0200, Heikki Linnakangas wrote: I had a quick look at this patch. I haven't been following this thread, so sorry if I'm repeating old arguments, but here we go: Thanks! - I'm surprised you need a new index AM funct

Re: Index Skip Scan (new UniqueKeys)

2020-12-01 Thread Dmitry Dolgov
> On Mon, Nov 30, 2020 at 04:42:20PM +0200, Heikki Linnakangas wrote: > > I had a quick look at this patch. I haven't been following this thread, so > sorry if I'm repeating old arguments, but here we go: Thanks! > - I'm surprised you need a new index AM function (amskip) for this. Can't > you ju

Re: Index Skip Scan (new UniqueKeys)

2020-11-30 Thread Heikki Linnakangas
On 24/10/2020 19:45, Dmitry Dolgov wrote: Here is a new version which doesn't require "scanstart" argument and contains few other changes to address the issues mentioned earlier. It's also based on the latest UniqueKeys patches with the valgrind issue fixed (as before they're attached also just f

Re: Index Skip Scan (new UniqueKeys)

2020-10-06 Thread Dmitry Dolgov
> On Mon, Sep 21, 2020 at 05:59:32PM -0700, Peter Geoghegan wrote: > > * I see the following compiler warning: > > /code/postgresql/patch/build/../source/src/backend/optimizer/path/uniquekeys.c: > In function ‘populate_baserel_uniquekeys’: > /code/postgresql/patch/build/../source/src/backend/optimi

Re: Index Skip Scan (new UniqueKeys)

2020-09-21 Thread Peter Geoghegan
On Mon, Sep 21, 2020 at 5:59 PM Peter Geoghegan wrote: > That's all I have for now. One more thing. I don't think that this should be a bitwise AND: if ((offnum > maxoff) & (so->currPos.nextPage == P_NONE)) {

Re: Index Skip Scan (new UniqueKeys)

2020-09-21 Thread Peter Geoghegan
On Sat, Aug 15, 2020 at 7:09 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > Here is a new version that hopefully address most of the concerns > mentioned in this thread so far. As before, first two patches are taken > from UniqueKeys thread and attached only for the reference. List of > changes

Re: Index Skip Scan (new UniqueKeys)

2020-08-02 Thread Andy Fan
Hi David: Thanks for looking into this. On Fri, Jul 31, 2020 at 11:07 AM David Rowley wrote: > On Mon, 13 Jul 2020 at 10:18, Floris Van Nee > wrote: > > One question about the unique keys - probably for Andy or David: I've > looked in the archives to find arguments for/against using Expr nodes

Re: Index Skip Scan (new UniqueKeys)

2020-07-30 Thread David Rowley
On Mon, 13 Jul 2020 at 10:18, Floris Van Nee wrote: > One question about the unique keys - probably for Andy or David: I've looked > in the archives to find arguments for/against using Expr nodes or > EquivalenceClasses in the Unique Keys patch. However, I couldn't really find > a clear answer

Re: Index Skip Scan (new UniqueKeys)

2020-07-27 Thread Dmitry Dolgov
> On Tue, Jul 21, 2020 at 04:35:55PM -0700, Peter Geoghegan wrote: > > > Well, it's obviously wrong, thanks for noticing. What is necessary is to > > compare two index tuples, the start and the next one, to test if they're > > the same (in which case if I'm not mistaken probably we can compare item

RE: Index Skip Scan (new UniqueKeys)

2020-07-23 Thread Floris Van Nee
> > One UniqueKey can have multiple corresponding expressions, which gives us > also possibility of having one unique key with (t1.a, t2.a) and it looks now > similar to EquivalenceClass. > I believe the current definition of a unique key with two expressions (t1.a, t2.a) means that it's unique

Re: Index Skip Scan (new UniqueKeys)

2020-07-23 Thread Dmitry Dolgov
> On Tue, Jul 14, 2020 at 06:18:50PM +, Floris Van Nee wrote: > > Due to the other changes I made in > create_distinct_paths/query_has_uniquekeys_for, it will choose a correct plan > now, even without the EC_MUST_BE_REDUNDANT check though, so it's difficult to > give an actual failing test c

Re: Index Skip Scan (new UniqueKeys)

2020-07-21 Thread Peter Geoghegan
On Sat, Jul 11, 2020 at 9:10 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > + currItem = &so->currPos.items[so->currPos.lastItem]; > > > + itup = (IndexTuple) (so->currTuples + > > > currItem->tupleOffset); > > > + nextOffset = ItemPointerGetOffsetN

RE: Index Skip Scan (new UniqueKeys)

2020-07-14 Thread Floris Van Nee
> > > - the uniquekeys that is built, still contains some redundant keys, that are > normally eliminated from the path keys lists. > > I guess you're talking about: > > + if (EC_MUST_BE_REDUNDANT(ec)) > + continue; > > Can you add some test cases to your changes to show the effect of it? It >

Re: Index Skip Scan (new UniqueKeys)

2020-07-14 Thread Dmitry Dolgov
> On Sun, Jul 12, 2020 at 12:48:47PM +, Floris Van Nee wrote: > > > > Good point, thanks for looking at this. With the latest planner version > > there > > are indeed more possibilities to use skipping. It never occured to me that > > some of those paths will still rely on index scan returning

Re: Index Skip Scan (new UniqueKeys)

2020-07-11 Thread Dmitry Dolgov
> On Fri, Jul 10, 2020 at 05:03:37PM +, Floris Van Nee wrote: > > Also took another look at the patch now, and found a case of incorrect > data. It looks related to the new way of creating the paths, as I > can't recall seeing this in earlier versions. > > create table t1 as select a,b,b%5 as c

Re: Index Skip Scan (new UniqueKeys)

2020-07-11 Thread Dmitry Dolgov
> On Wed, Jul 08, 2020 at 03:44:26PM -0700, Peter Geoghegan wrote: > > On Tue, Jun 9, 2020 at 3:20 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > * Btree-implementation contains btree specific code to implement amskip, > > introduced in the previous patch. > > The way that you're dealing wit

RE: Index Skip Scan (new UniqueKeys)

2020-07-10 Thread Floris Van Nee
Hi Dmitry, Also took another look at the patch now, and found a case of incorrect data. It looks related to the new way of creating the paths, as I can't recall seeing this in earlier versions. create table t1 as select a,b,b%5 as c, random() as d from generate_series(1, 10) a, generate_series

Re: Index Skip Scan (new UniqueKeys)

2020-07-08 Thread Peter Geoghegan
On Tue, Jun 9, 2020 at 3:20 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > * Btree-implementation contains btree specific code to implement amskip, > introduced in the previous patch. The way that you're dealing with B-Tree tuples here needs to account for posting list tuples: > +

Re: Index Skip Scan (new UniqueKeys)

2020-06-29 Thread Dmitry Dolgov
> On Thu, Jun 11, 2020 at 04:14:07PM +0800, Andy Fan wrote: > > I just get the rough idea of patch, looks we have to narrow down the > user cases where we can use this method. Consider the below example: Hi Not exactly narrow down, but rather get rid of wrong usage of skipping for index scan. Sin

Re: Index Skip Scan (new UniqueKeys)

2020-06-11 Thread Andy Fan
On Tue, Jun 9, 2020 at 6:20 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > Hi, > > Here is a new version of index skip scan patch, based on v8 patch for > UniqueKeys implementation from [1]. I want to start a new thread to > simplify navigation, hopefully I didn't forget anyone who actively > p