Re: MDAM techniques and Index Skip Scan patch

2022-03-28 Thread Peter Geoghegan
On Mon, Mar 28, 2022 at 7:07 PM Tom Lane wrote: > Right, that's the case I had in mind --- apologies if my terminology > was faulty. btree can actually handle such a case now, but what it > fails to do is re-descend from the tree root instead of plowing > forward in the index to find the next mat

Re: MDAM techniques and Index Skip Scan patch

2022-03-28 Thread Tom Lane
Peter Geoghegan writes: > The terminology in this area is a mess. MySQL calls > SELECT-DISTINCT-that-matches-an-index "loose index scans". I think > that you're talking about skip scan when you say "loose index scan". > Skip scan is where there is an omitted prefix of columns in the SQL > query --

Re: MDAM techniques and Index Skip Scan patch

2022-03-28 Thread Peter Geoghegan
On Mon, Mar 28, 2022 at 5:21 PM Tom Lane wrote: > In any case, what I was on about is _bt_preprocess_keys() and > adjacent code. I'm surprised that those aren't more expensive > than one palloc in _bt_first. Maybe that logic falls through very > quickly in simple cases, though. I assume that it

Re: MDAM techniques and Index Skip Scan patch

2022-03-28 Thread Peter Geoghegan
On Tue, Mar 22, 2022 at 1:55 PM Tom Lane wrote: > Peter asked me off-list to spend some time thinking about the overall > direction we ought to be pursuing here. Thanks for taking a look! "5.5 Exploiting Key Prefixes" and "5.6 Ordered Retrieval" from "Modern B-Tree Techniques" are also good, BTW

Re: MDAM techniques and Index Skip Scan patch

2022-03-28 Thread Tom Lane
Peter Geoghegan writes: > We could get rid of dynamic allocations for BTStackData in > _bt_first(), perhaps. The problem is that there is no simple, > reasonable proof of the maximum height on a B-tree, even though a > B-Tree with more than 7 or 8 levels seems extraordinarily unlikely. Start with

Re: MDAM techniques and Index Skip Scan patch

2022-03-28 Thread Peter Geoghegan
On Tue, Mar 22, 2022 at 4:06 PM Andres Freund wrote: > Are you thinking of just moving the setup stuff in nbtree (presumably parts of > _bt_first() / _bt_preprocess_keys()) or also stuff in > ExecIndexBuildScanKeys()? > > The latter does show up a bit more heavily in profiles than nbtree specific

Re: MDAM techniques and Index Skip Scan patch

2022-03-24 Thread Jesper Pedersen
On 3/23/22 18:22, Dmitry Dolgov wrote: The CF item could be set to RwF, what would you say, Jesper? We want to thank the community for the feedback that we have received over the years for this feature. Hopefully a future implementation can use Tom's suggestions to get closer to a committab

Re: MDAM techniques and Index Skip Scan patch

2022-03-23 Thread Dmitry Dolgov
> On Wed, Mar 23, 2022 at 05:32:46PM -0400, Tom Lane wrote: > Dmitry Dolgov <9erthali...@gmail.com> writes: > > On Tue, Mar 22, 2022 at 04:55:49PM -0400, Tom Lane wrote: > >> In short: I would throw out just about all the planner infrastructure > >> that's been proposed so far. It looks bulky, exp

Re: MDAM techniques and Index Skip Scan patch

2022-03-23 Thread Tom Lane
Dmitry Dolgov <9erthali...@gmail.com> writes: > On Tue, Mar 22, 2022 at 04:55:49PM -0400, Tom Lane wrote: >> In short: I would throw out just about all the planner infrastructure >> that's been proposed so far. It looks bulky, expensive, and >> drastically undercommented, and I don't think it's bu

Re: MDAM techniques and Index Skip Scan patch

2022-03-23 Thread Dmitry Dolgov
> On Tue, Mar 22, 2022 at 04:55:49PM -0400, Tom Lane wrote: > Peter Geoghegan writes: > > Like many difficult patches, the skip scan patch is not so much > > troubled by problems with the implementation as it is troubled by > > *ambiguity* about the design. Particularly concerning how skip scan >

Re: MDAM techniques and Index Skip Scan patch

2022-03-22 Thread Tom Lane
Andres Freund writes: > On 2022-03-22 16:55:49 -0400, Tom Lane wrote: >> BTW, I've had a bee in my bonnet for a long time about whether some of >> nbtree's scan setup work could be done once during planning, rather than >> over again during each indexscan start. > It does show up in simple-index-

Re: MDAM techniques and Index Skip Scan patch

2022-03-22 Thread Andres Freund
Hi, On 2022-03-22 16:55:49 -0400, Tom Lane wrote: > 4. I find each of the above ideas to be far more attractive than > optimizing SELECT-DISTINCT-that-matches-an-index, so I don't really > understand why the current patchsets seem to be driven largely > by that single use-case. It's something cau

Re: MDAM techniques and Index Skip Scan patch

2022-03-22 Thread Tom Lane
Peter Geoghegan writes: > Like many difficult patches, the skip scan patch is not so much > troubled by problems with the implementation as it is troubled by > *ambiguity* about the design. Particularly concerning how skip scan > meshes with existing designs, as well as future designs -- > particu

Re: MDAM techniques and Index Skip Scan patch

2022-03-22 Thread Thomas Munro
On Tue, Mar 22, 2022 at 2:34 PM Andres Freund wrote: > IMO it's pretty clear that having "duelling" patches below one CF entry is a > bad idea. I think they should be split, with inactive approaches marked as > returned with feeback or whatnot. I have the impression that this thread is getting so

Re: MDAM techniques and Index Skip Scan patch

2022-03-22 Thread Dmitry Dolgov
> On Mon, Mar 21, 2022 at 06:34:09PM -0700, Andres Freund wrote: > > > I don't mind having all of the alternatives under the same CF item, only > > one being tested seems to be only a small limitation of cfbot. > > IMO it's pretty clear that having "duelling" patches below one CF entry is a > bad i

Re: MDAM techniques and Index Skip Scan patch

2022-03-21 Thread Andres Freund
Hi, On 2022-01-22 22:37:19 +0100, Dmitry Dolgov wrote: > > On Fri, Jan 14, 2022 at 04:03:41PM +0800, Julien Rouhaud wrote: > > Hi, > > > > On Fri, Jan 14, 2022 at 08:55:26AM +0100, Dmitry Dolgov wrote: > > > > > > FYI, I've attached this thread to the CF item as an informational one, > > > but as

Re: MDAM techniques and Index Skip Scan patch

2022-01-22 Thread Dmitry Dolgov
> On Fri, Jan 14, 2022 at 04:03:41PM +0800, Julien Rouhaud wrote: > Hi, > > On Fri, Jan 14, 2022 at 08:55:26AM +0100, Dmitry Dolgov wrote: > > > > FYI, I've attached this thread to the CF item as an informational one, > > but as there are some patches posted here, folks may get confused. For > > th

Re: MDAM techniques and Index Skip Scan patch

2022-01-14 Thread Julien Rouhaud
Hi, On Fri, Jan 14, 2022 at 08:55:26AM +0100, Dmitry Dolgov wrote: > > FYI, I've attached this thread to the CF item as an informational one, > but as there are some patches posted here, folks may get confused. For > those who have landed here with no context, I feel obliged to mention > that now

Re: MDAM techniques and Index Skip Scan patch

2022-01-13 Thread Dmitry Dolgov
> On Thu, Jan 13, 2022 at 03:27:08PM +, Floris Van Nee wrote: > > > > Could you send a rebased version? In the meantime I will change the status > > on the cf app to Waiting on Author. > > Attached a rebased version. FYI, I've attached this thread to the CF item as an informational one, but a

Re: MDAM techniques and Index Skip Scan patch

2022-01-13 Thread Julien Rouhaud
Hi, On Sat, Oct 23, 2021 at 07:30:47PM +, Floris Van Nee wrote: > > From the patch series above, v9-0001/v9-0002 is the UniqueKeys patch series, > which focuses on the planner. v2-0001 is Dmitry's patch that extends it to > make it possible to use UniqueKeys for the skip scan. Both of these a

Re: MDAM techniques and Index Skip Scan patch

2021-10-25 Thread Jesper Pedersen
Hi Peter, On 10/21/21 22:16, Peter Geoghegan wrote: I returned to the 1995 paper "Efficient Search of Multidimensional B-Trees" [1] as part of the process of reviewing v39 of the skip scan patch, which was posted back in May. It's a great paper, and anybody involved in the skip scan effort shoul

Re: MDAM techniques and Index Skip Scan patch

2021-10-23 Thread Dmitry Dolgov
> On Thu, Oct 21, 2021 at 07:16:00PM -0700, Peter Geoghegan wrote: > > My general concern is that the skip scan patch may currently be > structured in a way that paints us into a corner, MDAM-wise. > > Note that the MDAM paper treats skipping a prefix of columns as a case > where the prefix is hand