Re: Invisible Indexes

2018-07-08 Thread Jeff Janes
On Mon, Jun 18, 2018 at 5:57 PM, Tom Lane wrote: > > I'm not sure about the "enforce constraint only" argument --- that > sounds like a made-up use-case to me. It's pretty hard to imagine > a case where a unique index applies to a query and yet you don't want > to use it. > I've not seen it wi

Re: Invisible Indexes

2018-07-04 Thread Peter Geoghegan
On Wed, Jul 4, 2018 at 7:09 PM, David Rowley wrote: > hmm. Maybe I missed any other use case. The mention of hypothetical > indexes seems a bit lost on this thread. Andrew's proposal mentions > that an invisible index will just not be considered by the planner. > I'd very much assume here that th

Re: Invisible Indexes

2018-07-04 Thread David Rowley
On 5 July 2018 at 13:31, Peter Geoghegan wrote: > On Wed, Jul 4, 2018 at 6:26 PM, David Rowley > wrote: >> Or would it be insanely weird to just not allow setting or unsetting >> this invisible flag if indcheckxmin is true? I can't imagine there >> will be many people adding an index and not wan

Re: Invisible Indexes

2018-07-04 Thread Peter Geoghegan
On Wed, Jul 4, 2018 at 6:26 PM, David Rowley wrote: > Or would it be insanely weird to just not allow setting or unsetting > this invisible flag if indcheckxmin is true? I can't imagine there > will be many people adding an index and not wanting to use it while > it's still being created. I thin

Re: Invisible Indexes

2018-07-04 Thread David Rowley
On 19 June 2018 at 09:56, Andres Freund wrote: > Be careful about that - currently it's not actually trivially possible > to ever update pg_index rows. No, I'm not kidding > you. pg_index.indexcheckxmin relies on the pg_index row's xmin. If you > have ALTER do a non inplace update, you'll break th

Re: Invisible Indexes

2018-06-24 Thread Bruce Momjian
On Sun, Jun 24, 2018 at 09:59:15AM -0400, Tom Lane wrote: > Andrew Dunstan writes: > > A major downside to a GUC is that you have to be aware of the current > > setting, since we're not going to have one settoing for each invisible > > index. Doing it at the SQL level you can treat each index se

Re: Invisible Indexes

2018-06-24 Thread Tom Lane
Andrew Dunstan writes: > A major downside to a GUC is that you have to be aware of the current > setting, since we're not going to have one settoing for each invisible > index. Doing it at the SQL level you can treat each index separately. A > GUC will actually involve more code, I suspect. I'

Re: Invisible Indexes

2018-06-24 Thread Andrew Dunstan
On 06/19/2018 02:05 PM, Robert Haas wrote: On Mon, Jun 18, 2018 at 6:12 PM, Tom Lane wrote: Peter Geoghegan writes: On Mon, Jun 18, 2018 at 2:57 PM, Tom Lane wrote: I think the actually desirable way to handle this sort of thing is through an "index advisor" sort of plugin, which can hid

Re: Invisible Indexes

2018-06-19 Thread Andres Freund
Hi, On 2018-06-19 14:05:24 -0400, Robert Haas wrote: > Yeah, I agree that a GUC seems more powerful and easier to roll out. > A downside is that there could be cached plans still using that old > index. If we did DDL on the index we could be sure they all got > invalidated, but otherwise how do w

Re: Invisible Indexes

2018-06-19 Thread Peter Geoghegan
ve but could confuse users if the effect is not > immediate (few words could explain cached plans x invisible indexes). If we're going to go that way, then we better not call them invisible indexes. Invisible indexes are generally understood to be indexes that are "invisible" to ev

Re: Invisible Indexes

2018-06-19 Thread Euler Taveira
s could explain cached plans x invisible indexes). > BTW, like you, I seem to remember somebody writing an extension that > did added a GUC that did exactly this, and demoing it at a conference. > Maybe Oleg or Teodor? > https://github.com/postgrespro/plant

Re: Invisible Indexes

2018-06-19 Thread Robert Haas
On Mon, Jun 18, 2018 at 6:12 PM, Tom Lane wrote: > Peter Geoghegan writes: >> On Mon, Jun 18, 2018 at 2:57 PM, Tom Lane wrote: >>> I think the actually desirable way to handle this sort of thing is through >>> an "index advisor" sort of plugin, which can hide a given index from the >>> planner w

Re: Invisible Indexes

2018-06-19 Thread Konstantin Knizhnik
or more "index advisor" plugins. They're doing the opposite, right? I.e. they return "hypothetical indexes", which then can be used by the planner. None of the ones I've seen currently mask out an existing index. I think that "invisible" indexes are tigh

Re: Invisible Indexes

2018-06-18 Thread David G. Johnston
On Mon, Jun 18, 2018 at 3:17 PM, Andrew Dunstan < andrew.duns...@2ndquadrant.com> wrote: > Yeah, but Peter makes the case that people want it for global > experimentation. "We think we can safely drop this humungous index that > would take us days to rebuild, but before we do let's make it invisib

Re: Invisible Indexes

2018-06-18 Thread Robert Treat
On Mon, Jun 18, 2018 at 6:11 PM, Andres Freund wrote: > On 2018-06-18 18:05:11 -0400, Tom Lane wrote: >> Andres Freund writes: >> > On 2018-06-18 17:57:04 -0400, Tom Lane wrote: >> >> I think the actually desirable way to handle this sort of thing is through >> >> an "index advisor" sort of plugi

Re: Invisible Indexes

2018-06-18 Thread David G. Johnston
On Mon, Jun 18, 2018 at 3:05 PM, Tom Lane wrote: > But if we feel this is worth > pulling into core, I think something along the lines of a GUC listing > indexes to ignore for planning purposes might be a better design. > It'd certainly dodge the issues you mentioned about lack of mutability > of

Re: Invisible Indexes

2018-06-18 Thread Tom Lane
Andrew Dunstan writes: > On 06/18/2018 06:12 PM, Tom Lane wrote: >> Anyway, if we do it with a GUC, the user can control the scope of >> the effects. > Yeah, but Peter makes the case that people want it for global > experimentation. "We think we can safely drop this humungous index that > would

Re: Invisible Indexes

2018-06-18 Thread Andrew Dunstan
On 06/18/2018 06:12 PM, Tom Lane wrote: Peter Geoghegan writes: On Mon, Jun 18, 2018 at 2:57 PM, Tom Lane wrote: I think the actually desirable way to handle this sort of thing is through an "index advisor" sort of plugin, which can hide a given index from the planner without any globally

Re: Invisible Indexes

2018-06-18 Thread Peter Geoghegan
On Mon, Jun 18, 2018 at 3:12 PM, Tom Lane wrote: > Perhaps there are use-cases where you want globally visible effects, > but the primary use-case Andrew cited (i.e. EXPLAIN experimentation) > would not want that. > > Anyway, if we do it with a GUC, the user can control the scope of > the effects.

Re: Invisible Indexes

2018-06-18 Thread Tom Lane
Andres Freund writes: > On 2018-06-18 18:05:11 -0400, Tom Lane wrote: >> Well, what I was thinking about was that this functionality already >> exists (I think) in one or more "index advisor" plugins. > They're doing the opposite, right? I.e. they return "hypothetical > indexes", which then can b

Re: Invisible Indexes

2018-06-18 Thread Julien Rouhaud
On Tue, Jun 19, 2018 at 12:05 AM, Tom Lane wrote: > > Well, what I was thinking about was that this functionality already > exists (I think) in one or more "index advisor" plugins. It's possible > that they've all bit-rotted for lack of support, which would not speak > highly of the demand for th

Re: Invisible Indexes

2018-06-18 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Jun 18, 2018 at 2:57 PM, Tom Lane wrote: >> I think the actually desirable way to handle this sort of thing is through >> an "index advisor" sort of plugin, which can hide a given index from the >> planner without any globally visible side-effects. > The globall

Re: Invisible Indexes

2018-06-18 Thread Andres Freund
On 2018-06-18 18:05:11 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-06-18 17:57:04 -0400, Tom Lane wrote: > >> I think the actually desirable way to handle this sort of thing is through > >> an "index advisor" sort of plugin, which can hide a given index from the > >> planner without

Re: Invisible Indexes

2018-06-18 Thread Tom Lane
Andres Freund writes: > On 2018-06-18 17:57:04 -0400, Tom Lane wrote: >> I think the actually desirable way to handle this sort of thing is through >> an "index advisor" sort of plugin, which can hide a given index from the >> planner without any globally visible side-effects. > Although I'm a bi

Re: Invisible Indexes

2018-06-18 Thread Peter Geoghegan
On Mon, Jun 18, 2018 at 2:57 PM, Tom Lane wrote: > Admittedly, this isn't great in a production environment, but neither > would be disabling the index in the way you suggest. > > I think the actually desirable way to handle this sort of thing is through > an "index advisor" sort of plugin, which

Re: Invisible Indexes

2018-06-18 Thread Andres Freund
Hi, On 2018-06-18 17:57:04 -0400, Tom Lane wrote: > Andrew Dunstan writes: > > This is a MySQL feature, where an index is not considered by the > > planner. Implementing it should be fairly straightforward, adding a new > > boolean to pg_index, and options to CREATE INDEX and ALTER INDEX. I >

Re: Invisible Indexes

2018-06-18 Thread Andres Freund
On 2018-06-18 17:50:44 -0400, Andrew Dunstan wrote: > > > On 06/18/2018 05:46 PM, Jaime Casanova wrote: > > On 18 June 2018 at 16:36, Andrew Dunstan > > wrote: > > > This is a MySQL feature, where an index is not considered by the planner. > > > Implementing it should be fairly straightforward,

Re: Invisible Indexes

2018-06-18 Thread Tom Lane
Andrew Dunstan writes: > This is a MySQL feature, where an index is not considered by the > planner. Implementing it should be fairly straightforward, adding a new > boolean to pg_index, and options to CREATE INDEX and ALTER INDEX. I > guess VISIBLE would become a new unreserved keyword. > The

Re: Invisible Indexes

2018-06-18 Thread Andrew Dunstan
On 06/18/2018 05:44 PM, Peter Geoghegan wrote: On Mon, Jun 18, 2018 at 2:36 PM, Andrew Dunstan wrote: This is a MySQL feature, where an index is not considered by the planner. Implementing it should be fairly straightforward, adding a new boolean to pg_index, and options to CREATE INDEX and

Re: Invisible Indexes

2018-06-18 Thread Andrew Dunstan
On 06/18/2018 05:46 PM, Jaime Casanova wrote: On 18 June 2018 at 16:36, Andrew Dunstan wrote: This is a MySQL feature, where an index is not considered by the planner. Implementing it should be fairly straightforward, adding a new boolean to pg_index, and options to CREATE INDEX and ALTER IN

Re: Invisible Indexes

2018-06-18 Thread Jaime Casanova
On 18 June 2018 at 16:36, Andrew Dunstan wrote: > > This is a MySQL feature, where an index is not considered by the planner. > Implementing it should be fairly straightforward, adding a new boolean to > pg_index, and options to CREATE INDEX and ALTER INDEX. I guess VISIBLE would > become a new un

Re: Invisible Indexes

2018-06-18 Thread Peter Geoghegan
On Mon, Jun 18, 2018 at 2:36 PM, Andrew Dunstan wrote: > > This is a MySQL feature, where an index is not considered by the planner. > Implementing it should be fairly straightforward, adding a new boolean to > pg_index, and options to CREATE INDEX and ALTER INDEX. I guess VISIBLE would > become a

Invisible Indexes

2018-06-18 Thread Andrew Dunstan
This is a MySQL feature, where an index is not considered by the planner. Implementing it should be fairly straightforward, adding a new boolean to pg_index, and options to CREATE INDEX and ALTER INDEX. I guess VISIBLE would become a new unreserved keyword. The most obvious use case is to s