Re: On disable_cost

2024-10-29 Thread David Rowley
On Tue, 29 Oct 2024 at 20:04, Laurenz Albe wrote: > That patch is good in my opinion. Thanks for checking. Pushed. David

Re: On disable_cost

2024-10-29 Thread Laurenz Albe
On Tue, 2024-10-29 at 12:21 +1300, David Rowley wrote: >     When using the enable/disable flags to disable plan node types, many of >     the flags only discourage the use of the corresponding plan node and don't >     outright disallow the planner's ability to use the plan node type.  This >    

Re: On disable_cost

2024-10-28 Thread David Rowley
On Sat, 19 Oct 2024 at 01:09, Laurenz Albe wrote: > Here is my attempt on that paragraph: > > When using the enable/disable flags to disable plan node types, many of > the flags only discourage the use of the corresponding plan node and don't > outright disallow the planner's ability to use

Re: On disable_cost

2024-10-18 Thread Laurenz Albe
Thanks for the fixes; I have only a few quibbles now. On Fri, 2024-10-18 at 23:54 +1300, David Rowley wrote: > --- a/doc/src/sgml/perform.sgml > +++ b/doc/src/sgml/perform.sgml > @@ -578,6 +578,33 @@ WHERE t1.unique1 < 100 AND t1.unique2 = t2.unique2; > discussed below. > > > + > +

Re: On disable_cost

2024-10-18 Thread David Rowley
On Sat, 12 Oct 2024 at 02:16, Laurenz Albe wrote: > > On Fri, 2024-10-11 at 20:45 +1300, David Rowley wrote: > > +When using the enable/disable flags to disable plan node types, the > > +majority of the flags only deprioritize the corresponding plan node > > I don't like "deprioritize". >

Re: On disable_cost

2024-10-11 Thread Laurenz Albe
On Fri, 2024-10-11 at 20:45 +1300, David Rowley wrote: > diff --git a/doc/src/sgml/perform.sgml b/doc/src/sgml/perform.sgml > index ff689b6524..861b9cf0bc 100644 > --- a/doc/src/sgml/perform.sgml > +++ b/doc/src/sgml/perform.sgml > @@ -578,6 +578,34 @@ WHERE t1.unique1 < 100 AND t1.unique2 = t2.uni

Re: On disable_cost

2024-10-11 Thread David Rowley
On Fri, 11 Oct 2024 at 19:44, Laurenz Albe wrote: > > On Fri, 2024-10-11 at 17:24 +1300, David Rowley wrote: > > I've now pushed this change and will look at the docs now. > > Thanks you for taking care of that! I've attached a patch for this. It's very similar to your patch and in the same locat

Re: On disable_cost

2024-10-10 Thread Laurenz Albe
On Fri, 2024-10-11 at 17:24 +1300, David Rowley wrote: > On Fri, 11 Oct 2024 at 02:02, Alena Rybakina > wrote: > > On 10.10.2024 15:43, David Rowley wrote: > > > > > If anyone wants to take a look at the attached, please do so. > > > Otherwise, I'm pretty happy with it and will likely push it on

Re: On disable_cost

2024-10-10 Thread David Rowley
On Fri, 11 Oct 2024 at 02:02, Alena Rybakina wrote: > On 10.10.2024 15:43, David Rowley wrote: > > > If anyone wants to take a look at the attached, please do so. > > Otherwise, I'm pretty happy with it and will likely push it on New > > Zealand Friday (aka later today). > > I think you missed som

Re: On disable_cost

2024-10-10 Thread Alena Rybakina
Hi! On 10.10.2024 15:43, David Rowley wrote: On Tue, 8 Oct 2024 at 04:14, Robert Haas wrote: I think you have adequate consensus to proceed with this. I'd just ask that you don't disappear completely if it turns out that there are problems. I accept that my commit created this problem and I'm

Re: On disable_cost

2024-10-10 Thread David Rowley
On Tue, 8 Oct 2024 at 04:14, Robert Haas wrote: > I think you have adequate consensus to proceed with this. I'd just ask > that you don't disappear completely if it turns out that there are > problems. I accept that my commit created this problem and I'm > certainly willing to be involved too if w

Re: On disable_cost

2024-10-08 Thread Alena Rybakina
On 08.10.2024 18:49, Laurenz Albe wrote: On Tue, 2024-10-08 at 18:12 +0300, Alena Rybakina wrote: However you are right that this display will not appear for all nodes that only contain a data collection procedure, such as Append, MergeAppend, Gather, etc. And I agree with you that we should in

Re: On disable_cost

2024-10-08 Thread Robert Haas
On Mon, Oct 7, 2024 at 6:41 PM Tom Lane wrote: > I don't buy your argument that this case is so special that it > warrants preserving disable_cost. I certainly didn't think it > was special when I added it. That's fair. I'm telling you what I think, not what you have to think. :-) > There may b

Re: On disable_cost

2024-10-08 Thread Laurenz Albe
On Tue, 2024-10-08 at 18:12 +0300, Alena Rybakina wrote: > > > However you are right that this display will not appear for all > > > nodes that only contain a data collection procedure, such as Append, > > > MergeAppend, Gather, etc. And I agree with you that we should > > > information about it. I

Re: On disable_cost

2024-10-08 Thread Alena Rybakina
On 07.10.2024 19:02, Laurenz Albe wrote: On Mon, 2024-10-07 at 10:17 +0300, Alena Rybakina wrote: diff --git a/doc/src/sgml/perform.sgml b/doc/src/sgml/perform.sgml index ff689b65245..db906841472 100644 --- a/doc/src/sgml/perform.sgml +++ b/doc/src/sgml/perform.sgml @@ -578,6 +578,28 @@ WHERE t

Re: On disable_cost

2024-10-07 Thread Tom Lane
Robert Haas writes: > On Sat, Oct 5, 2024 at 3:35 PM Tom Lane wrote: >> BTW, getting off the question of EXPLAIN output for a moment, >> I don't understand why disable_cost is still a thing. The >> one remaining usage seems trivial to replace, as attached. > I believe I commented on that somewh

Re: On disable_cost

2024-10-07 Thread Laurenz Albe
On Mon, 2024-10-07 at 10:17 +0300, Alena Rybakina wrote: > > diff --git a/doc/src/sgml/perform.sgml b/doc/src/sgml/perform.sgml > > index ff689b65245..db906841472 100644 > > --- a/doc/src/sgml/perform.sgml > > +++ b/doc/src/sgml/perform.sgml > > @@ -578,6 +578,28 @@ WHERE t1.unique1 < 100 AND t1.un

Re: On disable_cost

2024-10-07 Thread Laurenz Albe
On Mon, 2024-10-07 at 11:14 -0400, Robert Haas wrote: > I accept that my commit created this problem and I'm > certainly willing to be involved too if we need to sort out more > things. Thanks you. I think it is great that disabled nodes are now handled better, so I appreciate the change as such.

Re: On disable_cost

2024-10-07 Thread Robert Haas
On Mon, Oct 7, 2024 at 11:28 AM Alvaro Herrera wrote: > On 2024-Oct-03, Robert Haas wrote: > > One general thing to think about is that we really document very > > little about EXPLAIN. That might not be good, but we should consider > > whether it will look strange if we document a bunch of stuff

Re: On disable_cost

2024-10-07 Thread Alvaro Herrera
On 2024-Oct-03, Robert Haas wrote: > One general thing to think about is that we really document very > little about EXPLAIN. That might not be good, but we should consider > whether it will look strange if we document a bunch of stuff about > this and still don't talk about anything else. I comp

Re: On disable_cost

2024-10-07 Thread Robert Haas
On Sat, Oct 5, 2024 at 1:37 AM David Rowley wrote: > Thanks for explaining your point of view. I've not shifted my opinion > any, so I guess we just disagree. I feel a strong enough dislike for > the current EXPLAIN output to feel it's worth working harder to have a > better output. > > I won't pu

Re: On disable_cost

2024-10-07 Thread David G. Johnston
On Fri, Oct 4, 2024 at 10:37 PM David Rowley wrote: > I'd encourage anyone else on the sidelines who has an opinion on how > to display the disabled-ness of a plan node in EXPLAIN to speak up > now, even if it's just a +1 to something someone has already written. > It would be nice to see what mo

Re: On disable_cost

2024-10-07 Thread Robert Haas
On Sat, Oct 5, 2024 at 3:35 PM Tom Lane wrote: > BTW, getting off the question of EXPLAIN output for a moment, > I don't understand why disable_cost is still a thing. The > one remaining usage seems trivial to replace, as attached. I believe I commented on that somewhere upthread, but maybe I me

Re: On disable_cost

2024-10-07 Thread Alena Rybakina
On 03.10.2024 23:10, Laurenz Albe wrote: On Thu, 2024-10-03 at 14:24 -0400, Robert Haas wrote: On Thu, Oct 3, 2024 at 1:35 PM Alena Rybakina wrote: I prepared a patch that includes the information we can add. One general thing to think about is that we really document very little about EXPLAI

Re: On disable_cost

2024-10-06 Thread Alena Rybakina
On 06.10.2024 02:22, David Rowley wrote: To be honest, I don’t understand at all why we don’t count disabled nodes for append here? As I understand it, this is due to the fact that the partitioned table can also be scanned by an index. Besides mergeappend, in general it’s difficult for me to g

Re: On disable_cost

2024-10-05 Thread David Rowley
On Sun, 6 Oct 2024 at 08:35, Tom Lane wrote: > BTW, getting off the question of EXPLAIN output for a moment, > I don't understand why disable_cost is still a thing. The > one remaining usage seems trivial to replace, as attached. I didn't notice that any of these remained. I agree we should get

Re: On disable_cost

2024-10-05 Thread David Rowley
On Sun, 6 Oct 2024 at 06:29, Alena Rybakina wrote: > > On 04.10.2024 00:52, David Rowley wrote: > > Append > >-> Index Only Scan using lp1_a_idx on lp1 lp_1 > >-> Sort > > Disabled: true > > Sort Key: lp_2.a > > -> Seq Scan on lp2 lp_2 > To be honest, I don’

Re: On disable_cost

2024-10-05 Thread Tom Lane
BTW, getting off the question of EXPLAIN output for a moment, I don't understand why disable_cost is still a thing. The one remaining usage seems trivial to replace, as attached. regards, tom lane diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/

Re: On disable_cost

2024-10-05 Thread Alena Rybakina
Hi! On 04.10.2024 00:52, David Rowley wrote: One thing the patch did cause me to find is the missing propagation of disabled_nodes in make_sort(). It was very obviously wrong with the patched EXPLAIN output and wasn't so obvious with the current output, so perhaps you could look at this patch as

Re: On disable_cost

2024-10-05 Thread Tom Lane
David Rowley writes: > I'd encourage anyone else on the sidelines who has an opinion on how > to display the disabled-ness of a plan node in EXPLAIN to speak up > now, even if it's just a +1 to something someone has already written. > It would be nice to see what more people think. FWIW, I do not

Re: On disable_cost

2024-10-05 Thread Shayon Mukherjee
On Sat, Oct 5, 2024 at 1:37 AM David Rowley wrote: > On Sat, 5 Oct 2024 at 03:03, Robert Haas wrote: > > I tend to gravitate > > toward displaying things exactly as they exist internally because I've > > had so many bad experiences with having to try to reverse-engineer the > > value stored inte

Re: On disable_cost

2024-10-04 Thread David Rowley
On Sat, 5 Oct 2024 at 03:03, Robert Haas wrote: > I tend to gravitate > toward displaying things exactly as they exist internally because I've > had so many bad experiences with having to try to reverse-engineer the > value stored internally from whatever is printed. Thanks for explaining your po

Re: On disable_cost

2024-10-04 Thread Robert Haas
On Thu, Oct 3, 2024 at 5:52 PM David Rowley wrote: > It looks fine with the patch. The crux of the new logic is just > summing up the disabled_nodes from the child nodes and checking if the > disabled_nodes of the current node is higher than that sum. That's not > exactly hard logic. The biggest r

Re: On disable_cost

2024-10-03 Thread David Rowley
On Fri, 4 Oct 2024 at 02:15, Robert Haas wrote: > robert.haas=# explain with recursive foo as (select count(*) from > pgbench_accounts union all select aid from pgbench_accounts a, foo > where aid > foo.count) select * from pgbench_accounts, foo where aid = > foo.count; > You might expect that th

Re: On disable_cost

2024-10-03 Thread Laurenz Albe
On Thu, 2024-10-03 at 14:24 -0400, Robert Haas wrote: > On Thu, Oct 3, 2024 at 1:35 PM Alena Rybakina > wrote: > > I prepared a patch that includes the information we can add. > > One general thing to think about is that we really document very > little about EXPLAIN. That might not be good, but

Re: On disable_cost

2024-10-03 Thread Robert Haas
On Thu, Oct 3, 2024 at 1:35 PM Alena Rybakina wrote: > I think you are right, most users will perceive this parameter as the number > of rejected paths, and not in any other way. > > To be honest, I don't have much experience writing documentation, but I think > we should add a little more infor

Re: On disable_cost

2024-10-03 Thread Alena Rybakina
Overall, I think we need to do something here. There's no documentation about what Disabled Nodes means so we either need to make it easier to understand without documenting it or add something to the documents about it. If Laurenz, who has a huge amount of PostgreSQL experience didn't catch it, t

Re: On disable_cost

2024-10-03 Thread Alena Rybakina
On 03.10.2024 01:44, David Rowley wrote: On Thu, 3 Oct 2024 at 08:41, Alena Rybakina wrote: I may have misunderstood your message, but disabled nodes number must propagate up the tree, otherwise it will be incorrect. I think there are two misunderstandings on this thread: 1) You're misunders

Re: On disable_cost

2024-10-03 Thread Robert Haas
On Wed, Oct 2, 2024 at 8:11 PM David Rowley wrote: > I can't quite find the area you're talking about where the > disabled_nodes don't propagate through subquery levels. Looking at > cost_subqueryscan(), I see propagation of disabled_nodes. If the > SubqueryScan node isn't present then the propaga

Re: On disable_cost

2024-10-02 Thread Laurenz Albe
On Thu, 2024-10-03 at 11:44 +1300, David Rowley wrote: > 2) Laurenz is misunderstanding what "Disabled Nodes" means. It has > nothing to do with other Paths which were considered and rejected. It > might be better named as "Disabled Degree". It tracks how many plan > nodes below and including this

Re: On disable_cost

2024-10-02 Thread David Rowley
On Thu, 3 Oct 2024 at 03:04, Robert Haas wrote: > I don't think this will produce the right answer in all cases because > disabled node counts don't propagate across subquery levels. I see my patch didn't behave correctly when faced with a SubqueryScan as SubqueryScan does not use the "lefttree"

Re: On disable_cost

2024-10-02 Thread David Rowley
On Thu, 3 Oct 2024 at 08:41, Alena Rybakina wrote: > I may have misunderstood your message, but disabled nodes number must > propagate up the tree, otherwise it will be incorrect. I think there are two misunderstandings on this thread: 1) You're misunderstanding what Laurenz is complaining abou

Re: On disable_cost

2024-10-02 Thread Alena Rybakina
On 02.10.2024 22:08, Laurenz Albe wrote: On Wed, 2024-10-02 at 21:31 +0300, Alena Rybakina wrote: Honestly, I like this patch. Before this patch, when disabling any algorithm in the optimizer, the cost increased significantly and I’m not sure that this was a reliable solution due to the fact th

Re: On disable_cost

2024-10-02 Thread Alena Rybakina
I see; the merge join happened to be the preferred join path, so nothing had to be excluded. /* reset all parameters */ EXPLAIN (COSTS OFF) SELECT * FROM tab_a JOIN tab_b USING (id); QUERY PLAN ═ Merge Join Merge Cond: (tab_

Re: On disable_cost

2024-10-02 Thread Laurenz Albe
On Wed, 2024-10-02 at 21:31 +0300, Alena Rybakina wrote: > Honestly, I like this patch. Before this patch, when disabling any algorithm > in the optimizer, the cost increased significantly and I’m not sure that this > was a reliable solution due to the fact that the cost even without disabling > ca

Re: On disable_cost

2024-10-02 Thread Laurenz Albe
On Wed, 2024-10-02 at 21:13 +0300, Alena Rybakina wrote: > >   CREATE TABLE tab_a (id integer); > > > >   CREATE TABLE tab_b (id integer); > > > >   SET enable_nestloop = off; > >   SET enable_hashjoin = off; > > > >   EXPLAIN SELECT * FROM tab_a JOIN tab_b USING (id); > > > >  

Re: On disable_cost

2024-10-02 Thread Alena Rybakina
you can disable mergejoin, I think the output about this will appear. I did it and disabled nodes were displayed in the query explain: alena@postgres=#  CREATE TABLE tab_a (id integer); alena@postgres=#  CREATE TABLE tab_a (id integer); alena@postgres=#  CREATE TABLE tab_b (id integer); alena

Re: On disable_cost

2024-10-02 Thread Alena Rybakina
Hi! On 02.10.2024 21:04, Laurenz Albe wrote: I didn't want a running total, but maybe I misunderstood what a disabled node is; see below. If you see a join where two plans were disabled, that's useful information. I'm not sure if I follow what you mean here.  The patch will show "Disabled: tr

Re: On disable_cost

2024-10-02 Thread Laurenz Albe
On Wed, 2024-10-02 at 10:08 -0400, Robert Haas wrote: > On Fri, Sep 27, 2024 at 4:42 AM Laurenz Albe wrote: > > 1. The "disabled nodes" are always displayed. > >     I'd be happier if it were only shown for COSTS ON, but I think it > >     would be best if they were only shown with VERBOSE ON. > >

Re: On disable_cost

2024-10-02 Thread Laurenz Albe
On Wed, 2024-10-02 at 21:55 +1300, David Rowley wrote: > On Tue, 1 Oct 2024 at 06:17, Laurenz Albe wrote: > > Why did you change "Disabled" from an integer to a boolean? > > I just don't think "Disabled Nodes" is all that self-documenting and > I'm also unsure why the full integer value of disabl

Re: On disable_cost

2024-10-02 Thread Robert Haas
On Fri, Sep 27, 2024 at 4:42 AM Laurenz Albe wrote: > 1. The "disabled nodes" are always displayed. >I'd be happier if it were only shown for COSTS ON, but I think it >would be best if they were only shown with VERBOSE ON. > >After all, the messages are pretty verbose... I agree that

Re: On disable_cost

2024-10-02 Thread Robert Haas
On Wed, Oct 2, 2024 at 4:55 AM David Rowley wrote: > On Tue, 1 Oct 2024 at 06:17, Laurenz Albe wrote: > > Why did you change "Disabled" from an integer to a boolean? > > I just don't think "Disabled Nodes" is all that self-documenting and > I'm also unsure why the full integer value of disabled_n

Re: On disable_cost

2024-10-02 Thread David Rowley
On Tue, 1 Oct 2024 at 06:17, Laurenz Albe wrote: > Why did you change "Disabled" from an integer to a boolean? I just don't think "Disabled Nodes" is all that self-documenting and I'm also unsure why the full integer value of disabled_nodes is required over just displaying the boolean value of if

Re: On disable_cost

2024-09-30 Thread Laurenz Albe
On Sat, 2024-09-28 at 00:04 +1200, David Rowley wrote: > On Fri, 27 Sept 2024 at 20:42, Laurenz Albe wrote: > > 2. The "disabled nodes" are not only shown at the nodes where nodes > >were actually disabled, but also at every nodes above these nodes. > > I'm also not a fan either and I'd like

Re: On disable_cost

2024-09-27 Thread David Rowley
On Fri, 27 Sept 2024 at 20:42, Laurenz Albe wrote: > 2. The "disabled nodes" are not only shown at the nodes where nodes >were actually disabled, but also at every nodes above these nodes. I'm also not a fan either and I'd like to see this output improved. It seems like it's easy enough to i

Re: On disable_cost

2024-09-27 Thread Laurenz Albe
On Wed, 2024-08-21 at 10:29 -0400, Robert Haas wrote: > I went ahead and committed these patches. I know there's some debate > over whether we want to show the # of disabled nodes and if so whether > it should be controlled by COSTS, and I suspect I haven't completely > allayed David's concerns abo

Re: On disable_cost

2024-09-09 Thread Robert Haas
On Mon, Sep 9, 2024 at 12:09 AM Richard Guo wrote: > Fixed. Thanks to Alexander for the very good catch and to Richard for pushing the fix. (I started to respond to this last week but didn't quite get to it before I ran out of time/energy.) -- Robert Haas EDB: http://www.enterprisedb.com

Re: On disable_cost

2024-09-08 Thread Richard Guo
On Fri, Sep 6, 2024 at 5:27 PM Richard Guo wrote: > On Fri, Sep 6, 2024 at 5:00 PM Alexander Lakhin wrote: > > static void > > label_sort_with_costsize(PlannerInfo *root, Sort *plan, double limit_tuples) > > { > > ... > > cost_sort(&sort_path, root, NIL, > >lefttree->total_co

Re: On disable_cost

2024-09-06 Thread Alexander Lakhin
Hello Richard, 06.09.2024 12:51, Richard Guo wrote: Ah I see. label_sort_with_costsize is only used to label the Sort node nicely for EXPLAIN, and usually we do not display the cost numbers in regression tests. In fact, I see the error with the following (EXPLAIN-less) query: create table t (

Re: On disable_cost

2024-09-06 Thread Richard Guo
On Fri, Sep 6, 2024 at 5:27 PM Richard Guo wrote: > On Fri, Sep 6, 2024 at 5:00 PM Alexander Lakhin wrote: > > static void > > label_sort_with_costsize(PlannerInfo *root, Sort *plan, double limit_tuples) > (I'm a little surprised that this does not cause any plan diffs in the > regression tests.

Re: On disable_cost

2024-09-06 Thread Richard Guo
On Fri, Sep 6, 2024 at 5:00 PM Alexander Lakhin wrote: > static void > label_sort_with_costsize(PlannerInfo *root, Sort *plan, double limit_tuples) > { > ... > cost_sort(&sort_path, root, NIL, >lefttree->total_cost, >plan->plan.disabled_nodes, >

Re: On disable_cost

2024-09-06 Thread Alexander Lakhin
Hello Robert, 21.08.2024 17:29, Robert Haas wrote: I went ahead and committed these patches. ... Please take a look at the following code: static void label_sort_with_costsize(PlannerInfo *root, Sort *plan, double limit_tuples) { ...     cost_sort(&sort_path, root, NIL,   lefttree-

Re: On disable_cost

2024-08-23 Thread Jonathan S. Katz
On 8/23/24 5:33 PM, Jonathan S. Katz wrote: On 8/23/24 3:32 PM, Tom Lane wrote: Robert Haas writes: I think it's not that bad, because we can limit the knowledge of this hack to the amcostestimate interface, which doesn't really deal in "the user told us not to do it this way" at all.  That

Re: On disable_cost

2024-08-23 Thread Jonathan S. Katz
On 8/23/24 3:32 PM, Tom Lane wrote: Robert Haas writes: I find both of your proposed solutions above to be pretty inelegant, They are that. If we were working in a green field I'd not propose such things ... but we aren't. I believe there are now a fair number of out-of-core index AMs, so I

Re: On disable_cost

2024-08-23 Thread Jonathan S. Katz
On 8/23/24 2:29 PM, Robert Haas wrote: On Fri, Aug 23, 2024 at 2:20 PM Jonathan S. Katz wrote: I don't think extension maintainers necessarily have the same level of PostgreSQL internals as you or many of the other people who frequent -hackers, so I think it's fair for them to ask questions or

Re: On disable_cost

2024-08-23 Thread Tom Lane
Robert Haas writes: > I find both of your proposed solutions above to be pretty inelegant, They are that. If we were working in a green field I'd not propose such things ... but we aren't. I believe there are now a fair number of out-of-core index AMs, so I'd rather not break all of them if we

Re: On disable_cost

2024-08-23 Thread Heikki Linnakangas
On 23/08/2024 22:05, Robert Haas wrote: On Fri, Aug 23, 2024 at 2:48 PM Tom Lane wrote: If we're going to do this, I'd prefer a solution that doesn't force API changes onto the vast majority of index AMs that don't have a problem here. That's a fair concern. Yeah, although I don't think it'

Re: On disable_cost

2024-08-23 Thread Robert Haas
On Fri, Aug 23, 2024 at 2:48 PM Tom Lane wrote: > If we're going to do this, I'd prefer a solution that doesn't force > API changes onto the vast majority of index AMs that don't have a > problem here. That's a fair concern. > One way could be to formalize the hack we were just discussing: > "To

Re: On disable_cost

2024-08-23 Thread Tom Lane
Robert Haas writes: > On Fri, Aug 23, 2024 at 2:18 PM Heikki Linnakangas wrote: >> It would seem useful for an index AM to be able to say "nope, I can't do >> this". I don't remember how exactly this stuff works, but I'm surprised >> it doesn't already exist. > Yeah, I think so, too. While this

Re: On disable_cost

2024-08-23 Thread Robert Haas
On Fri, Aug 23, 2024 at 2:18 PM Heikki Linnakangas wrote: > It would seem useful for an index AM to be able to say "nope, I can't do > this". I don't remember how exactly this stuff works, but I'm surprised > it doesn't already exist. Yeah, I think so, too. While this particular problem is due to

Re: On disable_cost

2024-08-23 Thread Robert Haas
On Fri, Aug 23, 2024 at 2:20 PM Jonathan S. Katz wrote: > I don't think extension maintainers necessarily have the same level of > PostgreSQL internals as you or many of the other people who frequent > -hackers, so I think it's fair for them to ask questions or raise issues > with patches they don

Re: On disable_cost

2024-08-23 Thread Jonathan S. Katz
On 8/23/24 1:11 PM, Robert Haas wrote: On Fri, Aug 23, 2024 at 11:17 AM Jonathan S. Katz wrote: We hit an issue with pgvector[0] where a regular `SELECT count(*) FROM table`[1] is attempting to scan the index on the vector column when `enable_seqscan` is disabled. Credit to Andrew Kane (CC'd) f

Re: On disable_cost

2024-08-23 Thread Heikki Linnakangas
On 23/08/2024 20:11, Robert Haas wrote: I find it a little weird that hnsw thinks itself able to return all the tuples in an order the user chooses, but unable to return all of the tuples in an arbitrary order. HNSW is weird in many ways: - There is no inherent sort order. It cannot do "ORDER

Re: On disable_cost

2024-08-23 Thread Tom Lane
Robert Haas writes: > On Fri, Aug 23, 2024 at 1:26 PM Tom Lane wrote: >> It looks like amcostestimate could change the path's disabled_nodes >> count, since that's set up before invoking amcostestimate. I guess >> it could be set to INT_MAX to have a comparable solution to before. > It's probab

Re: On disable_cost

2024-08-23 Thread Robert Haas
On Fri, Aug 23, 2024 at 1:26 PM Tom Lane wrote: > It looks like amcostestimate could change the path's disabled_nodes > count, since that's set up before invoking amcostestimate. I guess > it could be set to INT_MAX to have a comparable solution to before. It's probably better to add a more mode

Re: On disable_cost

2024-08-23 Thread Tom Lane
Robert Haas writes: > On Fri, Aug 23, 2024 at 11:17 AM Jonathan S. Katz > wrote: >> We hit an issue with pgvector[0] where a regular `SELECT count(*) FROM >> table`[1] is attempting to scan the index on the vector column when >> `enable_seqscan` is disabled. Credit to Andrew Kane (CC'd) for flag

Re: On disable_cost

2024-08-23 Thread Robert Haas
On Fri, Aug 23, 2024 at 11:17 AM Jonathan S. Katz wrote: > We hit an issue with pgvector[0] where a regular `SELECT count(*) FROM > table`[1] is attempting to scan the index on the vector column when > `enable_seqscan` is disabled. Credit to Andrew Kane (CC'd) for flagging it. > > I was able to tr

Re: On disable_cost

2024-08-23 Thread Jonathan S. Katz
On 8/21/24 10:29 AM, Robert Haas wrote: I went ahead and committed these patches. I know there's some debate over whether we want to show the # of disabled nodes and if so whether it should be controlled by COSTS, and I suspect I haven't completely allayed David's concerns about the initial_cost

Re: On disable_cost

2024-08-22 Thread Robert Haas
On Thu, Aug 22, 2024 at 8:07 AM Jelte Fennema-Nio wrote: > Are the disabled node counts still expected to be stable even with > GEQO? If not, maybe we should have a way to turn them off after all. > Although I agree that always disabling them when COSTS OFF is set is > probably also undesirable. H

Re: On disable_cost

2024-08-22 Thread Jelte Fennema-Nio
On Wed, 31 Jul 2024 at 18:23, Robert Haas wrote: > - If we do commit 0002, I think it's a good idea to have the number of > disabled nodes displayed even with COSTS OFF, because it's stable, and > it's pretty useful to be able to see this in the regression output. I > have found while working on t

Re: On disable_cost

2024-08-21 Thread Robert Haas
On Fri, Aug 2, 2024 at 12:53 PM Robert Haas wrote: > On Fri, Aug 2, 2024 at 12:51 PM Tom Lane wrote: > > That absolutely is the expectation, and we'd better be careful not > > to break it. > > I have every intention of not breaking it. :-) I went ahead and committed these patches. I know there's

Re: On disable_cost

2024-08-02 Thread Robert Haas
On Fri, Aug 2, 2024 at 12:51 PM Tom Lane wrote: > That absolutely is the expectation, and we'd better be careful not > to break it. I have every intention of not breaking it. :-) -- Robert Haas EDB: http://www.enterprisedb.com

Re: On disable_cost

2024-08-02 Thread Tom Lane
Robert Haas writes: > On Fri, Aug 2, 2024 at 9:13 AM David Rowley wrote: >> ... That way you maintain the >> existing behaviour of not optimising for disabled node types and don't >> risk plan changes if the final cost comes out cheaper than the initial >> cost. > All three initial_cost_XXX func

Re: On disable_cost

2024-08-02 Thread Robert Haas
On Fri, Aug 2, 2024 at 9:13 AM David Rowley wrote: > I now think neither of us got it right. I now think what you'd need to > do to be aligned to the current behaviour is have > initial_cost_nestloop() add the disabled_nodes for the join's subnodes > *only* and have final_cost_nestloop() add the a

Re: On disable_cost

2024-08-02 Thread David Rowley
On Sat, 3 Aug 2024 at 00:17, Robert Haas wrote: > > On Thu, Aug 1, 2024 at 11:34 PM David Rowley wrote: > > I'm not planning on pushing this any further. I've just tried to > > highlight that there's the possibility of a behavioural change. You're > > claiming there isn't one. I claim there is. >

Re: On disable_cost

2024-08-02 Thread Robert Haas
On Thu, Aug 1, 2024 at 11:34 PM David Rowley wrote: > I'm not planning on pushing this any further. I've just tried to > highlight that there's the possibility of a behavioural change. You're > claiming there isn't one. I claim there is. I don't know what to tell you. The original version of the

Re: On disable_cost

2024-08-01 Thread David Rowley
On Fri, 2 Aug 2024 at 06:03, Robert Haas wrote: > I think this may be a bit hard to understand, so let me give a > concrete example. Suppose we're planning some join where one side can > only be planned with a sequential scan and sequential scans are > disabled. We have ten paths in the path list

Re: On disable_cost

2024-08-01 Thread Robert Haas
On Wed, Jul 31, 2024 at 10:01 PM David Rowley wrote: > I've reviewed both patches, here's what I noted down during my review: Thanks. > 0. I've not seen any mention so far about postgres_fdw's > use_remote_estimate. Maybe changing the costs is fixing an issue that > existed before. I'm just not

Re: On disable_cost

2024-07-31 Thread David Rowley
On Thu, 1 Aug 2024 at 04:23, Robert Haas wrote: > OK, here's a new patch version. I think we're going down the right path here. I've reviewed both patches, here's what I noted down during my review: 0. I've not seen any mention so far about postgres_fdw's use_remote_estimate. Maybe changing th

Re: On disable_cost

2024-07-31 Thread Robert Haas
OK, here's a new patch version. I earlier committed the refactoring to avoid using disable_cost to force WHERE CURRENT OF to be implemented by a TID scan. In this version, I've dropped everything related to reworking enable_indexscan or any other enable_* GUC. Hence, this version of the patch set j

Re: On disable_cost

2024-07-03 Thread David Rowley
On Wed, 3 Jul 2024 at 09:49, Tom Lane wrote: > > Heikki Linnakangas writes: > > 3. Oh right, bitmap scan, I forgot about that one. Let's disable that too: > > Yeah, I've hit that too, although more often (for me) it's the first > choice of plan. In any case, it usually takes more than one change

Re: On disable_cost

2024-07-03 Thread Robert Haas
On Tue, Jul 2, 2024 at 5:39 PM Heikki Linnakangas wrote: > I would be somewhat annoyed if we add another step to that, to also > disable index-only scans separately. It would be nice if > enable_indexscan=off would also disable bitmap scans, that would > eliminate one step from the above. Almost a

Re: On disable_cost

2024-07-02 Thread Tom Lane
Heikki Linnakangas writes: > 3. Oh right, bitmap scan, I forgot about that one. Let's disable that too: Yeah, I've hit that too, although more often (for me) it's the first choice of plan. In any case, it usually takes more than one change to get to a seqscan. > It almost feels like we should h

Re: On disable_cost

2024-07-02 Thread Heikki Linnakangas
On 02/07/2024 22:54, Robert Haas wrote: On Tue, Jul 2, 2024 at 3:36 PM Tom Lane wrote: One could argue for other things, of course. And maybe those other things are fine, if they're properly justified and documented. [ shrug... ] This isn't a hill that I'm prepared to die on. But I see no go

Re: On disable_cost

2024-07-02 Thread Tom Lane
Robert Haas writes: > Well, I don't really know where to go from here. I mean, I think that > three committers (David, Heikki, yourself) have expressed some > concerns about changing the behavior. So maybe we shouldn't. But I > don't understand how it's reasonable to have two very similarly named

Re: On disable_cost

2024-07-02 Thread Robert Haas
On Tue, Jul 2, 2024 at 3:36 PM Tom Lane wrote: > > One could argue for other things, of course. And maybe those other > > things are fine, if they're properly justified and documented. > > [ shrug... ] This isn't a hill that I'm prepared to die on. > But I see no good reason to change the very lo

Re: On disable_cost

2024-07-02 Thread Tom Lane
Robert Haas writes: > What the patch does is: if you set either enable_indexscan=false or > enable_indexonlyscan=false, then the corresponding path type is not > generated, and the other is unaffected. To me, that seems like the > logical way to clean this up. > One could argue for other things,

Re: On disable_cost

2024-07-02 Thread Robert Haas
On Tue, Jul 2, 2024 at 2:37 PM Tom Lane wrote: > Robert Haas writes: > > What happens right now is: > > > - If you set enable_indexscan=false, then disable_cost is added to the > > cost of index scan paths and the cost of index-only scan paths. > > > - If you set enable_indexonlyscan=false, then

Re: On disable_cost

2024-07-02 Thread Tom Lane
Robert Haas writes: > What happens right now is: > - If you set enable_indexscan=false, then disable_cost is added to the > cost of index scan paths and the cost of index-only scan paths. > - If you set enable_indexonlyscan=false, then index-only scan paths > are not generated at all. Hm. The

Re: On disable_cost

2024-07-02 Thread Robert Haas
On Tue, Jul 2, 2024 at 1:40 PM Tom Lane wrote: > FWIW, I disagree completely. I think it's entirely natural to > consider bitmap index scans to be a subset of index scans, so that > enable_indexscan should affect both. I admit that the current set > of GUCs doesn't let you force a bitmap scan ov

  1   2   >