Re: Removing unneeded self joins

2025-07-15 Thread Alexander Korotkov
On Wed, Jul 16, 2025 at 1:16 AM Michael Paquier wrote: > On Wed, Jul 16, 2025 at 12:38:58AM +0300, Alexander Korotkov wrote: > > I recently got notification this is in Open Items. > > https://wiki.postgresql.org/wiki/PostgreSQL_18_Open_Items > > What is your opinion on this: do we need additional

Re: Removing unneeded self joins

2025-07-15 Thread Michael Paquier
On Wed, Jul 16, 2025 at 12:38:58AM +0300, Alexander Korotkov wrote: > I recently got notification this is in Open Items. > https://wiki.postgresql.org/wiki/PostgreSQL_18_Open_Items > What is your opinion on this: do we need additional hook, fair to > leave this "as is" or another option? I'm OK wi

Re: Removing unneeded self joins

2025-07-15 Thread Alexander Korotkov
Hi, Mechael! On Fri, Jun 27, 2025 at 3:55 PM Alexander Korotkov wrote: > On Fri, Jun 27, 2025 at 3:26 AM Michael Paquier wrote: > > On Thu, Jun 26, 2025 at 08:54:55AM +0200, Andrei Lepikhov wrote: > > > Before diving into the pg_hint_plan code, I wonder why you don't have > > > similar issues wi

Re: Removing unneeded self joins

2025-06-27 Thread Alexander Korotkov
Hi, Michael! On Fri, Jun 27, 2025 at 3:26 AM Michael Paquier wrote: > On Thu, Jun 26, 2025 at 08:54:55AM +0200, Andrei Lepikhov wrote: > > Before diving into the pg_hint_plan code, I wonder why you don't have > > similar issues with the remove_useless_joins. We intentionally designed SJE > > coup

Re: Removing unneeded self joins

2025-06-26 Thread Andrei Lepikhov
On 27/6/2025 02:26, Michael Paquier wrote: The point regarding the search join hook may stand, though. Perhaps somebody should check if we're still OK with this change in the context of the self-join work. I tend to think that we are and I agree that removing the joins when calling the hook can

Re: Removing unneeded self joins

2025-06-26 Thread Michael Paquier
On Thu, Jun 26, 2025 at 08:54:55AM +0200, Andrei Lepikhov wrote: > Before diving into the pg_hint_plan code, I wonder why you don't have > similar issues with the remove_useless_joins. We intentionally designed SJE > coupled with the left-join removal feature to avoid such type of complaints: > >

Re: Removing unneeded self joins

2025-06-25 Thread Andrei Lepikhov
On 26/6/2025 07:40, Michael Paquier wrote: Anyway, it seems to me that we may need to do something here before the release. Note that if the consensus is "you should update your module and not rely on the past behavior", I'm OK with that. I just wanted to raise the issue before this goes GA. A

Re: Removing unneeded self joins

2025-06-25 Thread Michael Paquier
On Sat, Apr 26, 2025 at 11:05:27PM +0300, Alexander Korotkov wrote: >> I did some improvements to PHVs patch: revised comments and commit >> message. I'm going to push it if no objections. > > Uh, v2 was there already. That should be v3. I was doing some work on pg_hint_plan, evaluating the amo

Re: Removing unneeded self joins

2025-04-26 Thread Alexander Korotkov
On Sat, Apr 26, 2025 at 11:04 PM Alexander Korotkov wrote: > On Sun, Apr 6, 2025 at 12:02 AM Alena Rybakina > wrote: > > Should we add more regression tests covering these cases? > > > > I experimented with some examples like this and noticed that it does affect > > cardinality estimation, thoug

Re: Removing unneeded self joins

2025-04-26 Thread Alexander Korotkov
Hi, Alena! On Sun, Apr 6, 2025 at 12:02 AM Alena Rybakina wrote: > Should we add more regression tests covering these cases? > > I experimented with some examples like this and noticed that it does affect > cardinality estimation, though I'm not sure the impact is significant. > I used the table

Re: Removing unneeded self joins

2025-04-07 Thread Alexander Korotkov
Hi, Richard! On Mon, Apr 7, 2025 at 6:12 AM Richard Guo wrote: > > FWIW, I reported some issues with this commit in [1]. Any thoughts on > how to fix them? > > [1] > https://www.postgresql.org/message-id/flat/CAMbWs49PE3CvnV8vrQ0Dr%3DHqgZZmX0tdNbzVNJxqc8yg-8kDQQ%40mail.gmail.com Thank you for

Re: Removing unneeded self joins

2025-04-06 Thread Richard Guo
FWIW, I reported some issues with this commit in [1]. Any thoughts on how to fix them? [1] https://www.postgresql.org/message-id/flat/CAMbWs49PE3CvnV8vrQ0Dr%3DHqgZZmX0tdNbzVNJxqc8yg-8kDQQ%40mail.gmail.com Thanks Richard

Re: Removing unneeded self joins

2025-04-06 Thread Alexander Korotkov
On Sun, Apr 6, 2025 at 5:41 PM Tom Lane wrote: > Alexander Korotkov writes: > > Nevertheless, should we consider revisiting this flag? I see the only > > other GUC simultaneously QUERY_TUNING_METHOD and GUC_NOT_IN_SAMPLE is > > optimize_bounded_sort, which is not exposed in a standard build. > >

Re: Removing unneeded self joins

2025-04-06 Thread Tom Lane
Alexander Korotkov writes: > Nevertheless, should we consider revisiting this flag? I see the only > other GUC simultaneously QUERY_TUNING_METHOD and GUC_NOT_IN_SAMPLE is > optimize_bounded_sort, which is not exposed in a standard build. enable_self_join_elimination is documented, and it has no

Re: Removing unneeded self joins

2025-04-06 Thread Tender Wang
Alexander Korotkov 于2025年4月6日周日 19:50写道: > On Sun, Apr 6, 2025 at 2:42 PM Tender Wang wrote: > > Daniel Gustafsson 于2025年4月6日周日 19:23写道: > >> > >> > On 6 Apr 2025, at 06:02, Tender Wang wrote: > >> > >> > I find that the postgresql.conf.sample file doesn't contain > enable_self_join_eliminatio

Re: Removing unneeded self joins

2025-04-06 Thread Alexander Korotkov
On Sun, Apr 6, 2025 at 2:42 PM Tender Wang wrote: > Daniel Gustafsson 于2025年4月6日周日 19:23写道: >> >> > On 6 Apr 2025, at 06:02, Tender Wang wrote: >> >> > I find that the postgresql.conf.sample file doesn't contain >> > enable_self_join_elimination guc. >> > If this is necessary, please see the at

Re: Removing unneeded self joins

2025-04-06 Thread Tender Wang
Daniel Gustafsson 于2025年4月6日周日 19:23写道: > > On 6 Apr 2025, at 06:02, Tender Wang wrote: > > > I find that the postgresql.conf.sample file doesn't contain > enable_self_join_elimination guc. > > If this is necessary, please see the attached patch. > > The GUC is marked as not supposed by the in t

Re: Removing unneeded self joins

2025-04-06 Thread Daniel Gustafsson
> On 6 Apr 2025, at 06:02, Tender Wang wrote: > I find that the postgresql.conf.sample file doesn't contain > enable_self_join_elimination guc. > If this is necessary, please see the attached patch. The GUC is marked as not supposed by the in the sample file, either it really shouldn't or that

Re: Removing unneeded self joins

2025-04-05 Thread Tender Wang
Hi, I find that the postgresql.conf.sample file doesn't contain enable_self_join_elimination guc. If this is necessary, please see the attached patch. -- Thanks, Tender Wang From f27c99aebbd07d4008173492c7913749b149b540 Mon Sep 17 00:00:00 2001 From: Tender Wang Date: Sun, 6 Apr 2025 11:54:49 +0

Re: Removing unneeded self joins

2025-04-05 Thread Alena Rybakina
On 05.04.2025 13:09, Alexander Korotkov wrote: On Fri, Apr 4, 2025 at 11:35 AM Andrei Lepikhov wrote: On 4/4/25 04:53, Richard Guo wrote: On Fri, Apr 4, 2025 at 1:02 AM Alexander Korotkov wrote: I've got an off-list bug report from Alexander Lakhin involving a placeholder variable. Alena an

Re: Removing unneeded self joins

2025-04-05 Thread Alexander Korotkov
On Fri, Apr 4, 2025 at 11:35 AM Andrei Lepikhov wrote: > > On 4/4/25 04:53, Richard Guo wrote: > > On Fri, Apr 4, 2025 at 1:02 AM Alexander Korotkov > > wrote: > >> I've got an off-list bug report from Alexander Lakhin involving a > >> placeholder variable. Alena and Andrei proposed a fix. It

Re: Removing unneeded self joins

2025-04-04 Thread Andrei Lepikhov
On 4/4/25 04:53, Richard Guo wrote: On Fri, Apr 4, 2025 at 1:02 AM Alexander Korotkov wrote: I've got an off-list bug report from Alexander Lakhin involving a placeholder variable. Alena and Andrei proposed a fix. It is fairly simple: we just shouldn't remove PHVs during self-join elimination

Re: Removing unneeded self joins

2025-04-03 Thread Richard Guo
On Fri, Apr 4, 2025 at 1:02 AM Alexander Korotkov wrote: > I've got an off-list bug report from Alexander Lakhin involving a > placeholder variable. Alena and Andrei proposed a fix. It is fairly > simple: we just shouldn't remove PHVs during self-join elimination, as > they might still be refere

Re: Removing unneeded self joins

2025-04-03 Thread Alexander Korotkov
Hi all, I've got an off-list bug report from Alexander Lakhin involving a placeholder variable. Alena and Andrei proposed a fix. It is fairly simple: we just shouldn't remove PHVs during self-join elimination, as they might still be referenced from other parts of a query. The patch is attached.

Re: Removing unneeded self joins

2025-03-21 Thread Ranier Vilela
Hi. Em sáb., 15 de fev. de 2025 às 06:20, Alexander Korotkov < aekorot...@gmail.com> escreveu: > On Thu, Feb 13, 2025 at 1:00 AM Alexander Korotkov > wrote: > > > > On Tue, Feb 11, 2025 at 5:31 PM Alena Rybakina > > wrote: > > > Hi! Thank you for the work with this subject, I think it is really

Re: Removing unneeded self joins

2025-03-06 Thread Andrei Lepikhov
On 26/2/2025 13:14, Alexander Korotkov wrote: On Mon, Feb 24, 2025 at 2:22 PM Andrei Lepikhov wrote: On 24/2/2025 11:57, Alexander Korotkov wrote: Could you, please, elaborate more on what you mean by "new technique of query tree reduction"? I mean any transformations and optimisations that r

Re: Removing unneeded self joins

2025-02-26 Thread Alexander Korotkov
On Mon, Feb 24, 2025 at 2:22 PM Andrei Lepikhov wrote: > > On 24/2/2025 11:57, Alexander Korotkov wrote: > > Hi, Andrei! > > > > Thank you for your feedback. > > > > On Mon, Feb 24, 2025 at 12:12 AM Andrei Lepikhov wrote: > >> On 23/2/2025 22:15, Alexander Korotkov wrote: > >>> There is my attemp

Re: Removing unneeded self joins

2025-02-26 Thread Alexander Korotkov
On Mon, Feb 24, 2025 at 12:57 PM Alexander Korotkov wrote: > On Mon, Feb 24, 2025 at 12:12 AM Andrei Lepikhov wrote: > > On 23/2/2025 22:15, Alexander Korotkov wrote: > > > There is my attempt to implement this approach. Getting rid of local > > > variable (and computation of the same value othe

Re: Removing unneeded self joins

2025-02-24 Thread Andrei Lepikhov
On 24/2/2025 11:57, Alexander Korotkov wrote: Hi, Andrei! Thank you for your feedback. On Mon, Feb 24, 2025 at 12:12 AM Andrei Lepikhov wrote: On 23/2/2025 22:15, Alexander Korotkov wrote: There is my attempt to implement this approach. Getting rid of local variable (and computation of the

Re: Removing unneeded self joins

2025-02-24 Thread Alexander Korotkov
Hi, Andrei! Thank you for your feedback. On Mon, Feb 24, 2025 at 12:12 AM Andrei Lepikhov wrote: > On 23/2/2025 22:15, Alexander Korotkov wrote: > > There is my attempt to implement this approach. Getting rid of local > > variable (and computation of the same value other way) required to > > ch

Re: Removing unneeded self joins

2025-02-23 Thread Andrei Lepikhov
On 23/2/2025 22:15, Alexander Korotkov wrote: There is my attempt to implement this approach. Getting rid of local variable (and computation of the same value other way) required to change arguments of remove_rel_from_eclass() as well. I'm going to further polish this tomorrow. I passed throug

Re: Removing unneeded self joins

2025-02-23 Thread Alexander Korotkov
On Sun, Feb 23, 2025 at 7:02 PM Tom Lane wrote: > > Alexander Korotkov writes: > > I've corrected some spelling error reported by Alexander Lakhin > > privately to me. Also, I've revised comments around ChangeVarNodes() > > and ChangeVarNodesExtended(). I'm going to continue nitpicking this > >

Re: Removing unneeded self joins

2025-02-23 Thread Tom Lane
Alexander Korotkov writes: > I've corrected some spelling error reported by Alexander Lakhin > privately to me. Also, I've revised comments around ChangeVarNodes() > and ChangeVarNodesExtended(). I'm going to continue nitpicking this > patch during next couple days then push it if no objections.

Re: Removing unneeded self joins

2025-02-17 Thread Alena Rybakina
On 13.02.2025 02:00, Alexander Korotkov wrote: Thank you. I've integrated that into a patch. However, I've to change keep_rinfo_list to be passed by pointer to add_non_redundant_clauses(), because it might be changed in distribute_restrictinfo_to_rels(). Without that there is a case of duplica

Re: Removing unneeded self joins

2025-02-11 Thread Alena Rybakina
Hi! Thank you for the work with this subject, I think it is really important. On 10.02.2025 22:58, Alexander Korotkov wrote: Hi! On Mon, Feb 10, 2025 at 7:19 AM Andrei Lepikhov wrote: On 9/2/2025 18:41, Alexander Korotkov wrote: Regarding adjust_relid_set() and replace_relid(). I think the

Re: Removing unneeded self joins

2025-02-09 Thread Andrei Lepikhov
On 9/2/2025 18:41, Alexander Korotkov wrote: Regarding adjust_relid_set() and replace_relid(). I think they are now strictly equivalent, except for the case then old relid is given and not found. In this case adjust_relid_set() returns the original relids while replace_relid() returns a copy.

Re: Removing unneeded self joins

2024-12-11 Thread Andrei Lepikhov
On 9/12/2024 13:03, Alexander Korotkov wrote: remove_self_join_rel(). Should we better add comments to PlannerInfo and other relevant structures saying: if you're going to modify this, consider how that affects remove_self_join_rel()? Any thoughts? As I see, it is quite typical to keep two part

Re: Removing unneeded self joins

2024-12-09 Thread Alexander Korotkov
On Sat, Jul 20, 2024 at 2:38 PM Alexander Korotkov wrote: > We initially didn't use ChangeVarNodes() in SJE at all. See the last > patch version without it [1]. We're trying to address Tom Lane's > proposal to re-use more of existing tree-manipulation infrastructure > [2]. I agree with you that

Re: Removing unneeded self joins

2024-07-24 Thread Dean Rasheed
On Sat, 20 Jul 2024 at 12:39, Alexander Korotkov wrote: > > > The new function replace_relid() looks to be the same as adjust_relid_set(). > > They are similar, not the same. replace_relid() has handling for > negative newId, while adjust_relid_set() hasn't. One thing I'd like > to borrow from a

Re: Removing unneeded self joins

2024-07-21 Thread Andrei Lepikhov
On 20/7/2024 18:38, Alexander Korotkov wrote: On Fri, Jul 19, 2024 at 11:30 AM Dean Rasheed wrote: On Wed, 17 Jul 2024 at 01:45, Alexander Korotkov wrote: We initially didn't use ChangeVarNodes() in SJE at all. See the last patch version without it [1]. We're trying to address Tom Lane's pr

Re: Removing unneeded self joins

2024-07-19 Thread Dean Rasheed
On Wed, 17 Jul 2024 at 01:45, Alexander Korotkov wrote: > > Jian He gave a try to ChangeVarNodes() [1]. That gives some > improvement, but the vast majority of complexity is still here. I > think the reason for complexity of SJE is that it's the first time we > remove relation, which is actually

Re: Removing unneeded self joins

2024-07-17 Thread Pogosyan Vardan
On 16.07.2024 21:30, Alexander Korotkov wrote: Hi, Vardan! Great, thank you! On Tue, Jul 16, 2024 at 5:26 PM Вардан Погосян wrote: I did the SJE testing at Andrey's request. To do this, I used the automatic testing tool EET (Equivalent Expression Transformation) [1] with some modifications.

Re: Removing unneeded self joins

2024-07-16 Thread Alexander Korotkov
Hi, Tom! I'd like to give you and update on the progress with SJE. On Mon, May 6, 2024 at 6:54 PM Tom Lane wrote: > Robert Haas writes: > > I want to go on record right now as disagreeing with the plan proposed > > in the commit message for the revert commit, namely, committing this > > again e

Re: Removing unneeded self joins

2024-07-16 Thread Alexander Korotkov
Hi, Vardan! Great, thank you! On Tue, Jul 16, 2024 at 5:26 PM Вардан Погосян wrote: > I did the SJE testing at Andrey's request. > To do this, I used the automatic testing tool EET (Equivalent Expression > Transformation) [1] with some modifications. > EET transforms the logical conditions in a

Re: Removing unneeded self joins

2024-07-16 Thread Вардан Погосян
Hi! I did the SJE testing at Andrey's request.To do this, I used the automatic testing tool EET (Equivalent _expression_ Transformation) [1] with some modifications. EET transforms the logical conditions in a query, creating multiple queries waiting for the same number of rows. In order to make sur

Re: Removing unneeded self joins

2024-07-15 Thread Andrei Lepikhov
On 7/15/24 14:35, jian he wrote: On Mon, Jul 15, 2024 at 2:08 PM Andrei Lepikhov wrote: On 7/15/24 12:31, jian he wrote: hi. Here is the latest patch (v6), I've made the following changes. * disallow original Query->resultRelation participate in SJE. for SELECT, nothing is changed. for UPDAT

Re: Re: Removing unneeded self joins

2024-07-15 Thread jian he
On Mon, Jul 15, 2024 at 2:08 PM Andrei Lepikhov wrote: > > On 7/15/24 12:31, jian he wrote: > > hi. > > Here is the latest patch (v6), > > I've made the following changes. > > > > * disallow original Query->resultRelation participate in SJE. > > for SELECT, nothing is changed. for UPDATE/DELETE/ME

Re: Removing unneeded self joins

2024-07-14 Thread Andrei Lepikhov
On 7/15/24 12:31, jian he wrote: hi. Here is the latest patch (v6), I've made the following changes. * disallow original Query->resultRelation participate in SJE. for SELECT, nothing is changed. for UPDATE/DELETE/MERGE we can do: EXPLAIN (COSTS OFF) UPDATE sj sq SET b = sq.b + sz.a FROM (select

Re: Removing unneeded self joins

2024-07-12 Thread Alexander Korotkov
On Fri, Jul 12, 2024 at 1:30 PM Alexander Korotkov wrote: > On Fri, Jul 12, 2024 at 6:05 AM Andrei Lepikhov wrote: > > On 7/11/24 14:43, jian he wrote: > > > On Tue, Jul 9, 2024 at 2:06 PM Andrei Lepikhov wrote: > > >> > > >> On 7/2/24 07:25, jian he wrote: > > >>> to make sure it's correct, I h

Re: Removing unneeded self joins

2024-07-12 Thread Alexander Korotkov
Hi, Andrei! On Fri, Jul 12, 2024 at 6:05 AM Andrei Lepikhov wrote: > > On 7/11/24 14:43, jian he wrote: > > On Tue, Jul 9, 2024 at 2:06 PM Andrei Lepikhov wrote: > >> > >> On 7/2/24 07:25, jian he wrote: > >>> to make sure it's correct, I have added a lot of tests, > >>> Some of this may be cont

Re: Removing unneeded self joins

2024-07-11 Thread Andrei Lepikhov
On 7/11/24 14:43, jian he wrote: On Tue, Jul 9, 2024 at 2:06 PM Andrei Lepikhov wrote: On 7/2/24 07:25, jian he wrote: to make sure it's correct, I have added a lot of tests, Some of this may be contrived, maybe some of the tests are redundant. Thanks for your job! I passed through the patch

Re: Removing unneeded self joins

2024-07-08 Thread Alexander Korotkov
On Thu, Jul 4, 2024 at 11:40 AM jian he wrote: > On Thu, Jul 4, 2024 at 11:04 AM Alexander Korotkov > wrote: > > > > On Thu, Jul 4, 2024 at 5:15 AM jian he wrote: > > > in remove_self_join_rel, i have > > > ```ChangeVarNodes((Node *) root->parse, toRemove->relid, toKeep->relid, > > > 0);``` >

Re: Removing unneeded self joins

2024-07-04 Thread jian he
On Thu, Jul 4, 2024 at 11:04 AM Alexander Korotkov wrote: > > On Thu, Jul 4, 2024 at 5:15 AM jian he wrote: > > in remove_self_join_rel, i have > > ```ChangeVarNodes((Node *) root->parse, toRemove->relid, toKeep->relid, > > 0);``` > > which will change the joinlist(RangeTblRef) from (1,2) to (

Re: Removing unneeded self joins

2024-07-03 Thread Alexander Korotkov
On Thu, Jul 4, 2024 at 5:15 AM jian he wrote: > in remove_self_join_rel, i have > ```ChangeVarNodes((Node *) root->parse, toRemove->relid, toKeep->relid, 0);``` > which will change the joinlist(RangeTblRef) from (1,2) to (2,2). > Immediately after this call, I wrote a function (restore_rangetblr

Re: Removing unneeded self joins

2024-07-03 Thread jian he
On Wed, Jul 3, 2024 at 11:39 AM Alexander Korotkov wrote: > > On Mon, Jun 17, 2024 at 3:00 AM jian he wrote: > > On Mon, May 6, 2024 at 11:55 PM Tom Lane wrote: > > > > > > Robert Haas writes: > > > > I want to go on record right now as disagreeing with the plan proposed > > > > in the commit m

Re: Removing unneeded self joins

2024-07-02 Thread Alexander Korotkov
On Mon, Jun 17, 2024 at 3:00 AM jian he wrote: > On Mon, May 6, 2024 at 11:55 PM Tom Lane wrote: > > > > Robert Haas writes: > > > I want to go on record right now as disagreeing with the plan proposed > > > in the commit message for the revert commit, namely, committing this > > > again early i

Re: Removing unneeded self joins

2024-06-16 Thread Alexander Korotkov
Hi! On Thu, Jun 13, 2024 at 6:45 AM Andrei Lepikhov wrote: > On 5/7/24 02:59, Robert Haas wrote: > > On Mon, May 6, 2024 at 3:27 PM Alexander Korotkov > > wrote: > >> I agree it was a hurry to put the plan into commit message. I think > >> Tom already gave valuable feedback [1] and probably we

Re: Removing unneeded self joins

2024-05-06 Thread Robert Haas
On Mon, May 6, 2024 at 3:27 PM Alexander Korotkov wrote: > I agree it was a hurry to put the plan into commit message. I think > Tom already gave valuable feedback [1] and probably we will get more. > So, plan is to be decided. One way or the other I'm not going to > re-commit this without expli

Re: Removing unneeded self joins

2024-05-06 Thread Alexander Korotkov
On Mon, May 6, 2024 at 5:44 PM Robert Haas wrote: > I want to go on record right now as disagreeing with the plan proposed > in the commit message for the revert commit, namely, committing this > again early in the v18 cycle. I don't think Tom would have proposed > reverting this feature unless he

Re: Removing unneeded self joins

2024-05-06 Thread Alexander Korotkov
On Mon, May 6, 2024 at 6:54 PM Tom Lane wrote: > Robert Haas writes: > > I want to go on record right now as disagreeing with the plan proposed > > in the commit message for the revert commit, namely, committing this > > again early in the v18 cycle. I don't think Tom would have proposed > > reve

Re: Removing unneeded self joins

2024-05-06 Thread Bruce Momjian
On Mon, May 6, 2024 at 12:24:41PM -0400, Robert Haas wrote: > Now that being said, I do also agree that the planner code is quite > hard to understand, for various reasons. I don't think the structure > of that code and the assumptions underlying it are as well-documented > as they could be, and n

Re: Removing unneeded self joins

2024-05-06 Thread Robert Haas
On Mon, May 6, 2024 at 12:01 PM Andrei Lepikhov wrote: > Right now, it evolves extensively - new structures, variables, > alternative copies of the same node trees with slightly changed > properties ... This way allows us to quickly introduce some planning > features (a lot of changes in planner l

Re: Removing unneeded self joins

2024-05-06 Thread Andrei Lepikhov
On 6/5/2024 21:44, Robert Haas wrote: On Sat, May 4, 2024 at 10:46 PM Andrei Lepikhov wrote: Having no objective negative feedback, we have no reason to change anything in the design or any part of the code. It looks regrettable and unusual. To me, this sounds like you think it's someone else

Re: Removing unneeded self joins

2024-05-06 Thread Tom Lane
Robert Haas writes: > I want to go on record right now as disagreeing with the plan proposed > in the commit message for the revert commit, namely, committing this > again early in the v18 cycle. I don't think Tom would have proposed > reverting this feature unless he believed that it had more ser

Re: Removing unneeded self joins

2024-05-06 Thread Bruce Momjian
On Mon, May 6, 2024 at 10:44:33AM -0400, Robert Haas wrote: > I want to go on record right now as disagreeing with the plan proposed > in the commit message for the revert commit, namely, committing this > again early in the v18 cycle. I don't think Tom would have proposed > reverting this feature

Re: Removing unneeded self joins

2024-05-06 Thread Robert Haas
On Sat, May 4, 2024 at 10:46 PM Andrei Lepikhov wrote: > Having no objective negative feedback, we have no reason to change > anything in the design or any part of the code. It looks regrettable and > unusual. To me, this sounds like you think it's someone else's job to tell you what is wrong wit

Re: Removing unneeded self joins

2024-05-04 Thread Andrei Lepikhov
On 3/5/2024 20:55, Robert Haas wrote: One of my most embarrassing gaffes in this area personally was a448e49bcbe40fb72e1ed85af910dd216d45bad8. I don't know how I managed to commit the original patch without realizing it was going to cause an increase in the WAL size, but I can tell you that when

Re: Removing unneeded self joins

2024-05-03 Thread Robert Haas
On Fri, May 3, 2024 at 4:57 AM Alexander Korotkov wrote: > I agree to revert it for v17, but I'm not exactly sure the issue is > design (nevertheless design review is very welcome as any other type > of review). The experience of the bugs arising with the SJE doesn't > show me a particular weak s

Re: Removing unneeded self joins

2024-05-03 Thread Alexander Korotkov
Hi, Tom! On Fri, May 3, 2024 at 2:19 AM Tom Lane wrote: > Alexander Korotkov writes: > > I would appreciate your review of this patchset, and review from Andrei as > > well. > > I hate to say this ... but if we're still finding bugs this > basic in SJE, isn't it time to give up on it for v17? >

Re: Removing unneeded self joins

2024-05-02 Thread Andrei Lepikhov
On 5/3/24 06:19, Tom Lane wrote: Alexander Korotkov writes: I would appreciate your review of this patchset, and review from Andrei as well. I hate to say this ... but if we're still finding bugs this basic in SJE, isn't it time to give up on it for v17? I might feel better about it if there

Re: Removing unneeded self joins

2024-05-02 Thread Tom Lane
Alexander Korotkov writes: > I would appreciate your review of this patchset, and review from Andrei as > well. I hate to say this ... but if we're still finding bugs this basic in SJE, isn't it time to give up on it for v17? I might feel better about it if there were any reason to think these

Re: Removing unneeded self joins

2024-05-02 Thread Alexander Korotkov
Hi, Richard! On Thu, May 2, 2024 at 4:14 PM Richard Guo wrote: > On Thu, May 2, 2024 at 6:08 PM Alexander Korotkov > wrote: >> On Thu, May 2, 2024 at 12:45 PM Andrei Lepikhov >> wrote: >> > One question for me is: Do we anticipate other lateral self-references >> > except the TABLESAMPLE case?

Re: Removing unneeded self joins

2024-05-02 Thread Richard Guo
On Thu, May 2, 2024 at 6:08 PM Alexander Korotkov wrote: > On Thu, May 2, 2024 at 12:45 PM Andrei Lepikhov > wrote: > > One question for me is: Do we anticipate other lateral self-references > > except the TABLESAMPLE case? Looking into the extract_lateral_references > > implementation, I see th

Re: Removing unneeded self joins

2024-05-02 Thread Alexander Korotkov
On Thu, May 2, 2024 at 12:45 PM Andrei Lepikhov wrote: > > On 5/1/24 18:59, Alexander Korotkov wrote: > > I think we probably could forbid SJE for the tables with TABLESAMPLE > > altogether. Please, check the attached patch. > Your patch looks good to me. I added some comments and test case into

Re: Removing unneeded self joins

2024-05-02 Thread Andrei Lepikhov
On 5/1/24 18:59, Alexander Korotkov wrote: I think we probably could forbid SJE for the tables with TABLESAMPLE altogether. Please, check the attached patch. Your patch looks good to me. I added some comments and test case into the join.sql. One question for me is: Do we anticipate other late

Re: Removing unneeded self joins

2024-05-01 Thread Alexander Korotkov
On Wed, May 1, 2024 at 2:00 PM Alexander Lakhin wrote: > 30.04.2024 13:20, Alexander Korotkov wrote: > > On Tue, Apr 30, 2024 at 9:00 AM Alexander Lakhin > > wrote: > >> I've discovered another failure, introduced by d3d55ce57. > >> Please try the following: > >> CREATE TABLE t (a int unique, b

Re: Removing unneeded self joins

2024-05-01 Thread Alexander Lakhin
30.04.2024 13:20, Alexander Korotkov wrote: On Tue, Apr 30, 2024 at 9:00 AM Alexander Lakhin wrote: I've discovered another failure, introduced by d3d55ce57. Please try the following: CREATE TABLE t (a int unique, b float); SELECT * FROM t NATURAL JOIN LATERAL (SELECT * FROM t t2 TABLESAMPLE

Re: Removing unneeded self joins

2024-04-30 Thread Alexander Korotkov
On Tue, Apr 30, 2024 at 9:00 AM Alexander Lakhin wrote: > 23.10.2023 12:47, Alexander Korotkov wrote: > > I think this patch makes substantial improvement to query planning. > > It has received plenty of reviews. The code is currently in quite > > good shape. I didn't manage to find the cases wh

Re: Removing unneeded self joins

2024-04-30 Thread Alexander Korotkov
Hi, Alexander! On Tue, Apr 30, 2024 at 9:00 AM Alexander Lakhin wrote: > 23.10.2023 12:47, Alexander Korotkov wrote: > > I think this patch makes substantial improvement to query planning. > > It has received plenty of reviews. The code is currently in quite > > good shape. I didn't manage to f

Re: Removing unneeded self joins

2024-04-29 Thread Alexander Lakhin
Hello Alexander, 23.10.2023 12:47, Alexander Korotkov wrote: I think this patch makes substantial improvement to query planning. It has received plenty of reviews. The code is currently in quite good shape. I didn't manage to find the cases when this optimization causes significant overhead to

Re: Removing unneeded self joins

2024-02-24 Thread Noah Misch
Hello, On Sat, Feb 24, 2024 at 01:02:01PM +0200, Alexander Korotkov wrote: > On Sat, Feb 24, 2024 at 7:12 AM Noah Misch wrote: > > On Sat, Feb 24, 2024 at 12:36:59AM +0200, Alexander Korotkov wrote: > > > On Thu, Feb 22, 2024 at 10:51 AM Andrei Lepikhov > > > wrote: > > > > On 21/2/2024 14:26,

Re: Removing unneeded self joins

2024-02-24 Thread Alexander Korotkov
Hi, Noah! On Sat, Feb 24, 2024 at 7:12 AM Noah Misch wrote: > On Sat, Feb 24, 2024 at 12:36:59AM +0200, Alexander Korotkov wrote: > > On Thu, Feb 22, 2024 at 10:51 AM Andrei Lepikhov > > wrote: > > > On 21/2/2024 14:26, Richard Guo wrote: > > > > I think the right fix for these issues is to int

Re: Removing unneeded self joins

2024-02-23 Thread Noah Misch
On Sat, Feb 24, 2024 at 12:36:59AM +0200, Alexander Korotkov wrote: > On Thu, Feb 22, 2024 at 10:51 AM Andrei Lepikhov > wrote: > > On 21/2/2024 14:26, Richard Guo wrote: > > > I think the right fix for these issues is to introduce a new element > > > 'sublevels_up' in ReplaceVarnoContext, and en

Re: Removing unneeded self joins

2024-02-23 Thread Alexander Korotkov
On Thu, Feb 22, 2024 at 10:51 AM Andrei Lepikhov wrote: > On 21/2/2024 14:26, Richard Guo wrote: > > I think the right fix for these issues is to introduce a new element > > 'sublevels_up' in ReplaceVarnoContext, and enhance replace_varno_walker > > to 1) recurse into subselects with sublevels_up

Re: Removing unneeded self joins

2024-02-22 Thread Andrei Lepikhov
On 21/2/2024 14:26, Richard Guo wrote: I think the right fix for these issues is to introduce a new element 'sublevels_up' in ReplaceVarnoContext, and enhance replace_varno_walker to 1) recurse into subselects with sublevels_up increased, and 2) perform the replacement only when varlevelsup is eq

Re: Removing unneeded self joins

2024-02-21 Thread Andrei Lepikhov
On 21/2/2024 14:26, Richard Guo wrote: This patch also includes some cosmetic tweaks for SJE test cases.  It does not change the test cases using catalog tables though.  I think that should be a seperate patch. Thanks for this catch, it is really messy thing, keeping aside the question why we ne

Re: Removing unneeded self joins

2024-02-20 Thread Richard Guo
On Mon, Feb 19, 2024 at 1:24 PM Andrei Lepikhov wrote: > On 18/2/2024 23:18, Alexander Korotkov wrote: > > On Sun, Feb 18, 2024 at 5:04 PM Alexander Korotkov > wrote: > >> On Sun, Feb 18, 2024 at 3:00 PM Alexander Lakhin > wrote: > >>> Please look at the following query which fails with an erro

Re: Removing unneeded self joins

2024-02-18 Thread Andrei Lepikhov
On 18/2/2024 23:18, Alexander Korotkov wrote: On Sun, Feb 18, 2024 at 5:04 PM Alexander Korotkov wrote: On Sun, Feb 18, 2024 at 3:00 PM Alexander Lakhin wrote: 09.01.2024 01:09, Alexander Korotkov wrote: Fixed in 30b4955a46. Please look at the following query which fails with an error sinc

Re: Removing unneeded self joins

2024-02-18 Thread Alexander Lakhin
18.02.2024 19:18, Alexander Korotkov wrote: Attached is a draft patch fixing this query. Could you, please, recheck? Yes, this patch fixes the behavior for that query (I've also tried several similar queries). Though I don't know the code well enough to judge the code change. Best regards, Ale

Re: Removing unneeded self joins

2024-02-18 Thread Alexander Korotkov
On Sun, Feb 18, 2024 at 5:04 PM Alexander Korotkov wrote: > On Sun, Feb 18, 2024 at 3:00 PM Alexander Lakhin wrote: > > 09.01.2024 01:09, Alexander Korotkov wrote: > > > > Fixed in 30b4955a46. > > > > > > Please look at the following query which fails with an error since > > d3d55ce57: > > > > cr

Re: Removing unneeded self joins

2024-02-18 Thread Alexander Korotkov
On Sun, Feb 18, 2024 at 3:00 PM Alexander Lakhin wrote: > 09.01.2024 01:09, Alexander Korotkov wrote: > > Fixed in 30b4955a46. > > > Please look at the following query which fails with an error since > d3d55ce57: > > create table t (i int primary key); > > select t3.i from t t1 > join t t2 on t1.

Re: Removing unneeded self joins

2024-02-18 Thread Alexander Lakhin
Hi Alexander, 09.01.2024 01:09, Alexander Korotkov wrote: Fixed in 30b4955a46. Please look at the following query which fails with an error since d3d55ce57: create table t (i int primary key); select t3.i from t t1  join t t2 on t1.i = t2.i,  lateral (select t1.i limit 1) t3; ERROR:  non-LA

Re: Removing unneeded self joins

2024-01-09 Thread Alexander Korotkov
On Tue, Jan 9, 2024 at 8:08 AM Alexander Korotkov wrote: > On Tue, Jan 9, 2024 at 6:00 AM Alexander Lakhin wrote: > > 09.01.2024 01:09, Alexander Korotkov wrote: > > > Fixed in 30b4955a46. > > > > Thank you for fixing that! > > > > I've found another anomaly coined with d3d55ce57. This query: > >

Re: Removing unneeded self joins

2024-01-08 Thread Alexander Korotkov
On Tue, Jan 9, 2024 at 6:00 AM Alexander Lakhin wrote: > 09.01.2024 01:09, Alexander Korotkov wrote: > > Fixed in 30b4955a46. > > Thank you for fixing that! > > I've found another anomaly coined with d3d55ce57. This query: > CREATE TABLE t(a int PRIMARY KEY, b int); > INSERT INTO t VALUES (1, 1),

Re: Removing unneeded self joins

2024-01-08 Thread Alexander Lakhin
09.01.2024 01:09, Alexander Korotkov wrote: Fixed in 30b4955a46. Thank you for fixing that! I've found another anomaly coined with d3d55ce57. This query: CREATE TABLE t(a int PRIMARY KEY, b int); INSERT INTO t VALUES  (1, 1), (2, 1); WITH t1 AS (SELECT * FROM t) UPDATE t SET b = t1.b + 1 FROM

Re: Removing unneeded self joins

2024-01-08 Thread Alexander Korotkov
On Mon, Jan 8, 2024 at 10:20 PM Alexander Korotkov wrote: > On Mon, Jan 8, 2024 at 10:00 PM Alexander Lakhin wrote: > > Please look at the following query which produces an incorrect result since > > d3d55ce57: > > CREATE TABLE t(a int PRIMARY KEY, b int); > > INSERT INTO t VALUES (1, 1), (2, 1)

Re: Removing unneeded self joins

2024-01-08 Thread Alexander Korotkov
On Mon, Jan 8, 2024 at 10:00 PM Alexander Lakhin wrote: > Please look at the following query which produces an incorrect result since > d3d55ce57: > CREATE TABLE t(a int PRIMARY KEY, b int); > INSERT INTO t VALUES (1, 1), (2, 1); > SELECT * FROM t WHERE EXISTS (SELECT * FROM t t2 WHERE t2.a = t.b

Re: Removing unneeded self joins

2024-01-08 Thread Alexander Lakhin
Hello Andrei and Alexander, Please look at the following query which produces an incorrect result since d3d55ce57: CREATE TABLE t(a int PRIMARY KEY, b int); INSERT INTO t VALUES  (1, 1), (2, 1); SELECT * FROM t WHERE EXISTS (SELECT * FROM t t2 WHERE t2.a = t.b AND t2.b > 0);  a | b ---+---  1 |

Re: Removing unneeded self joins

2023-12-29 Thread Alexander Lakhin
Hi Andrei, 29.12.2023 12:58, Andrei Lepikhov wrote: Thanks for the report! The problem is with the resultRelation field. We forget to replace the relid here. Could you check your issue with the patch in the attachment? Does it resolve this case? Yes, with the patch applied I see no error. T

Re: Removing unneeded self joins

2023-12-29 Thread Andrei Lepikhov
On 29/12/2023 12:00, Alexander Lakhin wrote: Hi Alexander, 23.10.2023 14:29, Alexander Korotkov wrote: Fixed all of the above. Thank you for catching this! I've discovered that starting from d3d55ce57 the following query: CREATE TABLE t(a int PRIMARY KEY); WITH tt AS (SELECT * FROM t) UPDATE

  1   2   3   >