Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-10 Thread Richard Guo
On Fri, Apr 7, 2023 at 3:28 PM Richard Guo wrote: > On Tue, Aug 2, 2022 at 3:13 PM Richard Guo wrote: > >> On Sun, Jul 31, 2022 at 12:07 AM Tom Lane wrote: >> >>> [ wanders away wondering if JOIN_RIGHT_SEMI should become a thing ... ] >> >> Maybe this is something we can do. Currently for the q

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-07 Thread Richard Guo
On Tue, Aug 2, 2022 at 3:13 PM Richard Guo wrote: > On Sun, Jul 31, 2022 at 12:07 AM Tom Lane wrote: > >> [ wanders away wondering if JOIN_RIGHT_SEMI should become a thing ... ] > > Maybe this is something we can do. Currently for the query below: > > # explain select * from foo where a in (sele

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-06 Thread Thomas Munro
On Thu, Apr 6, 2023 at 6:40 PM Richard Guo wrote: > Seems it wins if the parallel scan becomes part of a hash join in final > plan. I wonder if we have a way to know that in this early stage. I haven't tried but I'm not sure off the top of my head how to make a decision that early unless it's su

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-05 Thread Richard Guo
On Thu, Apr 6, 2023 at 1:06 PM Tom Lane wrote: > This: > > > +#if 0 > > /* If any limit was set to zero, the user doesn't want a > > parallel scan. */ > > if (parallel_workers <= 0) > > return; > > +#endif > > seems like it adds a lot of new paths with a lot lower

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-05 Thread Richard Guo
On Thu, Apr 6, 2023 at 8:18 AM Thomas Munro wrote: > On Thu, Apr 6, 2023 at 9:11 AM Tom Lane wrote: > > Richard Guo writes: > > > Thanks for reminding. Attached is the rebased patch, with no other > > > changes. I think the patch is ready for commit. > > > > Pushed after a little further fool

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-05 Thread Tom Lane
Thomas Munro writes: > ... It works if you're OK creating partial paths > for everything... Hmm. The committed patch already causes us to investigate more paths than before, which I was okay with because it only costs more if there's an antijoin involved --- which it seems like there's at least

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-05 Thread Thomas Munro
On Thu, Apr 6, 2023 at 12:17 PM Thomas Munro wrote: > I tried the original example from the top of this thread and saw a > decent speedup from parallelism, but only if I set > min_parallel_table_scan_size=0, and otherwise it doesn't choose > Parallel Hash Right Anti Join. Same if I embiggen bar s

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-05 Thread Thomas Munro
On Thu, Apr 6, 2023 at 9:11 AM Tom Lane wrote: > Richard Guo writes: > > Thanks for reminding. Attached is the rebased patch, with no other > > changes. I think the patch is ready for commit. > > Pushed after a little further fooling with the comments. I also had > to rebase it over 11c2d6fdf

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-05 Thread Tom Lane
Richard Guo writes: > Thanks for reminding. Attached is the rebased patch, with no other > changes. I think the patch is ready for commit. Pushed after a little further fooling with the comments. I also had to rebase it over 11c2d6fdf (Parallel Hash Full Join). I think I did that correctly, b

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-03-15 Thread Richard Guo
On Wed, Mar 15, 2023 at 2:25 AM Gregory Stark (as CFM) wrote: > So what is the status of this patch? > > It looks like you received some feedback from Emre, Tom, Ronan, and > Alvaro but it also looks like you responded to most or all of that. > Are you still blocked waiting for feedback? Anything

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-03-14 Thread Gregory Stark (as CFM)
So what is the status of this patch? It looks like you received some feedback from Emre, Tom, Ronan, and Alvaro but it also looks like you responded to most or all of that. Are you still blocked waiting for feedback? Anything specific you need help with? Or is the patch ready for commit now? In w

Re: Using each rel as both outer and inner for JOIN_ANTI

2022-08-10 Thread Richard Guo
On Wed, Aug 10, 2022 at 4:40 PM Alvaro Herrera wrote: > On 2022-Aug-10, Richard Guo wrote: > > > The right-anti join plan has the same cost estimation with right join > > plan in this case. So would you please help to test what the right join > > plan looks like in your env for the query below? >

Re: Using each rel as both outer and inner for JOIN_ANTI

2022-08-10 Thread Alvaro Herrera
On 2022-Aug-10, Richard Guo wrote: > The right-anti join plan has the same cost estimation with right join > plan in this case. So would you please help to test what the right join > plan looks like in your env for the query below? > > select * from foo left join bar on foo.a = bar.c; You're ri

Re: Using each rel as both outer and inner for JOIN_ANTI

2022-08-10 Thread Richard Guo
On Tue, Aug 9, 2022 at 6:54 PM Alvaro Herrera wrote: > I suppose this looks good as far as the plan goes, but the cost estimation > might be a little bit too optimistic: it is reporting that the new plan > costs 50% of the original, yet the execution time is only 5% lower. Thanks for trying thi

Re: Using each rel as both outer and inner for JOIN_ANTI

2022-08-09 Thread Alvaro Herrera
Just for kicks, I ran query in your original post under EXPLAIN ANALYZE in both patched and unpatched with this last version. I got this (best of three): Unpatched: 55432 16devel 437532=# explain (analyze, buffers) select * from foo left join bar on foo.a = bar.c where bar.c is null;

Re: Using each rel as both outer and inner for JOIN_ANTI

2022-08-09 Thread Richard Guo
On Tue, Aug 2, 2022 at 3:13 PM Richard Guo wrote: > On Sun, Jul 31, 2022 at 12:07 AM Tom Lane wrote: > >> I took a quick look through this. The executor changes are indeed >> impressively short, but that's largely because you've paid zero >> attention to updating obsoleted comments. For exampl

Re: Using each rel as both outer and inner for JOIN_ANTI

2022-08-02 Thread Richard Guo
On Sun, Jul 31, 2022 at 12:07 AM Tom Lane wrote: > Richard Guo writes: > > [ v4-0001-Using-each-rel-as-both-outer-and-inner-for-anti-j.patch ] > > I took a quick look through this. The executor changes are indeed > impressively short, but that's largely because you've paid zero > attention to u

Re: Using each rel as both outer and inner for JOIN_ANTI

2022-07-30 Thread Tom Lane
Richard Guo writes: > [ v4-0001-Using-each-rel-as-both-outer-and-inner-for-anti-j.patch ] I took a quick look through this. The executor changes are indeed impressively short, but that's largely because you've paid zero attention to updating obsoleted comments. For example, in nodeHashjoin.c th

Re: Using each rel as both outer and inner for JOIN_ANTI

2022-07-24 Thread Richard Guo
On Fri, Jul 2, 2021 at 11:23 AM Richard Guo wrote: > Thanks! Test cases are updated in v3 patch. Also merge join can do the > 'right anti join' too in the same patch. > > Thanks again for reviewing this patch. > Rebased this patch with latest master, with no other changes. Thanks Richard v4-0

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-07-21 Thread Richard Guo
On Fri, Jul 2, 2021 at 11:59 AM Zhihong Yu wrote: > > > On Thu, Jul 1, 2021 at 8:24 PM Richard Guo wrote: > >> >> On Thu, Jul 1, 2021 at 3:18 PM Ronan Dunklau >> wrote: >> >>> Le jeudi 1 juillet 2021, 09:09:38 CEST Ronan Dunklau a écrit : >>> > > Yes, thanks! I was making a big mistake here thi

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-07-01 Thread Zhihong Yu
On Thu, Jul 1, 2021 at 8:24 PM Richard Guo wrote: > > On Thu, Jul 1, 2021 at 3:18 PM Ronan Dunklau > wrote: > >> Le jeudi 1 juillet 2021, 09:09:38 CEST Ronan Dunklau a écrit : >> > > Yes, thanks! I was making a big mistake here thinking the executor can >> > > stop after the first match. That's

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-07-01 Thread Richard Guo
On Thu, Jul 1, 2021 at 3:18 PM Ronan Dunklau wrote: > Le jeudi 1 juillet 2021, 09:09:38 CEST Ronan Dunklau a écrit : > > > Yes, thanks! I was making a big mistake here thinking the executor can > > > stop after the first match. That's not true. We need to use each outer > > > tuple to find all th

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-07-01 Thread Ronan Dunklau
Le jeudi 1 juillet 2021, 09:09:38 CEST Ronan Dunklau a écrit : > > Yes, thanks! I was making a big mistake here thinking the executor can > > stop after the first match. That's not true. We need to use each outer > > tuple to find all the matches and mark the corresponding hashtable > > entries. I

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-07-01 Thread Ronan Dunklau
> Yes, thanks! I was making a big mistake here thinking the executor can > stop after the first match. That's not true. We need to use each outer > tuple to find all the matches and mark the corresponding hashtable > entries. I have updated the patch with the fix. It looks OK to me. > > > I think

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-06-29 Thread Richard Guo
On Tue, Jun 29, 2021 at 10:41 PM Ronan Dunklau wrote: > Le mardi 29 juin 2021, 10:55:59 CEST Richard Guo a écrit : > > On Tue, Jun 29, 2021 at 3:55 PM Emre Hasegeli wrote: > > > > Thanks for the explanation. Attached is a demo code for the hash-join > > > > case, which is only for PoC to show ho

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-06-29 Thread Ronan Dunklau
Le mardi 29 juin 2021, 10:55:59 CEST Richard Guo a écrit : > On Tue, Jun 29, 2021 at 3:55 PM Emre Hasegeli wrote: > > > Thanks for the explanation. Attached is a demo code for the hash-join > > > case, which is only for PoC to show how we can make it work. It's far > > > from complete, at least we

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-06-29 Thread Richard Guo
On Tue, Jun 29, 2021 at 3:55 PM Emre Hasegeli wrote: > > Thanks for the explanation. Attached is a demo code for the hash-join > > case, which is only for PoC to show how we can make it work. It's far > > from complete, at least we need to adjust the cost calculation for this > > 'right anti join

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-06-29 Thread Emre Hasegeli
> Thanks for the explanation. Attached is a demo code for the hash-join > case, which is only for PoC to show how we can make it work. It's far > from complete, at least we need to adjust the cost calculation for this > 'right anti join'. I applied the patch and executed some queries. Hash Right

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-06-26 Thread Richard Guo
On Thu, Jun 24, 2021 at 10:14 PM Tom Lane wrote: > Heikki Linnakangas writes: > > On 24/06/2021 12:50, Richard Guo wrote: > >> I believe if we use the smaller table 'foo' as inner side for this > >> query, we would have a cheaper plan. > > > How would that work? > > I think you could make it wor

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-06-24 Thread Tom Lane
Heikki Linnakangas writes: > On 24/06/2021 12:50, Richard Guo wrote: >> I believe if we use the smaller table 'foo' as inner side for this >> query, we would have a cheaper plan. > How would that work? I think you could make it work for the hash-join case by extending the existing mechanism for

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-06-24 Thread Heikki Linnakangas
On 24/06/2021 12:50, Richard Guo wrote: Hi hackers, We may have anti-joins in several cases. Sublinks of 'NOT EXISTS' may be pulled up as anti-joins. Left joins whose join quals are strict for any nullable var that is forced null by higher qual levels will also be reduced to anti-joins. So anti-

Using each rel as both outer and inner for JOIN_ANTI

2021-06-24 Thread Richard Guo
Hi hackers, We may have anti-joins in several cases. Sublinks of 'NOT EXISTS' may be pulled up as anti-joins. Left joins whose join quals are strict for any nullable var that is forced null by higher qual levels will also be reduced to anti-joins. So anti-joins are very commonly used in practice.