Re: Consider parallel for lateral subqueries with limit

2023-03-23 Thread Gregory Stark (as CFM)
This patch has been "Needs Review" and bouncing from CF to CF. It actually looks like it got quite a bit of design discussion and while James Coleman seems convinced of its safety it doesn't sound like Tom Lane and Robert Haas are yet and it doesn't look like that's going to happen in this CF. I t

Re: Consider parallel for lateral subqueries with limit

2022-09-26 Thread James Coleman
On Mon, Sep 26, 2022 at 10:37 AM Robert Haas wrote: > > On Thu, Sep 22, 2022 at 5:19 PM James Coleman wrote: > > > Your sample query gets a plan like this: > > > > > > Nested Loop Left Join (cost=0.00..1700245.00 rows=1 width=8) > > >-> Seq Scan on foo (cost=0.00..145.00 rows=1 wi

Re: Consider parallel for lateral subqueries with limit

2022-09-26 Thread Robert Haas
On Thu, Sep 22, 2022 at 5:19 PM James Coleman wrote: > > Your sample query gets a plan like this: > > > > Nested Loop Left Join (cost=0.00..1700245.00 rows=1 width=8) > >-> Seq Scan on foo (cost=0.00..145.00 rows=1 width=4) > >-> Limit (cost=0.00..170.00 rows=1 width=4) > >

Re: Consider parallel for lateral subqueries with limit

2022-09-24 Thread James Coleman
On Thu, Sep 22, 2022 at 5:19 PM James Coleman wrote: > > On Mon, Sep 19, 2022 at 4:29 PM Robert Haas wrote: > > > > On Mon, Sep 19, 2022 at 3:58 PM James Coleman wrote: > > > But in the case where there's correlation via LATERAL we already don't > > > guarantee unique executions for a given set

Re: Consider parallel for lateral subqueries with limit

2022-09-22 Thread James Coleman
On Mon, Sep 19, 2022 at 4:29 PM Robert Haas wrote: > > On Mon, Sep 19, 2022 at 3:58 PM James Coleman wrote: > > But in the case where there's correlation via LATERAL we already don't > > guarantee unique executions for a given set of params into the lateral > > subquery execution, right? For exam

Re: Consider parallel for lateral subqueries with limit

2022-09-19 Thread Robert Haas
On Mon, Sep 19, 2022 at 3:58 PM James Coleman wrote: > But in the case where there's correlation via LATERAL we already don't > guarantee unique executions for a given set of params into the lateral > subquery execution, right? For example, suppose we have: > > select * > from foo > left joi

Re: Consider parallel for lateral subqueries with limit

2022-09-19 Thread James Coleman
On Tue, Mar 1, 2022 at 5:35 PM Tom Lane wrote: > > But more generally, I don't think you've addressed the fundamental > concern, which is that a query involving Limit is potentially > nondeterministic (if it lacks a fully-deterministic ORDER BY), > so that different workers could get different ans

Re: Consider parallel for lateral subqueries with limit

2022-08-02 Thread Jacob Champion
This entry has been waiting on author input for a while (our current threshold is roughly two weeks), so I've marked it Returned with Feedback. Once you think the patchset is ready for review again, you (or any interested party) can resurrect the patch entry by visiting https://commitfest.pos

Re: Consider parallel for lateral subqueries with limit

2022-03-01 Thread Tom Lane
James Coleman writes: > On Tue, Jan 4, 2022 at 9:59 PM James Coleman wrote: >> On Tue, Jan 4, 2022 at 5:31 PM Tom Lane wrote: >>> I don't really see why this patch is even a little bit safe. > Suppose lateral is not in play. Then if we have a plan like: > Gather > NestLoop >

Re: Consider parallel for lateral subqueries with limit

2022-01-13 Thread James Coleman
On Tue, Jan 4, 2022 at 9:59 PM James Coleman wrote: > > On Tue, Jan 4, 2022 at 5:31 PM Tom Lane wrote: > > > > Greg Nancarrow writes: > > > The patch LGTM. > > > I have set the status to "Ready for Committer". > > > > I don't really see why this patch is even a little bit safe. > > The argument

Re: Consider parallel for lateral subqueries with limit

2022-01-04 Thread James Coleman
On Tue, Jan 4, 2022 at 5:31 PM Tom Lane wrote: > > Greg Nancarrow writes: > > The patch LGTM. > > I have set the status to "Ready for Committer". > > I don't really see why this patch is even a little bit safe. > The argument for it seems to be that a lateral subquery will > necessarily be execut

Re: Consider parallel for lateral subqueries with limit

2022-01-04 Thread Tom Lane
Greg Nancarrow writes: > The patch LGTM. > I have set the status to "Ready for Committer". I don't really see why this patch is even a little bit safe. The argument for it seems to be that a lateral subquery will necessarily be executed in such a way that each complete iteration of the subquery,

Re: Consider parallel for lateral subqueries with limit

2021-11-03 Thread Greg Nancarrow
On Thu, Nov 4, 2021 at 12:49 AM James Coleman wrote: > > Greg, > > Do you believe this is now ready for committer? > The patch LGTM. I have set the status to "Ready for Committer". Regards, Greg Nancarrow Fujitsu Australia

Re: Consider parallel for lateral subqueries with limit

2021-11-03 Thread James Coleman
On Fri, Jul 16, 2021 at 3:16 PM James Coleman wrote: > > On Thu, May 27, 2021 at 9:01 PM Greg Nancarrow wrote: > > > > On Tue, Dec 8, 2020 at 10:46 AM James Coleman wrote: > > > > > > While I haven't actually tracked down to guarantee this is handled > > > elsewhere, a thought experiment -- I th

Re: Consider parallel for lateral subqueries with limit

2021-07-16 Thread James Coleman
On Thu, May 27, 2021 at 9:01 PM Greg Nancarrow wrote: > > On Tue, Dec 8, 2020 at 10:46 AM James Coleman wrote: > > > > While I haven't actually tracked down to guarantee this is handled > > elsewhere, a thought experiment -- I think -- shows it must be so. > > Here's why: suppose we don't have a

Re: Consider parallel for lateral subqueries with limit

2021-05-27 Thread Greg Nancarrow
On Tue, Dec 8, 2020 at 10:46 AM James Coleman wrote: > > While I haven't actually tracked down to guarantee this is handled > elsewhere, a thought experiment -- I think -- shows it must be so. > Here's why: suppose we don't have a limit here, but the query return > order is different in different

Re: Consider parallel for lateral subqueries with limit

2021-03-10 Thread David Steele
On 12/7/20 6:45 PM, James Coleman wrote: On Sun, Dec 6, 2020 at 7:34 PM Brian Davis wrote: Played around with this a bit, here's a non-correlated subquery that gets us to that if statement While I haven't actually tracked down to guarantee this is handled elsewhere, a thought experiment --

Re: Consider parallel for lateral subqueries with limit

2020-12-07 Thread James Coleman
Brian's email didn't keep the relevant headers, and so didn't show up as a reply, so I've pasted it here and am replying in this original thread. You can find the original at [1]. On Sun, Dec 6, 2020 at 7:34 PM Brian Davis wrote: > > > Note that near the end of grouping planner we have a similar

Re: Consider parallel for lateral subqueries with limit

2020-12-06 Thread Brian Davis
> Note that near the end of grouping planner we have a similar check: > > if (final_rel->consider_parallel && root->query_level > 1 && > !limit_needed(parse)) > > guarding copying the partial paths from the current rel to the final > rel. I haven't managed to come up with a test case that

Re: Consider parallel for lateral subqueries with limit

2020-12-01 Thread James Coleman
On Mon, Nov 30, 2020 at 7:00 PM James Coleman wrote: > > I've been investigating parallelizing certain correlated subqueries, > and during that work stumbled across the fact that > set_rel_consider_parallel disallows parallel query on what seems like > a fairly simple case. > > Consider this query

Consider parallel for lateral subqueries with limit

2020-11-30 Thread James Coleman
I've been investigating parallelizing certain correlated subqueries, and during that work stumbled across the fact that set_rel_consider_parallel disallows parallel query on what seems like a fairly simple case. Consider this query: select t.unique1 from tenk1 t join lateral (select t.unique1 fro