Re: Subplan result caching

2020-05-20 Thread Andy Fan
On Wed, May 20, 2020 at 7:47 PM David Rowley wrote: > On Mon, 27 Apr 2020 at 00:37, Andy Fan wrote: > > I was feeling that we may have to maintain some extra status if we use > hash > > table rather than tuple store, but that might be not a major concern. I > can > > wait and see your patch. >

Re: Subplan result caching

2020-05-20 Thread David Rowley
On Mon, 27 Apr 2020 at 00:37, Andy Fan wrote: > I was feeling that we may have to maintain some extra status if we use hash > table rather than tuple store, but that might be not a major concern. I can > wait and see your patch. I've posted the patch and lots of details about it in https://postg

Re: Subplan result caching

2020-04-26 Thread Andy Fan
On Sun, Apr 26, 2020 at 5:49 PM David Rowley wrote: > On Sun, 26 Apr 2020 at 19:08, Andy Fan wrote: > > If we want to handle this case as well, one of the changes would > > be it needs to cache multi records for one input parameter, or return > > one row each time but return mutli times for one

Re: Subplan result caching

2020-04-26 Thread David Rowley
On Sun, 26 Apr 2020 at 19:08, Andy Fan wrote: > If we want to handle this case as well, one of the changes would > be it needs to cache multi records for one input parameter, or return > one row each time but return mutli times for one input parameter, > Tuplestore may be a good option for this ca

Re: Subplan result caching

2020-04-26 Thread Andy Fan
On Sun, Apr 26, 2020 at 2:11 PM David Rowley wrote: > On 23 May 2018 at 21:31, Heikki Linnakangas wrote: > > I've been working on a patch to add a little cache to SubPlans, to speed > up > > queries with correlated subqueries, where the same subquery is currently > > executed multiple times with

Re: Subplan result caching

2018-10-01 Thread David Rowley
On 19 July 2018 at 06:27, Robert Haas wrote: > On Mon, Jul 9, 2018 at 5:08 AM, David Rowley >> "LazyMaterialize" seems like a good option for a name. It seems better >> than "LazyHash" since you may not want to restrict it to a hash table >> based cache in the future. A binary search tree may be

Re: Subplan result caching

2018-10-01 Thread Michael Paquier
On Mon, Jul 09, 2018 at 09:08:07PM +1200, David Rowley wrote: > I'll mark as waiting on author in the meantime. > > It's great to see someone working on this. Not that actively unfortunately. The patch status has remained unchanged since, so I am marking it as returned with feedback. -- Michael

Re: Subplan result caching

2018-07-18 Thread Robert Haas
On Mon, Jul 9, 2018 at 5:08 AM, David Rowley wrote: > From looking at the code I see that the caching is being done inside > nodeSubplan.c. I don't think this is the right approach to the > problem. The problem exists for any parameterized path, so I think a > more general approach would be much b

Re: Subplan result caching

2018-07-09 Thread David Rowley
On 23 May 2018 at 21:31, Heikki Linnakangas wrote: > I've been working on a patch to add a little cache to SubPlans, to speed up > queries with correlated subqueries, where the same subquery is currently > executed multiple times with the same parameters. The idea is to cache the > result of the s

Re: Subplan result caching

2018-07-01 Thread Peter Eisentraut
On 23.05.18 11:31, Heikki Linnakangas wrote: > I've been working on a patch to add a little cache to SubPlans, to speed > up queries with correlated subqueries, where the same subquery is > currently executed multiple times with the same parameters. The idea is > to cache the result of the subpl

Re: Subplan result caching

2018-05-25 Thread Robert Haas
On Wed, May 23, 2018 at 6:03 AM, Laurenz Albe wrote: > I think the cache should be limited in size, perhaps by work_mem. > Also, there probably should be a GUC for this, defaulting to "off". Eh, why? Generally we want performance optimizations turned on by default; otherwise only highly-knowledg

Re: Subplan result caching

2018-05-25 Thread Robert Haas
On Wed, May 23, 2018 at 12:51 PM, Tom Lane wrote: > Ah. That would work, though it'd make the number of subquery executions > even less predictable (since some logically-equal values would compare > as physically unequal). In most cases that seems fine. It might not be fine with the subquery co

Re: Subplan result caching

2018-05-23 Thread David Rowley
On 24 May 2018 at 04:25, Tom Lane wrote: > That's doable no doubt, but I wonder whether that leaves you in a place > that's any better than the plan-time-decorrelation approach you proposed > in the earlier thread. I liked that better TBH; this one seems like > a very ad-hoc reinvention of a hash

Re: Subplan result caching

2018-05-23 Thread Andres Freund
On 2018-05-23 12:51:38 -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > On 23/05/18 19:25, Tom Lane wrote: > >> To make this > >> patch safe, I think you'd need to grovel through the subquery and make > >> sure that the parameters are only used as inputs to operators that belong > >> to the

Re: Subplan result caching

2018-05-23 Thread Tom Lane
Heikki Linnakangas writes: > On 23/05/18 19:25, Tom Lane wrote: >> To make this >> patch safe, I think you'd need to grovel through the subquery and make >> sure that the parameters are only used as inputs to operators that belong >> to the type's default btree or hash opfamily. (Many other cases

Re: Subplan result caching

2018-05-23 Thread Heikki Linnakangas
On 23/05/18 19:25, Tom Lane wrote: Heikki Linnakangas writes: I've been working on a patch to add a little cache to SubPlans, to speed up queries with correlated subqueries, where the same subquery is currently executed multiple times with the same parameters. The idea is to cache the result of

Re: Subplan result caching

2018-05-23 Thread Tom Lane
Heikki Linnakangas writes: > I've been working on a patch to add a little cache to SubPlans, to speed > up queries with correlated subqueries, where the same subquery is > currently executed multiple times with the same parameters. The idea is > to cache the result of the subplan, with the corr

Re: Subplan result caching

2018-05-23 Thread David Rowley
On 23 May 2018 at 21:31, Heikki Linnakangas wrote: > I've been working on a patch to add a little cache to SubPlans, to speed up > queries with correlated subqueries, where the same subquery is currently > executed multiple times with the same parameters. The idea is to cache the > result of the s

Re: Subplan result caching

2018-05-23 Thread Vladimir Sitnikov
>I like the idea. +1 >Also, there probably should be a GUC for this, defaulting to "off". I think the feature could be on by default provided it can properly identify "volatile" functions/tables hidden behind views. Vladimir

Re: Subplan result caching

2018-05-23 Thread Laurenz Albe
Heikki Linnakangas wrote: > I've been working on a patch to add a little cache to SubPlans, to speed > up queries with correlated subqueries, where the same subquery is > currently executed multiple times with the same parameters. The idea is > to cache the result of the subplan, with the correl

Subplan result caching

2018-05-23 Thread Heikki Linnakangas
Hi, I've been working on a patch to add a little cache to SubPlans, to speed up queries with correlated subqueries, where the same subquery is currently executed multiple times with the same parameters. The idea is to cache the result of the subplan, with the correlation vars as the cache key