Re: parallelizing subplan execution (was: [HACKERS] explain and PARAM_EXEC)

2010-06-30 Thread Mark Wong
On Sat, Jun 26, 2010 at 6:01 PM, Robert Haas wrote: > On Fri, Jun 25, 2010 at 10:47 PM, Mark Wong wrote: >> http://pages.cs.wisc.edu/~dewitt/includes/publications.html >> >> Some of these papers aren't the type of parallelism we're talking >> about here, but the ones that I think are relevant tal

Re: parallelizing subplan execution (was: [HACKERS] explain and PARAM_EXEC)

2010-06-27 Thread Simon Riggs
On Sat, 2010-06-26 at 21:01 -0400, Robert Haas wrote: > The section (from that same paper) on parallelizing hash joins and > merge-join-over-sort is interesting, and I can definitely imagine > those techniques being a win for us. But I'm not too sure how we'd > know when to apply them - that is,

Re: parallelizing subplan execution (was: [HACKERS] explain and PARAM_EXEC)

2010-06-26 Thread Robert Haas
On Fri, Jun 25, 2010 at 10:47 PM, Mark Wong wrote: > http://pages.cs.wisc.edu/~dewitt/includes/publications.html > > Some of these papers aren't the type of parallelism we're talking > about here, but the ones that I think are relevant talk mostly about > parallelizing hash based joins.  I think w

Re: parallelizing subplan execution (was: [HACKERS] explain and PARAM_EXEC)

2010-06-25 Thread Mark Wong
Hi all, Sorry for jumping in over 4 months later... On Sat, Feb 20, 2010 at 8:25 PM, Robert Haas wrote: > On Sat, Feb 20, 2010 at 8:31 AM, Dimitri Fontaine > wrote: >>> This is really a topic for another thread, but at 100,000 feet it >>> seems to me that the hardest question is - how will you

Re: [HACKERS] explain and PARAM_EXEC

2010-04-12 Thread Yeb Havinga
The patch I sent earlier is flaud with respect to subplan parameter numbering, I counted from zero where the parParam list had to be used. Yeb Havinga wrote: See patch below against HEAD. Example of query against catalog: postgres=# explain verbose select oid::int + 1,(select oid from pg_cl

Re: [HACKERS] explain and PARAM_EXEC

2010-04-02 Thread Yeb Havinga
Tom Lane wrote: Anyway, I hadn't looked at your patch before, but now that I have, it's not even approximately what I was suggesting. What I thought you should do was change ruleutils.c to print the parameter expressions at the call site, ie in the T_SubPlan and T_AlternativeSubPlan cases in get

Re: [HACKERS] explain and PARAM_EXEC

2010-04-01 Thread Tom Lane
Yeb Havinga writes: > > > > > > > Tom Lane wrote: > > > In principle it could look something like [ etc etc ] Please do not send HTML-only email to the lists. It's a PITA to work with. Anyway, I hadn't looked at your patch before, but now that I have, it's not even approximately wh

Re: [HACKERS] explain and PARAM_EXEC

2010-04-01 Thread Yeb Havinga
Tom Lane wrote: In principle it could look something like (SubPlan N ($0 := b.oid)) but with a few parameters and a bunch of other stuff on the same line that would get out of hand.  The patch I submitted to implement this, hits bogus varno:65001 in get_vari

Re: [HACKERS] explain and PARAM_EXEC

2010-03-19 Thread Yeb Havinga
Tom Lane wrote: Robert Haas writes: So I guess there are two issues here: (1) somehow I feel like we should be telling the user what expression is being used to initialize $0, $1, etc. when they are PARAM_EXEC parameters; Maybe, but the only reasonable place to put it would be within

Re: parallelizing subplan execution (was: [HACKERS] explain and PARAM_EXEC)

2010-02-20 Thread Robert Haas
On Sat, Feb 20, 2010 at 8:31 AM, Dimitri Fontaine wrote: >> This is really a topic for another thread, but at 100,000 feet it >> seems to me that the hardest question is - how will you decide which >> operations to parallelize in the first place?  Actually making it >> happen is really hard, too,

Re: [HACKERS] explain and PARAM_EXEC

2010-02-20 Thread Dimitri Fontaine
Tom Lane writes: > Indeed, and if I were setting out to parallelize queries in PG (which > I am not), subplans would be the last thing I would think about. You > could put an enormous amount of work in there and have nothing much > to show for it, because the construct doesn't even arise in many

Re: [HACKERS] explain and PARAM_EXEC

2010-02-20 Thread Jaime Casanova
On Fri, Feb 19, 2010 at 10:22 PM, Tom Lane wrote: > Robert Haas writes: >> So I guess there are two issues here: (1) somehow I feel like we >> should be telling the user what expression is being used to initialize >> $0, $1, etc. when they are PARAM_EXEC parameters; > > Maybe, but the only reason

Re: [HACKERS] explain and PARAM_EXEC

2010-02-20 Thread Tom Lane
Robert Haas writes: > On Sat, Feb 20, 2010 at 7:53 AM, Greg Stark wrote: >> Perhaps it would be clearer to display the "(Subplan 1)" in a function >> call style format like Subplan1(b.oid) > I thought about that, too... maybe for 9.1 we should consider it. It > might be nice to add some sort o

Re: [HACKERS] explain and PARAM_EXEC

2010-02-20 Thread Tom Lane
Robert Haas writes: > On Sat, Feb 20, 2010 at 6:57 AM, Dimitri Fontaine >> Ok that's a far stretch from the question at hand, but would that be a >> plausible approach to have parallel queries in PostgreSQL ? > This is really a topic for another thread, but at 100,000 feet it > seems to me that t

parallelizing subplan execution (was: [HACKERS] explain and PARAM_EXEC)

2010-02-20 Thread Dimitri Fontaine
Robert Haas writes: > On Sat, Feb 20, 2010 at 6:57 AM, Dimitri Fontaine > wrote: >> How much does this stuff is dependent on the current state of the >> backend? > > A whole lot. Bad news. >> Ok that's a far stretch from the question at hand, but would that be a >> plausible approach to have pa

Re: [HACKERS] explain and PARAM_EXEC

2010-02-20 Thread Robert Haas
On Sat, Feb 20, 2010 at 7:53 AM, Greg Stark wrote: > On Sat, Feb 20, 2010 at 4:33 AM, Tom Lane wrote: >> It's really not much different from a function call with subplans as >> functions. > > Perhaps it would be clearer to display the "(Subplan 1)" in a function > call style format like Subplan1(

Re: [HACKERS] explain and PARAM_EXEC

2010-02-20 Thread Robert Haas
On Sat, Feb 20, 2010 at 6:57 AM, Dimitri Fontaine wrote: > Tom Lane writes: >> It's really not much different from a function call with subplans as >> functions.  The PARAM_EXEC stuff looks just like 1950's era >> non-reentrant function parameter passing mechanisms, back before anybody >> had tho

Re: [HACKERS] explain and PARAM_EXEC

2010-02-20 Thread Robert Haas
On Fri, Feb 19, 2010 at 11:58 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Feb 19, 2010 at 11:33 PM, Tom Lane wrote: >>> We can also fetch that tuple's >>> relfilenode and pass it to the subplan, which we do by setting the $0 >>> Param value before invoking the subplan. > >> Are the same

Re: [HACKERS] explain and PARAM_EXEC

2010-02-20 Thread Greg Stark
On Sat, Feb 20, 2010 at 4:33 AM, Tom Lane wrote: > It's really not much different from a function call with subplans as > functions. Perhaps it would be clearer to display the "(Subplan 1)" in a function call style format like Subplan1(b.oid) -- greg -- Sent via pgsql-hackers mailing list (pg

Re: [HACKERS] explain and PARAM_EXEC

2010-02-20 Thread Dimitri Fontaine
Tom Lane writes: > It's really not much different from a function call with subplans as > functions. The PARAM_EXEC stuff looks just like 1950's era > non-reentrant function parameter passing mechanisms, back before anybody > had thought of recursive functions and they passed a function's > param

Re: [HACKERS] explain and PARAM_EXEC

2010-02-19 Thread Tom Lane
Robert Haas writes: > On Fri, Feb 19, 2010 at 11:33 PM, Tom Lane wrote: >> We can also fetch that tuple's >> relfilenode and pass it to the subplan, which we do by setting the $0 >> Param value before invoking the subplan. > Are the same tuples in scope when evaluating the expression that sets >

Re: [HACKERS] explain and PARAM_EXEC

2010-02-19 Thread Robert Haas
On Fri, Feb 19, 2010 at 11:33 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Feb 19, 2010 at 10:22 PM, Tom Lane wrote: >>> Maybe, but the only reasonable place to put it would be within the >>> (SubPlan N) reference, > >> I thought maybe it could do something like this: > >> SubPlan 1 >>  

Re: [HACKERS] explain and PARAM_EXEC

2010-02-19 Thread Tom Lane
Robert Haas writes: > On Fri, Feb 19, 2010 at 10:22 PM, Tom Lane wrote: >> Maybe, but the only reasonable place to put it would be within the >> (SubPlan N) reference, > I thought maybe it could do something like this: > SubPlan 1 > Parameters: $0 := b.oid > -> Index Scan etc. No, that's t

Re: [HACKERS] explain and PARAM_EXEC

2010-02-19 Thread Robert Haas
On Fri, Feb 19, 2010 at 11:05 PM, Robert Haas wrote: > I don't think it's a stretch to say that I'm the one who is confused. > I am under the (perhaps faulty) impression that when evaluating an > expression there can only ever be three tuples in score: inner, outer, s/score/scope. > and scan.  S

Re: [HACKERS] explain and PARAM_EXEC

2010-02-19 Thread Robert Haas
On Fri, Feb 19, 2010 at 10:22 PM, Tom Lane wrote: > Robert Haas writes: >> So I guess there are two issues here: (1) somehow I feel like we >> should be telling the user what expression is being used to initialize >> $0, $1, etc. when they are PARAM_EXEC parameters; > > Maybe, but the only reason

Re: [HACKERS] explain and PARAM_EXEC

2010-02-19 Thread Tom Lane
Robert Haas writes: > So I guess there are two issues here: (1) somehow I feel like we > should be telling the user what expression is being used to initialize > $0, $1, etc. when they are PARAM_EXEC parameters; Maybe, but the only reasonable place to put it would be within the (SubPlan N) refere

[HACKERS] explain and PARAM_EXEC

2010-02-19 Thread Robert Haas
Consider the following (rather lame) query: rhaas=# explain (verbose) select (select oid from pg_class a where a.oid = b.relfilenode) from pg_class b; QUERY PLAN -