Re: [HACKERS] Pull up aggregate subquery

2011-06-04 Thread Hitoshi Harada
2011/6/4 Simon Riggs : > > I like your simple patch and looks like it fixes your concern. Thanks for your interest. I forgot to mention but this type of query is quite general in one-to-many entities and likely to be generated by simple ORMappers. > Your problem statement ignores the fact that mo

Re: [HACKERS] Pull up aggregate subquery

2011-06-04 Thread Simon Riggs
On Tue, May 24, 2011 at 3:47 AM, Hitoshi Harada wrote: > That's true. But if the planning cost is an only issue, why not adding > new GUC for user to choose if they prefer it or not? Of course if we > have some method to predict which way to go before proving both ways, > it's great. Do you have

Re: [HACKERS] Pull up aggregate subquery

2011-05-25 Thread Tom Lane
Robert Haas writes: > I think getting it working is probably a good first goal. I am not > really sure that we want to commit it that way, and I think my vote > would be for you to work on the approach we discussed before rather > than this one, but it's your project, and I think you'll probably

Re: [HACKERS] Pull up aggregate subquery

2011-05-25 Thread Robert Haas
On Wed, May 25, 2011 at 1:37 PM, Hitoshi Harada wrote: >> How do you decide whether or not to push down? > > Yeah, that's the problem. In addition to the conditions of join-qual > == grouping key && outer is unique on qual, we need some criteria if > it should be done. At first I started to think

Re: [HACKERS] Pull up aggregate subquery

2011-05-25 Thread Ross J. Reedstrom
On Mon, May 23, 2011 at 11:08:40PM -0400, Robert Haas wrote: > > I don't really like the idea of adding a GUC for this, unless we > convince ourselves that nothing else is sensible. I mean, that leads > to conversations like this: > > Newbie: My query is slow. > Hacker: Turn on enable_magic_pixi

Re: [HACKERS] Pull up aggregate subquery

2011-05-25 Thread Hitoshi Harada
2011/5/26 Robert Haas : > On Wed, May 25, 2011 at 10:35 AM, Hitoshi Harada wrote: >> 2011/5/25 Hitoshi Harada : >>> So I'm still >>> thinking which of pulling up and parameterized scan is better. >> >> After more investigation I came up with third idea, pushing down >> RangeTblEntry to aggregate s

Re: [HACKERS] Pull up aggregate subquery

2011-05-25 Thread Robert Haas
On Wed, May 25, 2011 at 10:35 AM, Hitoshi Harada wrote: > 2011/5/25 Hitoshi Harada : >> So I'm still >> thinking which of pulling up and parameterized scan is better. > > After more investigation I came up with third idea, pushing down > RangeTblEntry to aggregate subquery. This sounds like a craz

Re: [HACKERS] Pull up aggregate subquery

2011-05-25 Thread Hitoshi Harada
2011/5/26 Tom Lane : > Hitoshi Harada writes: >> 2011/5/25 Hitoshi Harada : >>> So I'm still >>> thinking which of pulling up and parameterized scan is better. > >> After more investigation I came up with third idea, pushing down >> RangeTblEntry to aggregate subquery. This sounds like a crazy ide

Re: [HACKERS] Pull up aggregate subquery

2011-05-25 Thread Tom Lane
Hitoshi Harada writes: > 2011/5/25 Hitoshi Harada : >> So I'm still >> thinking which of pulling up and parameterized scan is better. > After more investigation I came up with third idea, pushing down > RangeTblEntry to aggregate subquery. This sounds like a crazy idea, Yes, it sure does. Won't

Re: [HACKERS] Pull up aggregate subquery

2011-05-25 Thread Hitoshi Harada
2011/5/25 Hitoshi Harada : > So I'm still > thinking which of pulling up and parameterized scan is better. After more investigation I came up with third idea, pushing down RangeTblEntry to aggregate subquery. This sounds like a crazy idea, but it seems to me like it is slightly easier than pulling

Re: [HACKERS] Pull up aggregate subquery

2011-05-24 Thread Robert Haas
On Tue, May 24, 2011 at 12:34 PM, Tom Lane wrote: >> Oh, I see.  I have a general gripe with nested loop plans: we already >> consider too many of them.  IIRC, when I last fooled around with this, >> the number of nested loop paths that we generate far exceeded the >> number of merge or hash join

Re: [HACKERS] Pull up aggregate subquery

2011-05-24 Thread Hitoshi Harada
2011/5/25 Tom Lane : > Robert Haas writes: >> That was my first thought, too, but then I wondered if I was getting >> cheap. > > Yeah, it's certainly possible that we're worrying too much.  Usually > I only get concerned about added planner logic if it will impact the > planning time for simple qu

Re: [HACKERS] Pull up aggregate subquery

2011-05-24 Thread Tom Lane
Robert Haas writes: > On Tue, May 24, 2011 at 11:11 AM, Tom Lane wrote: >> The point I was trying to make is that the ultimate reason for having a >> parameterized portion-of-a-plan will be that there's a parameterized >> indexscan somewhere down at the bottom. > Oh, I see. I have a general gri

Re: [HACKERS] Pull up aggregate subquery

2011-05-24 Thread Robert Haas
On Tue, May 24, 2011 at 11:11 AM, Tom Lane wrote: >> I must be misunderstanding you, because index scans are the thing we >> already *do* parameterize; and what else would make any sense? > > The point I was trying to make is that the ultimate reason for having a > parameterized portion-of-a-plan

Re: [HACKERS] Pull up aggregate subquery

2011-05-24 Thread Tom Lane
Robert Haas writes: > On Mon, May 23, 2011 at 4:02 PM, Tom Lane wrote: >> Yeah. For simple scan/join queries it seems likely that we only care >> about parameterizing indexscans, since the main opportunity for a win is >> to not scan all of a large table. Restricting things that way would >> he

Re: [HACKERS] Pull up aggregate subquery

2011-05-23 Thread Robert Haas
On Mon, May 23, 2011 at 10:47 PM, Hitoshi Harada wrote: > That's true. But if the planning cost is an only issue, why not adding > new GUC for user to choose if they prefer it or not? Of course if we > have some method to predict which way to go before proving both ways, > it's great. Do you have

Re: [HACKERS] Pull up aggregate subquery

2011-05-23 Thread Robert Haas
On Mon, May 23, 2011 at 4:02 PM, Tom Lane wrote: >> ...we'd need to plan the subquery twice, once with a parameterized >> qual m_id = $1 pushed down, and once without that.  We could then >> compare the cost of a nest-loop with the qual to the cost of a merge >> or hash join without it.  But this

Re: [HACKERS] Pull up aggregate subquery

2011-05-23 Thread Hitoshi Harada
2011/5/24 Tom Lane : > Robert Haas writes: > Yeah.  I fixed the executor in 9.1, but got hung up on how to fix the > planner.  So the planner still only knows how to do this for the case of > an inner indexscan, ie, it can't handle parameterizing any piece of a > plan larger than a single indexsca

Re: [HACKERS] Pull up aggregate subquery

2011-05-23 Thread Hitoshi Harada
2011/5/24 Robert Haas : > On Sat, May 21, 2011 at 12:49 PM, Hitoshi Harada wrote: >> 2011/5/5 Hitoshi Harada : >> Do I understand correctly? If so, could someone explain more detail of >> how to get Parameterized Scan in the planner? > > I think we're going to need Tom to give the definitive word

Re: [HACKERS] Pull up aggregate subquery

2011-05-23 Thread Tom Lane
Robert Haas writes: > On Sat, May 21, 2011 at 12:49 PM, Hitoshi Harada wrote: >> Do I understand correctly? If so, could someone explain more detail of >> how to get Parameterized Scan in the planner? > I think we're going to need Tom to give the definitive word on this, > but I believe that the

Re: [HACKERS] Pull up aggregate subquery

2011-05-23 Thread Robert Haas
On Sat, May 21, 2011 at 12:49 PM, Hitoshi Harada wrote: > 2011/5/5 Hitoshi Harada : >> https://commitfest.postgresql.org/action/patch_view?id=548 >> >> I'll work further if I find time. > > After more thought, pulling up aggregate subquery in narrow > conditional cases is quite hard path, especial

Re: [HACKERS] Pull up aggregate subquery

2011-05-21 Thread Hitoshi Harada
2011/5/5 Hitoshi Harada : > https://commitfest.postgresql.org/action/patch_view?id=548 > > I'll work further if I find time. After more thought, pulling up aggregate subquery in narrow conditional cases is quite hard path, especially when the joinrel is more than 2. It will be hard to check pullin

Re: [HACKERS] Pull up aggregate subquery

2011-05-05 Thread Hitoshi Harada
2011/5/5 Tom Lane : > Hitoshi Harada writes: >> I sometimes wonder if we could pull up aggregate query in the optimizer. > > I don't have time to look at this right now, but please add to the > upcoming commitfest. Done. https://commitfest.postgresql.org/action/patch_view?id=548 I'll work furthe

Re: [HACKERS] Pull up aggregate subquery

2011-05-04 Thread Tom Lane
Hitoshi Harada writes: > I sometimes wonder if we could pull up aggregate query in the optimizer. I don't have time to look at this right now, but please add to the upcoming commitfest. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

[HACKERS] Pull up aggregate subquery

2011-05-04 Thread Hitoshi Harada
I sometimes wonder if we could pull up aggregate query in the optimizer. My typical problem is: Consider two relations, medium size M and large size L. L has reference column to M's primary key. Say, create table size_m as select i as id, repeat(i::text, i % 100) as val from generate_series(