Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Kenneth Marshall
On Thu, Jul 16, 2009 at 06:49:08PM +0200, Andres Freund wrote: > On Thursday 16 July 2009 17:59:58 Tom Lane wrote: > > Andres Freund writes: > > > The default settings currently make it relatively hard to trigger geqo at > > > all. > > > > Yes, and that was intentional. One of the implications of

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Andres Freund
On Thursday 16 July 2009 19:22:30 Robert Haas wrote: > On Thu, Jul 16, 2009 at 11:32 AM, Tom Lane wrote: > > I wrote: > >> If I set both collapse_limit variables to very high values (I used 999), > >> it takes ... um ... not sure; I gave up waiting after half an hour. > >> I also tried with geqo_ef

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Tom Lane
Robert Haas writes: > On Thu, Jul 16, 2009 at 11:32 AM, Tom Lane wrote: >> So maybe a redesign of the equivalence-class joinclause mechanism is in >> order.  Still, this is unlikely to fix the fundamental issue that the >> time for large join problems grows nonlinearly. > Nonlinear is one thing,

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Andres Freund
On Thursday 16 July 2009 19:13:55 Robert Haas wrote: > On Thu, Jul 16, 2009 at 12:49 PM, Andres Freund wrote: > > On Thursday 16 July 2009 17:59:58 Tom Lane wrote: > >> Andres Freund writes: > >> > The default settings currently make it relatively hard to trigger geqo > >> > at all. > >> > >> Yes,

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Robert Haas
On Thu, Jul 16, 2009 at 11:32 AM, Tom Lane wrote: > I wrote: >> If I set both collapse_limit variables to very high values (I used 999), >> it takes ... um ... not sure; I gave up waiting after half an hour. >> I also tried with geqo_effort reduced to the minimum of 1, but that >> didn't produce a

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Robert Haas
On Thu, Jul 16, 2009 at 12:49 PM, Andres Freund wrote: > On Thursday 16 July 2009 17:59:58 Tom Lane wrote: >> Andres Freund writes: >> > The default settings currently make it relatively hard to trigger geqo at >> > all. >> >> Yes, and that was intentional.  One of the implications of what we're >

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Andres Freund
On Thursday 16 July 2009 17:59:58 Tom Lane wrote: > Andres Freund writes: > > The default settings currently make it relatively hard to trigger geqo at > > all. > > Yes, and that was intentional. One of the implications of what we're > discussing here is that geqo would get used a lot more for "t

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Andres Freund
On Thursday 16 July 2009 18:23:06 Tom Lane wrote: > Andres Freund writes: > > On Thursday 16 July 2009 17:16:31 Tom Lane wrote: > >> I tried the example query and couldn't get "Failed to make a valid plan" > >> out of it ... what settings do you need for that? > > > > It unfortunately depends on s

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Tom Lane
Andres Freund writes: > On Thursday 16 July 2009 17:16:31 Tom Lane wrote: >> I tried the example query and couldn't get "Failed to make a valid plan" >> out of it ... what settings do you need for that? > It unfortunately depends on settings and luck. This dependence on luck was > the > reason

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Tom Lane
Andres Freund writes: > The default settings currently make it relatively hard to trigger geqo at all. Yes, and that was intentional. One of the implications of what we're discussing here is that geqo would get used a lot more for "typical complex queries" (if there is any such thing as a typica

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Andres Freund
On Thursday 16 July 2009 17:27:39 Greg Stark wrote: > On Thu, Jul 16, 2009 at 4:16 PM, Tom Lane wrote: > > However, I do observe that this seems a sufficient counterexample > > against the theory that we can just remove the collapse limits and let > > GEQO save us on very complex queries. On my ma

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Tom Lane
Greg Stark writes: > On Thu, Jul 16, 2009 at 4:32 PM, Tom Lane wrote: >> So maybe a redesign of the equivalence-class joinclause mechanism is in >> order.  Still, this is unlikely to fix the fundamental issue that the >> time for large join problems grows nonlinearly. > Perhaps it's GEQO's fault

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Andres Freund
On Thursday 16 July 2009 17:16:31 Tom Lane wrote: > Andres Freund writes: > > On Thursday 16 July 2009 15:13:02 Tom Lane wrote: > >> Andres Freund writes: > >>> "Error: Failed to make a valid plan" > >> > >> We're not going to be able to fix this unless you show us examples. > > > > In the other

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Tom Lane
I wrote: > If I set both collapse_limit variables to very high values (I used 999), > it takes ... um ... not sure; I gave up waiting after half an hour. > I also tried with geqo_effort reduced to the minimum of 1, but that > didn't produce a plan in reasonable time either (I gave up after ten > mi

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Tom Lane
Andres Freund writes: > On Thursday 16 July 2009 15:13:02 Tom Lane wrote: >> Andres Freund writes: >>> "Error: Failed to make a valid plan" >> We're not going to be able to fix this unless you show us examples. > In the other thread I attached a similar to the real schema + example query. > No

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Andres Freund
On Thursday 16 July 2009 15:18:01 Andres Freund wrote: > On Thursday 16 July 2009 15:13:02 Tom Lane wrote: > > Andres Freund writes: > > > The queries on the second reporting schema unfortunately are different. > > > Its the one were I copied the crazy example I attached in the original > > > thre

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Andres Freund
On Thursday 16 July 2009 15:13:02 Tom Lane wrote: > Andres Freund writes: > > The queries on the second reporting schema unfortunately are different. > > Its the one were I copied the crazy example I attached in the original > > thread. With geqo=off a good part of the queries used daily use too m

Re: [HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Tom Lane
Andres Freund writes: > The queries on the second reporting schema unfortunately are different. Its > the > one were I copied the crazy example I attached in the original thread. > With geqo=off a good part of the queries used daily use too much memory to > plan > sensibly and geqo=on outright

[HACKERS] Review remove {join,from}_collapse_limit, add enable_join_ordering

2009-07-16 Thread Andres Freund
Hi Robert, Hi all, The patch applies cleanly and works as intended - no surprise here. After the changes the documentation is at least as easy to understand as before and the code changes look sensible Also not surprisingly that's not the area I expected problems I guess ;-) For performance tes