Re: [BUGS] Massive memory use for star query

2011-04-16 Thread Mark Kirkwood
On 17/04/11 02:58, Tom Lane wrote: Greg Stark writes: The planner uses various heuristics to avoid combinatoric growth wherever it can but there's no way to completely avoid it. Yeah. The collapse_limit variables can be seen as another heuristic to deal with this type of problem: they artific

Re: [BUGS] Massive memory use for star query

2011-04-16 Thread Mark Kirkwood
On 15/04/11 16:35, Mark Kirkwood wrote: Here's a simplified example using synthetic data (see attached to generate if desired): For anyone else who might be want to play with this: Patch with correction to make the directory reassignment work correctly, plus an additional comment in the RE

Re: [BUGS] Massive memory use for star query

2011-04-16 Thread Tom Lane
Greg Stark writes: > On Sat, Apr 16, 2011 at 8:21 AM, Mark Kirkwood > wrote: >> I guess you have answered my first question - i.e yes this should eat >> massive amount of ram as written - however are you sure there is no memory >> leaking going on here? > The planner uses various heuristics to a

Re: [BUGS] Massive memory use for star query

2011-04-16 Thread Greg Stark
On Sat, Apr 16, 2011 at 8:21 AM, Mark Kirkwood wrote: > > I guess you have answered my first question - i.e yes this should eat > massive amount of ram as written - however are you sure there is no memory > leaking going on here? The planner doesn't try to free up memory while it's working, it ge

Re: [BUGS] Massive memory use for star query

2011-04-16 Thread Mark Kirkwood
On 16/04/11 04:43, Tom Lane wrote: Mark Kirkwood writes: I've recently seen examples of star-like queries using vast amounts of memory in one of our production systems. Here's a simplified example using synthetic data (see attached to generate if desired): SET geqo_threshold = 14; SET from_coll

Re: [BUGS] Massive memory use for star query

2011-04-16 Thread Mark Kirkwood
On 16/04/11 01:59, Kevin Grittner wrote: Mark Kirkwood wrote: Here's a simplified example using synthetic data (see attached to generate if desired): Doesn't work for me: kgrittn@kgrittn-desktop:~/work/starjoin$ ./gendata.pl generate cat cannot open cat.dat: No such file or directory at ./g

Re: [BUGS] Massive memory use for star query

2011-04-15 Thread Tom Lane
Mark Kirkwood writes: > I've recently seen examples of star-like queries using vast amounts of > memory in one of our production systems. Here's a simplified example > using synthetic data (see attached to generate if desired): > SET geqo_threshold = 14; > SET from_collapse_limit = 14; > SET jo

Re: [BUGS] Massive memory use for star query

2011-04-15 Thread Kevin Grittner
"Kevin Grittner" wrote: > Mark Kirkwood wrote: > >> Here's a simplified example using synthetic data (see attached to >> generate if desired): > > Doesn't work for me: Edited scripts to change hard-coded directory. Issue confirmed. > If you run pmap -d on the pid, what does the last line

Re: [BUGS] Massive memory use for star query

2011-04-15 Thread Kevin Grittner
Mark Kirkwood wrote: > Here's a simplified example using synthetic data (see attached to > generate if desired): Doesn't work for me: kgrittn@kgrittn-desktop:~/work/starjoin$ ./gendata.pl generate cat cannot open cat.dat: No such file or directory at ./gendata.pl line 17. > Here's what a p

[BUGS] Massive memory use for star query

2011-04-14 Thread Mark Kirkwood
I've recently seen examples of star-like queries using vast amounts of memory in one of our production systems. Here's a simplified example using synthetic data (see attached to generate if desired): SET geqo_threshold = 14; SET from_collapse_limit = 14; SET join_collapse_limit = 14; EXPLAIN S