Re: [PERFORM] Problems with hash join over nested loop

2013-10-30 Thread Tom Lane
Jim Nasby writes: > Oh, and in my 28 table case ru_minflt was 428 4k memory pages (1.7MB). Not a > great measurement, but better than nothing. I didn't detect anything > noticeable on vmstat either, so I don't think the consumption is huge (an > email in the older thread mentioned 1GB... I'm no

Re: [PERFORM] Problems with hash join over nested loop

2013-10-30 Thread Jim Nasby
On 10/30/13 5:27 PM, Jim Nasby wrote: On 10/29/13 3:36 PM, Tom Lane wrote: Jim Nasby writes: Is there a way to measure memory consumption during planning, short of something like strace? (I've got no dev tools available on our servers.) Nothing built-in, I'm pretty sure. You could probably

Re: [PERFORM] Problems with hash join over nested loop

2013-10-30 Thread Jim Nasby
On 10/29/13 3:36 PM, Tom Lane wrote: Jim Nasby writes: Is there a way to measure memory consumption during planning, short of something like strace? (I've got no dev tools available on our servers.) Nothing built-in, I'm pretty sure. You could probably add some instrumentation, but that wou

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Tom Lane
Jim Nasby writes: > Is there a way to measure memory consumption during planning, short of > something like strace? (I've got no dev tools available on our servers.) Nothing built-in, I'm pretty sure. You could probably add some instrumentation, but that would require running modified executabl

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Jim Nasby
On 10/29/13 1:20 PM, Tom Lane wrote: Jim Nasby writes: On 10/29/13 11:45 AM, Tom Lane wrote: Jim Nasby writes: I'm also wondering if it's time to raise those limits. Yeah, possibly. The current default values were set on machines much smaller/slower than most current hardware. I think a

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Tom Lane
Jim Nasby writes: > On 10/29/13 11:45 AM, Tom Lane wrote: >> Jim Nasby writes: >>> I'm also wondering if it's time to raise those limits. >> Yeah, possibly. The current default values were set on machines much >> smaller/slower than most current hardware. >> >> I think also that the collapse l

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Jim Nasby
On 10/29/13 11:45 AM, Tom Lane wrote: Jim Nasby writes: I'm also wondering if it's time to raise those limits. Yeah, possibly. The current default values were set on machines much smaller/slower than most current hardware. I think also that the collapse limits were invented mainly to keep p

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Tom Lane
Jim Nasby writes: > I'm also wondering if it's time to raise those limits. Yeah, possibly. The current default values were set on machines much smaller/slower than most current hardware. I think also that the collapse limits were invented mainly to keep people out of GEQO's clutches, but we've

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Jim Nasby
On 10/29/13 9:10 AM, Merlin Moncure wrote: On Mon, Oct 28, 2013 at 6:13 PM, Tom Lane wrote: Jim Nasby writes: I've been working on trying to normalize a table that's got a bunch of text fields. Normalizing the first 4 has been a non-issue. But when I try and normalize 2 additional fields a

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Merlin Moncure
On Mon, Oct 28, 2013 at 6:13 PM, Tom Lane wrote: > Jim Nasby writes: >> I've been working on trying to normalize a table that's got a bunch of text >> fields. Normalizing the first 4 has been a non-issue. But when I try and >> normalize 2 additional fields a bunch of query plans go belly-up. >

Re: [PERFORM] Problems with hash join over nested loop

2013-10-28 Thread Jim Nasby
On 10/28/13 6:13 PM, Tom Lane wrote: Jim Nasby writes: I've been working on trying to normalize a table that's got a bunch of text fields. Normalizing the first 4 has been a non-issue. But when I try and normalize 2 additional fields a bunch of query plans go belly-up. Try increasing join_c

Re: [PERFORM] Problems with hash join over nested loop

2013-10-28 Thread Tom Lane
Jim Nasby writes: > I've been working on trying to normalize a table that's got a bunch of text > fields. Normalizing the first 4 has been a non-issue. But when I try and > normalize 2 additional fields a bunch of query plans go belly-up. Try increasing join_collapse_limit/from_collapse_limit.

[PERFORM] Problems with hash join over nested loop

2013-10-28 Thread Jim Nasby
I've been working on trying to normalize a table that's got a bunch of text fields. Normalizing the first 4 has been a non-issue. But when I try and normalize 2 additional fields a bunch of query plans go belly-up. I've seen this on our old 8.4 databases as well as 9.1. I haven't been able to