Re: [PERFORM] Re: Adding foreign key constraint holds exclusive lock for too long (on production database)

2013-10-29 Thread Tom Lane
David Johnston writes: > As noted in the referenced thread (and never contradicted) the current > algorithm is "for each record does the value in the FK column exist in the > PK table?" not "do all of the values currently found on the FK table exist > in the PK table?". Well, apparently nobody wh

[PERFORM] Re: Adding foreign key constraint holds exclusive lock for too long (on production database)

2013-10-29 Thread David Johnston
Ben Hoyt wrote > * http://www.postgresql.org/message-id/ > 51A11C97.90209@ > -- > indicates that the db ignores the index when add constraints As noted in the referenced thread (and never contradicted) the current algorithm is "for each record does the value in the FK column exist in the PK tab

[PERFORM] Adding foreign key constraint holds exclusive lock for too long (on production database)

2013-10-29 Thread Ben Hoyt
Hi folks, We're adding a foreign key constraint to a 20-million row table on our production database, and it's taking about 7 minutes. Because it's an ALTER TABLE, Postgres acquires an ACCESS EXCLUSIVE lock that prevents any reads/writes (though this particular table is very write-heavy, so even a

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Tom Lane
Jim Nasby writes: > Is there a way to measure memory consumption during planning, short of > something like strace? (I've got no dev tools available on our servers.) Nothing built-in, I'm pretty sure. You could probably add some instrumentation, but that would require running modified executabl

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Jim Nasby
On 10/29/13 1:20 PM, Tom Lane wrote: Jim Nasby writes: On 10/29/13 11:45 AM, Tom Lane wrote: Jim Nasby writes: I'm also wondering if it's time to raise those limits. Yeah, possibly. The current default values were set on machines much smaller/slower than most current hardware. I think a

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Tom Lane
Jim Nasby writes: > On 10/29/13 11:45 AM, Tom Lane wrote: >> Jim Nasby writes: >>> I'm also wondering if it's time to raise those limits. >> Yeah, possibly. The current default values were set on machines much >> smaller/slower than most current hardware. >> >> I think also that the collapse l

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Jim Nasby
On 10/29/13 11:45 AM, Tom Lane wrote: Jim Nasby writes: I'm also wondering if it's time to raise those limits. Yeah, possibly. The current default values were set on machines much smaller/slower than most current hardware. I think also that the collapse limits were invented mainly to keep p

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Tom Lane
Jim Nasby writes: > I'm also wondering if it's time to raise those limits. Yeah, possibly. The current default values were set on machines much smaller/slower than most current hardware. I think also that the collapse limits were invented mainly to keep people out of GEQO's clutches, but we've

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Jim Nasby
On 10/29/13 9:10 AM, Merlin Moncure wrote: On Mon, Oct 28, 2013 at 6:13 PM, Tom Lane wrote: Jim Nasby writes: I've been working on trying to normalize a table that's got a bunch of text fields. Normalizing the first 4 has been a non-issue. But when I try and normalize 2 additional fields a

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Merlin Moncure
On Mon, Oct 28, 2013 at 6:13 PM, Tom Lane wrote: > Jim Nasby writes: >> I've been working on trying to normalize a table that's got a bunch of text >> fields. Normalizing the first 4 has been a non-issue. But when I try and >> normalize 2 additional fields a bunch of query plans go belly-up. >