Re: [PERFORM] same plan, add 1 condition, 1900x slower

2005-11-11 Thread Tom Lane
Mitchell Skinner <[EMAIL PROTECTED]> writes: > On Fri, 2005-11-11 at 10:53 -0500, Tom Lane wrote: >> what you need to do is incorporate the "source" value into the >> external_id_map index key somehow. Then the index scan would be able to >> realize that there is no possibility of finding another

Re: [PERFORM] same plan, add 1 condition, 1900x slower

2005-11-11 Thread Mitchell Skinner
On Fri, 2005-11-11 at 10:53 -0500, Tom Lane wrote: > After re-reading your explanation of what you're doing with the data, > I thought of a possible explanation. Is the "source" value exactly > correlated with the external_id_map primary key? Sort of. In this case, at the beginning of external_i

Re: [PERFORM] same plan, add 1 condition, 1900x slower

2005-11-11 Thread Tom Lane
I wrote: > No, the visibility check happens first. The timing does seem consistent > with the idea that the comparison is being done at all 15M rows, but > your other EXPLAIN shows that only 2K rows are actually retrieved, which > presumably is because the merge doesn't need the rest. (Merge will

Re: [PERFORM] same plan, add 1 condition, 1900x slower

2005-11-11 Thread Tom Lane
Mitch Skinner <[EMAIL PROTECTED]> writes: > On Fri, 2005-11-11 at 09:17 -0500, Tom Lane wrote: >> Please don't, actually, until we understand what's going on. > Ack, I was the middle of the vacuum full already when I got this. Given what you said about no deletions or updates, the vacuum should h

Re: [PERFORM] same plan, add 1 condition, 1900x slower

2005-11-11 Thread Mitch Skinner
On Fri, 2005-11-11 at 09:17 -0500, Tom Lane wrote: > Richard Huxton writes: > > Does external_id_map really have 15 million rows? If not, try a VACUUM > > FULL on it. Be prepared to give it some time to complete. > > Please don't, actually, until we understand what's going on. Ack, I was the mi

Re: [PERFORM] same plan, add 1 condition, 1900x slower

2005-11-11 Thread Mitch Skinner
On Fri, 2005-11-11 at 11:51 +, Richard Huxton wrote: > Does external_id_map really have 15 million rows? If not, try a VACUUM > FULL on it. Be prepared to give it some time to complete. Thanks for the reply. It does indeed have that many rows: statgen=> select count(*) from util.external_id_

Re: [PERFORM] same plan, add 1 condition, 1900x slower

2005-11-11 Thread Tom Lane
Richard Huxton writes: > Mitch Skinner wrote: >> The version with the condition is definitely doing more I/O. The >> version without the condition doesn't read at all. > Does external_id_map really have 15 million rows? If not, try a VACUUM > FULL on it. Be prepared to give it some time to co

Re: [PERFORM] same plan, add 1 condition, 1900x slower

2005-11-11 Thread Tom Lane
Mitch Skinner <[EMAIL PROTECTED]> writes: > On Thu, 2005-11-10 at 12:23 -0500, Tom Lane wrote: >> Apparently, you are using a platform and/or locale in which strcoll() is >> spectacularly, god-awfully slow --- on the order of 10 msec per comparison. > The version with the condition is definitely d

Re: [PERFORM] same plan, add 1 condition, 1900x slower

2005-11-11 Thread Richard Huxton
Mitch Skinner wrote: The version with the condition is definitely doing more I/O. The version without the condition doesn't read at all. [snip] relname | relpages | reltuples -+--+- external_id_map | 126883

Re: [PERFORM] same plan, add 1 condition, 1900x slower

2005-11-11 Thread Mitch Skinner
On Thu, 2005-11-10 at 12:23 -0500, Tom Lane wrote: > Apparently, you are using a platform and/or locale in which strcoll() is > spectacularly, god-awfully slow --- on the order of 10 msec per comparison. The version with the condition is definitely doing more I/O. The version without the conditio

Re: [PERFORM] same plan, add 1 condition, 1900x slower

2005-11-10 Thread Tom Lane
Mitch Skinner <[EMAIL PROTECTED]> writes: > This is with Postgres 8.0.3. Any advice is appreciated. These are exactly the same plan, except for the addition of the extra filter condition ... >-> Index Scan using external_id_map_primary_key on external_id_map > eim (cost=0.00..2345747.01 ro