Re: [PERFORM] merge join killing performance

2010-05-27 Thread Tom Lane
Scott Marlowe writes: > On Thu, May 27, 2010 at 7:16 PM, Tom Lane wrote: >> I've committed a patch for this, if you're interested in testing that >> it fixes your situation. > Cool, do we have a snapshot build somewhere or do I need to get all > the extra build bits like flex or yacc or bison or

Re: [PERFORM] merge join killing performance

2010-05-27 Thread Scott Marlowe
On Thu, May 27, 2010 at 7:16 PM, Tom Lane wrote: > Scott Marlowe writes: >> So, Tom, so you think it's possible that the planner isn't noticing >> all those nulls and thinks it'll just take a row or two to get to the >> value it needs to join on? > > I've committed a patch for this, if you're int

Re: [PERFORM] merge join killing performance

2010-05-27 Thread Tom Lane
Scott Marlowe writes: > So, Tom, so you think it's possible that the planner isn't noticing > all those nulls and thinks it'll just take a row or two to get to the > value it needs to join on? I've committed a patch for this, if you're interested in testing that it fixes your situation.

Re: [PERFORM] merge join killing performance

2010-05-20 Thread Scott Marlowe
On Thu, May 20, 2010 at 8:28 AM, Tom Lane wrote: > Scott Marlowe writes: >> So, Tom, so you think it's possible that the planner isn't noticing >> all those nulls and thinks it'll just take a row or two to get to the >> value it needs to join on? > > Could be.  I don't have time right now to chas

Re: [PERFORM] merge join killing performance

2010-05-20 Thread Tom Lane
Scott Marlowe writes: > So, Tom, so you think it's possible that the planner isn't noticing > all those nulls and thinks it'll just take a row or two to get to the > value it needs to join on? Could be. I don't have time right now to chase through the code, but that sounds like a plausible theor

Re: [PERFORM] merge join killing performance

2010-05-19 Thread Scott Marlowe
On Wed, May 19, 2010 at 8:06 PM, Scott Marlowe wrote: > On Wed, May 19, 2010 at 8:04 PM, Scott Marlowe > wrote: >> On Wed, May 19, 2010 at 7:46 PM, Matthew Wakeling >> wrote: >>> On Wed, 19 May 2010, Scott Marlowe wrote: > > It's apparently estimating (wrongly) that the merge join won'

Re: [PERFORM] merge join killing performance

2010-05-19 Thread Scott Marlowe
On Wed, May 19, 2010 at 8:04 PM, Scott Marlowe wrote: > On Wed, May 19, 2010 at 7:46 PM, Matthew Wakeling wrote: >> On Wed, 19 May 2010, Scott Marlowe wrote: It's apparently estimating (wrongly) that the merge join won't have to scan very much of "files" before it can stop because

Re: [PERFORM] merge join killing performance

2010-05-19 Thread Scott Marlowe
On Wed, May 19, 2010 at 7:46 PM, Matthew Wakeling wrote: > On Wed, 19 May 2010, Scott Marlowe wrote: >>> >>> It's apparently estimating (wrongly) that the merge join won't have to >>> scan very much of "files" before it can stop because it finds an eid >>> value larger than any eid in the other ta

Re: [PERFORM] merge join killing performance

2010-05-19 Thread Matthew Wakeling
On Wed, 19 May 2010, Scott Marlowe wrote: It's apparently estimating (wrongly) that the merge join won't have to scan very much of "files" before it can stop because it finds an eid value larger than any eid in the other table.  So the issue here is an inexact stats value for the max eid. I wan

Re: [PERFORM] merge join killing performance

2010-05-19 Thread Scott Marlowe
On Wed, May 19, 2010 at 2:27 PM, Scott Marlowe wrote: > On Wed, May 19, 2010 at 10:53 AM, Tom Lane wrote: >> Matthew Wakeling writes: >>> On Tue, 18 May 2010, Scott Marlowe wrote: Aggregate  (cost=902.41..902.42 rows=1 width=4)     ->  Merge Join  (cost=869.97..902.40 rows=1 width=4) >

Re: [PERFORM] merge join killing performance

2010-05-19 Thread Scott Marlowe
On Wed, May 19, 2010 at 10:53 AM, Tom Lane wrote: > Matthew Wakeling writes: >> On Tue, 18 May 2010, Scott Marlowe wrote: >>> Aggregate  (cost=902.41..902.42 rows=1 width=4) >>>     ->  Merge Join  (cost=869.97..902.40 rows=1 width=4) >>>         Merge Cond: (f.eid = ev.eid) >>>         ->  Index

Re: [PERFORM] merge join killing performance

2010-05-19 Thread Scott Marlowe
On Wed, May 19, 2010 at 10:53 AM, Tom Lane wrote: > Matthew Wakeling writes: >> On Tue, 18 May 2010, Scott Marlowe wrote: >>> Aggregate  (cost=902.41..902.42 rows=1 width=4) >>>     ->  Merge Join  (cost=869.97..902.40 rows=1 width=4) >>>         Merge Cond: (f.eid = ev.eid) >>>         ->  Index

Re: [PERFORM] merge join killing performance

2010-05-19 Thread Tom Lane
Matthew Wakeling writes: > On Tue, 18 May 2010, Scott Marlowe wrote: >> Aggregate (cost=902.41..902.42 rows=1 width=4) >> -> Merge Join (cost=869.97..902.40 rows=1 width=4) >> Merge Cond: (f.eid = ev.eid) >> -> Index Scan using files_eid_idx on files f >> (cost=0.00

Re: [PERFORM] merge join killing performance

2010-05-18 Thread Scott Marlowe
On Tue, May 18, 2010 at 9:00 PM, Matthew Wakeling wrote: > On Tue, 18 May 2010, Scott Marlowe wrote: >> >> Aggregate  (cost=902.41..902.42 rows=1 width=4) >>  ->  Merge Join  (cost=869.97..902.40 rows=1 width=4) >>        Merge Cond: (f.eid = ev.eid) >>        ->  Index Scan using files_eid_idx on

Re: [PERFORM] merge join killing performance

2010-05-18 Thread Matthew Wakeling
On Tue, 18 May 2010, Scott Marlowe wrote: Aggregate (cost=902.41..902.42 rows=1 width=4) -> Merge Join (cost=869.97..902.40 rows=1 width=4) Merge Cond: (f.eid = ev.eid) -> Index Scan using files_eid_idx on files f (cost=0.00..157830.39 rows=3769434 width=8) Okay, that's we

[PERFORM] merge join killing performance

2010-05-18 Thread Scott Marlowe
Machine: 8 core AMD opteron 2.1GHz, 12 disk RAID-10, 2 disk pg_xlog, RHEL 5.4 pg version 8.3.9 (upgrading soon to 8.3.11 or so) This query: SELECT sum(f.bytes) AS sum FROM files f INNER JOIN events ev ON f.eid = ev.eid WHERE ev.orgid = 969677; is choosing a merge join, which never returns from ex