Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-19 Thread Steve Singer
On 13-04-14 08:06 PM, Steve Singer wrote: On 13-04-13 04:54 PM, Jeff Janes wrote: If you are trying to make your own private copy of 9.2, then removing the fudge factor altogether is probably the way to go. But if you want to help improve future versions, you probably need to test with the m

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-14 Thread Steve Singer
On 13-04-13 04:54 PM, Jeff Janes wrote: On Sat, Apr 13, 2013 at 9:14 AM, Steve Singer mailto:ssin...@ca.afilias.info>> wrote: indexTotalCost += index->pages * spc_random_page_cost / 10.0; Is driving my high costs on the inner loop. The index has 2-5 million pages depending on t

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-13 Thread Jeff Janes
On Sat, Apr 13, 2013 at 9:14 AM, Steve Singer wrote: > >> >> > indexTotalCost += index->pages * spc_random_page_cost / 10.0; > > Is driving my high costs on the inner loop. The index has 2-5 million > pages depending on the partition . If I run this against 9.2.2 with / > 1.0 the estimat

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-13 Thread Steve Singer
On 13-04-12 09:20 PM, Jeff Janes wrote: On Thursday, April 11, 2013, Steve Singer wrote: I think the reason why it is picking the hash join based plans is because of Index Scan using table_b_1_ptid_orgid_ym_unq on table_b_1 b (cost=0.00..503.86 rows=1 width=10) (actual time=0.0

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-12 Thread Jeff Janes
On Thursday, April 11, 2013, Steve Singer wrote: > > I think the reason why it is picking the hash join based plans is because > of > > Index Scan using table_b_1_ptid_orgid_ym_unq on table_b_1 b > (cost=0.00..503.86 rows=1 width=10) (actual time=0.016..0.017 rows=1 > loops=414249) >

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-11 Thread Steve Singer
On 13-04-10 07:54 PM, Steve Singer wrote: On 13-04-10 02:06 PM, Jeff Janes wrote: On Wed, Apr 10, 2013 at 6:49 AM, Steve Singer mailto:ssin...@ca.afilias.info>> wrote: I think the index recheck means your bitmap is overflowing (i.e. needing more space than work_mem) and so keeping only the p

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread Steve Singer
On 13-04-10 02:06 PM, Jeff Janes wrote: On Wed, Apr 10, 2013 at 6:49 AM, Steve Singer mailto:ssin...@ca.afilias.info>> wrote: I think the index recheck means your bitmap is overflowing (i.e. needing more space than work_mem) and so keeping only the pages which have at least one match, which m

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread Jeff Janes
On Wed, Apr 10, 2013 at 8:56 AM, Steve Singer wrote: > On 13-04-10 09:56 AM, k...@rice.edu wrote: > >> On Wed, Apr 10, 2013 at 09:49:55AM -0400, Steve Singer wrote: >> > > >> Hi Steve, >> >> The one thing that stands out to me is that you are working with 200GB of >> data on a machine with 4-8GB o

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread Jeff Janes
On Wed, Apr 10, 2013 at 6:49 AM, Steve Singer wrote: > I'm encountering an issue where PG 9.2.4 (we also see this with 9.2.3) is > picking a plan involving a bitmap heap scan that turns out to be much > slower than a nested-loop plan using indexes. > > The planner picks the hashjoin plan by defaul

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread k...@rice.edu
On Wed, Apr 10, 2013 at 11:56:32AM -0400, Steve Singer wrote: > On 13-04-10 09:56 AM, k...@rice.edu wrote: > >On Wed, Apr 10, 2013 at 09:49:55AM -0400, Steve Singer wrote: > > > > >Hi Steve, > > > >The one thing that stands out to me is that you are working with 200GB of > >data on a machine with

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread Steve Singer
On 13-04-10 09:56 AM, k...@rice.edu wrote: On Wed, Apr 10, 2013 at 09:49:55AM -0400, Steve Singer wrote: Hi Steve, The one thing that stands out to me is that you are working with 200GB of data on a machine with 4-8GB of ram and you have the random_page_cost set to 2.0. That is almost comple

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread k...@rice.edu
On Wed, Apr 10, 2013 at 09:49:55AM -0400, Steve Singer wrote: > I'm encountering an issue where PG 9.2.4 (we also see this with > 9.2.3) is picking a plan involving a bitmap heap scan that turns out > to be much slower than a nested-loop plan using indexes. > > The planner picks the hashjoin plan