Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread Jeff Janes
On Tue, Dec 4, 2012 at 10:03 AM, wrote: > > Though that doesn't account for the 70x difference between the speed of the > two queries in actuality given a pretty similar expected speed (does it?). It kind of does. The expected speed is predicated on the number of rows being 200 fold higher. If

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread Jeff Janes
On Tue, Dec 4, 2012 at 9:47 AM, wrote: > >>> But the row estimates are not precise at the top of the join/filter. >>> It thinks there will 2120 rows, but there are only 11. > >>Ah... I didn't spot that one... > > Yes, you are right there - this is probably a slightly atypical query of > this sort

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread Sergey Konoplev
On Tue, Dec 4, 2012 at 9:47 AM, wrote: > eagerly awaiting 6pm when I can bring the DB down and start tweaking. The > effective_work_mem setting is going from 6Gb->88Gb which I think will make > quite a difference. I also wonder if increasing (say x10) of default_statistics_target or just doing A

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread Kevin Grittner
postgre...@foo.me.uk wrote: > Ah okay, thanks. I knew I could set various things but not > effective_work_mem (I tried reloading the edited config file but > it didn't seem to pick it up) Check the server log, maybe there was a typo or capitalization error. To test on a single connection you sho

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread postgresql
Ah okay, thanks. I knew I could set various things but not effective_work_mem (I tried reloading the edited config file but it didn't seem to pick it up) From: Vitalii Tymchyshyn [mailto:tiv...@gmail.com] Sent: 04 December 2012 18:51 To: postgre...@foo.me.uk Cc: postgres performance list Su

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread postgresql
Ah, okay - my reasoning was there's a big fancy-pants raid array behind it that makes disk operations faster relative to CPU ones. I'll test it and see if it actually makes any difference. -Original Message- From: Claudio Freire [mailto:klaussfre...@gmail.com] Sent: 04 December 2012 18:3

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread Vitalii Tymchyshyn
Well, you don't need to put anything down. Most settings that change planner decisions can be tuned on per-quey basis by issuing set commands in given session. This should not affect other queries more than it is needed to run query in the way planner chooses. Best regards, Vitalii Tymchyshyn 20

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread Claudio Freire
On Tue, Dec 4, 2012 at 3:31 PM, Philip Scott wrote: > r_p_c 2-> 1 (s_p_c 1->0.5): Is this really necessary? (looks like a no-op, unless your CPU is slow) -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.o

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread Claudio Freire
On Tue, Dec 4, 2012 at 3:03 PM, wrote: > > Though that doesn't account for the 70x difference between the speed of the > two queries in actuality given a pretty similar expected speed (does it?). > It does go some way to explaining why a bad choice of plan was made. I still don't think it does.

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread postgresql
> But the row estimates are not precise at the top of the join/filter. > It thinks there will 2120 rows, but there are only 11. > So it seems like there is a negative correlation between the two tables which is not recognized. Yes, you are right there. I am only just beginning to understand how t

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread postgresql
>> But the row estimates are not precise at the top of the join/filter. >> It thinks there will 2120 rows, but there are only 11. >Ah... I didn't spot that one... Yes, you are right there - this is probably a slightly atypical query of this sort actually, 2012 is a pretty good guess. On Claudio

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread Claudio Freire
On Tue, Dec 4, 2012 at 2:22 PM, Jeff Janes wrote: > On Tue, Dec 4, 2012 at 7:27 AM, Claudio Freire wrote: >> On Tue, Dec 4, 2012 at 12:06 PM, wrote: >>> Slow version with bitmapscan enabled: http://explain.depesz.com/s/6I7 >>> Fast version with bitmapscan disabled: http://explain.depesz.com/s/4

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread Jeff Janes
On Tue, Dec 4, 2012 at 7:27 AM, Claudio Freire wrote: > On Tue, Dec 4, 2012 at 12:06 PM, wrote: >> Slow version with bitmapscan enabled: http://explain.depesz.com/s/6I7 >> Fast version with bitmapscan disabled: http://explain.depesz.com/s/4MWG > > If you check the "fast" plan, it has a higher co

Re: [PERFORM] Comparative tps question

2012-12-04 Thread John Lister
On 29/11/2012 17:33, Merlin Moncure wrote: one thing that immediately jumps out here is that your wal volume could be holding you up. so it's possible we may want to move wal to the ssd volume. if you can scrounge up a 9.2 pgbench, we can gather more evidence for that by running pgbench with t

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread Claudio Freire
On Tue, Dec 4, 2012 at 12:06 PM, wrote: > Slow version with bitmapscan enabled: http://explain.depesz.com/s/6I7 > Fast version with bitmapscan disabled: http://explain.depesz.com/s/4MWG If you check the "fast" plan, it has a higher cost compared against the "slow" plan. The difference between c

Re: [PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread postgresql
Bad form to reply to yourself I know but just check-reading that for the third time I noticed two mistakes - The box has 128Gb of ram, not 512Mb - There is an additional constraint on the position_effect table (though I don't think it matters for this discussion): CONSTRAINT cons_pe_trade FO

[PERFORM] Slow query: bitmap scan troubles

2012-12-04 Thread postgresql
Hi guys (and girls) I've been banging my head over this for a few days now so if any of you kind souls could take a minute to take a look at this I would be eternally grateful. I have a pretty straightforward query that is very slow by default, and about 70 times faster when I set enable_bitmapsc