Re: [PERFORM] Bad performance of SELECT ... where id IN (...)

2009-09-26 Thread Xia Qingran
On Sat, Sep 26, 2009 at 10:59 PM, Craig James wrote: > > If your user_id is always in a narrow range like this, or even in any range > that is a small fraction of the total, then add a range condition, like > this: > > select * from event where user_id <= 500 and user_id >= 0 and user_id in > (...

Re: [PERFORM] Bad performance of SELECT ... where id IN (...)

2009-09-26 Thread Xia Qingran
On Sun, Sep 27, 2009 at 1:03 AM, Tom Lane wrote: > Xia Qingran writes: >> I have a big performance problem in my SQL select query: >> select * from event where user_id in >> (500,499,498, ... ,1,0); >> The above SELECT always spends 1200ms. > > Your EXPLAIN ANALYZE shows that the actual runtime i

Re: [PERFORM] PG 8.3 and large shared buffer settings

2009-09-26 Thread Greg Smith
On Sat, 26 Sep 2009, Jeff Janes wrote: On Sat, Sep 26, 2009 at 8:19 AM, Greg Smith wrote: Another problem spot are checkpoints. If you dirty a very large buffer cache, that whole thing will have to get dumped to disk eventually, and on some workloads people have found they have to reduce sha

Re: [PERFORM] PG 8.3 and large shared buffer settings

2009-09-26 Thread Jeff Janes
On Sat, Sep 26, 2009 at 8:19 AM, Greg Smith wrote: > > Another problem spot are checkpoints. If you dirty a very large buffer > cache, that whole thing will have to get dumped to disk eventually, and on > some workloads people have found they have to reduce shared_buffers > specifically to keep t

Re: [PERFORM] PG 8.3 and large shared buffer settings

2009-09-26 Thread Jeff Janes
On Sat, Sep 26, 2009 at 9:57 AM, Gerhard Wiesinger wrote: > On Sat, 26 Sep 2009, Greg Smith wrote: > >> On Fri, 25 Sep 2009, Jeff Janes wrote: >> >>> Does it do this even if the block was already in shared_buffers? >> >> Usually not. The buffer ring algorithm is used to manage pages that are >> r

Re: [PERFORM] Bad performance of SELECT ... where id IN (...)

2009-09-26 Thread Paul Ooi
On 26-Sep-2009, at 10:16 PM, Claus Guttesen wrote: I have a big performance problem in my SQL select query: select * from event where user_id in (500,499,498,497,496,495,494,493,492,491,490,489,488,487,486,485,484,483,482,481,480,479,478,477,476,475,474

Re: [PERFORM] Bad performance of SELECT ... where id IN (...)

2009-09-26 Thread Craig James
Xia Qingran wrote: Hi, I have a big performance problem in my SQL select query: select * from event where user_id in (500,499,498,497,496,495,494,493,492,491,490,489,488,487,486,485,484,483,482,481,480,479,478,477,476,475,474,473,472,471,470,469,468,467,4

Re: [PERFORM] PG 8.3 and large shared buffer settings

2009-09-26 Thread Greg Smith
On Fri, 25 Sep 2009, Jeff Janes wrote: Does it do this even if the block was already in shared_buffers? Usually not. The buffer ring algorithm is used to manage pages that are read in specifically to satisfy a sequential scan (there's a slightly different ring method used for VACUUM too).

Re: [PERFORM] Bad performance of SELECT ... where id IN (...)

2009-09-26 Thread Grzegorz Jaśkiewicz
if you reuse that set a lot, how about storing it in a table , and doing the join on db side ? if it is large, it sometimes makes sense to create temp table just for single query (I use that sort of stuff for comparing with few M records). But temp tables in that case have to be short lived, as the

Re: [PERFORM] PG 8.3 and large shared buffer settings

2009-09-26 Thread Pierre Frédéric Caillau d
Is there any practical limit to the number of shared buffers PG 8.3.7 can handle before more becomes counter-productive? It is more efficient to have the page in shared buffers, rather than doing a context switch to the OS, copying the entire page from the OS's cache into shared buffers,

Re: [PERFORM] Bad performance of SELECT ... where id IN (...)

2009-09-26 Thread Tom Lane
Xia Qingran writes: > I have a big performance problem in my SQL select query: > select * from event where user_id in > (500,499,498, ... ,1,0); > The above SELECT always spends 1200ms. Your EXPLAIN ANALYZE shows that the actual runtime is only about 240ms. So either the planning time is about 10

Re: [PERFORM] PG 8.3 and large shared buffer settings

2009-09-26 Thread Gerhard Wiesinger
On Sat, 26 Sep 2009, Greg Smith wrote: On Fri, 25 Sep 2009, Jeff Janes wrote: Does it do this even if the block was already in shared_buffers? Usually not. The buffer ring algorithm is used to manage pages that are read in specifically to satisfy a sequential scan (there's a slightly diffe

Re: [PERFORM] PG 8.3 and large shared buffer settings

2009-09-26 Thread Greg Smith
On Thu, 24 Sep 2009, Dan Sugalski wrote: Is there any practical limit to the number of shared buffers PG 8.3.7 can handle before more becomes counter-productive? There are actually two distinct questions here you should consider, because the popular wisdom here and what makes sense for your c

[PERFORM] Bad performance of SELECT ... where id IN (...)

2009-09-26 Thread Xia Qingran
Hi, I have a big performance problem in my SQL select query: select * from event where user_id in (500,499,498,497,496,495,494,493,492,491,490,489,488,487,486,485,484,483,482,481,480,479,478,477,476,475,474,473,472,471,470,469,468,467,466,465,464,463,462,46

Re: [PERFORM] Bad performance of SELECT ... where id IN (...)

2009-09-26 Thread Claus Guttesen
> I have a big performance problem in my SQL select query: > > > select * from event where user_id in > (500,499,498,497,496,495,494,493,492,491,490,489,488,487,486,485,484,483,482,481,480,479,478,477,476,475,474,473,472,471,470,469,468,467,466,465,464,463,4