Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-08-17 Thread Jim Vanns
Hello again. So sorry for resurrecting such an old thread but the problem still persists - I've just had very little to report on, until now... > > That latter test - won't that pretty much just read from the page > cache? > > 'sync' may well have forced dirty pages to disk but does it actually >

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-24 Thread Jim Vanns
> > That latter test - won't that pretty much just read from the page > cache? > > 'sync' may well have forced dirty pages to disk but does it actually > > evict them to? > > Basically, the cache is avoided because of the size of the file. > 600 blocks at 8k exceeds the size of RAM in the mach

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-24 Thread Martin French
Hi Jim,> > I've already tried something very similar using dd. No performance> penalties during a normal running of the system - or when this blocking> happens either actually. But I agree, it does indeed sound like some> sort of I/O problem. I just don't know what! I do have a few more tricks> up

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-24 Thread Jim Vanns
> This looks like this is a pure IO issue. You mentioned that this was a > software RAID system. I wonder if there's some complication there. > > Have you tried setting the disk queues to deadline? > > echo "deadline" > /sys/block/{DEVICE-NAME}/queue/scheduler > That might help. But to be honest,

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-24 Thread Jim Vanns
On Tue, 2012-07-24 at 08:30 +0800, Craig Ringer wrote: > On 07/23/2012 10:46 PM, Jim Vanns wrote: > > Nothing obvious - and we log a fair amount. No tmp table creations, > > no locks held. > > > > To add to this EXPLAIN reports it took only 0.23ms to run (for example) > > whereas the wall clock tim

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Martin French
> > Hi> > > > > > > > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > > > statements hang indefinitely until *something* (we don't know what)> > > releases some kind of resource or no longer becomes a massive bottle> > > neck. These are the symptoms.> > > > Is this in pgAdmin? Or psq

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Craig Ringer
On 07/23/2012 10:46 PM, Jim Vanns wrote: Nothing obvious - and we log a fair amount. No tmp table creations, no locks held. To add to this EXPLAIN reports it took only 0.23ms to run (for example) whereas the wall clock time is more like 20-30 minutes (or up to n hours as I said where everything

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Jim Vanns
On Mon, 2012-07-23 at 11:09 -0400, Tom Lane wrote: > Jim Vanns writes: > > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > > statements hang indefinitely until *something* (we don't know what) > > releases some kind of resource or no longer becomes a massive bottle > > neck. These

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Tom Lane
Jim Vanns writes: > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > statements hang indefinitely until *something* (we don't know what) > releases some kind of resource or no longer becomes a massive bottle > neck. These are the symptoms. Does anything show up as blocked in the pg

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Jim Vanns
On Mon, 2012-07-23 at 09:53 -0400, Andrew Dunstan wrote: > On 07/23/2012 04:41 AM, Jim Vanns wrote: > > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > > statements hang indefinitely until *something* (we don't know what) > > releases some kind of resource or no longer becomes a mass

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Jim Vanns
Thank you all for your replies, I shall try and qualify and confirm... On Mon, 2012-07-23 at 14:46 +0100, Martin French wrote: > Hi > > > > > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > > statements hang indefinitely until *something* (we don't know what) > > releases some ki

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Andrew Dunstan
On 07/23/2012 04:41 AM, Jim Vanns wrote: We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) statements hang indefinitely until *something* (we don't know what) releases some kind of resource or no longer becomes a massive bottle neck. These are the symptoms. I have seen this sort of

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Martin French
Hi> > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > statements hang indefinitely until *something* (we don't know what)> releases some kind of resource or no longer becomes a massive bottle> neck. These are the symptoms.Is this in pgAdmin? Or psql on the console?> However, the syst

[PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Jim Vanns
(First, apologies if this post now appears twice - it appears our mail server rewrites my address!) Hello all. I'm a pgsql performance virgin so hope I cross all the 't's and dot the lower-case 'j's when posting this query... On our production database server we're experiencing behaviour that se