Re: [PERFORM] CPU bound

2011-01-03 Thread Mladen Gogala
Jim Nasby wrote: On Dec 20, 2010, at 12:47 AM, Mladen Gogala wrote: Good time accounting is the most compelling reason for having a wait event interface, like Oracle. Without the wait event interface, one cannot really tell where the time is spent, at least not without profiling the databas

Re: [PERFORM] CPU bound

2011-01-02 Thread Jim Nasby
On Dec 20, 2010, at 12:47 AM, Mladen Gogala wrote: > Good time accounting is the most compelling reason for having a wait event > interface, like Oracle. Without the wait event interface, one cannot really > tell where the time is spent, at least not without profiling the database > code, which

Re: [PERFORM] CPU bound

2010-12-21 Thread Mladen Gogala
On 12/20/2010 10:33 AM, James Cloos wrote: And how exactly, given that the kernel does not know whether the CPU is active or waiting on ram, could an application do so? -JimC That particular aspect will remain hidden, it's a domain of the hardware architecture. Nevertheless, there are things l

Re: [PERFORM] CPU bound

2010-12-20 Thread Jeremy Harris
On 2010-12-20 15:48, Kenneth Marshall wrote: And how exactly, given that the kernel does not know whether the CPU is active or waiting on ram, could an application do so? Exactly. I have only seen this data from hardware emulators. It would be nice to have... :) There's no reason that the c

Re: [PERFORM] CPU bound

2010-12-20 Thread Kenneth Marshall
On Mon, Dec 20, 2010 at 10:33:26AM -0500, James Cloos wrote: > > "MG" == Mladen Gogala writes: > > MG> Good time accounting is the most compelling reason for having a wait > MG> event interface, like Oracle. Without the wait event interface, one > MG> cannot really tell where the time is spen

Re: [PERFORM] CPU bound

2010-12-20 Thread James Cloos
> "MG" == Mladen Gogala writes: MG> Good time accounting is the most compelling reason for having a wait MG> event interface, like Oracle. Without the wait event interface, one MG> cannot really tell where the time is spent, at least not without MG> profiling the database code, which is not a

Re: [PERFORM] CPU bound

2010-12-19 Thread Mladen Gogala
On 12/19/2010 7:57 PM, James Cloos wrote: "RA" == Royce Ausburn writes: RA> I notice that when restoring a DB on a laptop with an SDD, RA> typically postgres is maxing out a CPU - even during a COPY. The time the CPUs spend waiting on system RAM shows up as CPU time, not as Wait time. It co

Re: [PERFORM] CPU bound

2010-12-19 Thread James Cloos
> "RA" == Royce Ausburn writes: RA> I notice that when restoring a DB on a laptop with an SDD, RA> typically postgres is maxing out a CPU - even during a COPY. The time the CPUs spend waiting on system RAM shows up as CPU time, not as Wait time. It could be just that the SSD is fast enough

Re: [PERFORM] CPU bound

2010-12-14 Thread Royce Ausburn
Thanks guys - interesting. On 14/12/2010, at 5:59 AM, Josh Berkus wrote: > On 12/12/10 6:43 PM, Royce Ausburn wrote: >> Hi all, >> >> I notice that when restoring a DB on a laptop with an SDD, typically >> postgres is maxing out a CPU - even during a COPY. I wonder, what is >> postgres usu

Re: [PERFORM] CPU bound

2010-12-13 Thread Josh Berkus
On 12/12/10 6:43 PM, Royce Ausburn wrote: > Hi all, > > I notice that when restoring a DB on a laptop with an SDD, typically postgres > is maxing out a CPU - even during a COPY. I wonder, what is postgres usually > doing with the CPU? I would have thought the disk would usually be the > bottl

Re: [PERFORM] CPU bound

2010-12-13 Thread Craig Ringer
On 12/13/2010 10:43 AM, Royce Ausburn wrote: Hi all, I notice that when restoring a DB on a laptop with an SDD, typically postgres is maxing out a CPU - even during a COPY. I wonder, what is postgres usually doing with the CPU? I would have thought the disk would usually be the bottleneck i

[PERFORM] CPU bound

2010-12-12 Thread Royce Ausburn
Hi all, I notice that when restoring a DB on a laptop with an SDD, typically postgres is maxing out a CPU - even during a COPY. I wonder, what is postgres usually doing with the CPU? I would have thought the disk would usually be the bottleneck in the DB, but occasionally it's not. We're emb

Re: [PERFORM] cpu bound postgresql setup.

2010-06-29 Thread Bruce Momjian
Benjamin Krajmalnik wrote: > > -Original Message- > > From: Bruce Momjian [mailto:br...@momjian.us] > > Sent: Monday, June 28, 2010 3:45 PM > > To: Benjamin Krajmalnik > > Cc: Rajesh Kumar Mallah; Kevin Grittner; pgsql- > > performa...@postgresql.org

Re: [PERFORM] cpu bound postgresql setup.

2010-06-29 Thread Benjamin Krajmalnik
ian.us] > Sent: Monday, June 28, 2010 3:45 PM > To: Benjamin Krajmalnik > Cc: Rajesh Kumar Mallah; Kevin Grittner; pgsql- > performa...@postgresql.org > Subject: Re: [PERFORM] cpu bound postgresql setup. > > Benjamin Krajmalnik wrote: > > Rajesh, > > > > I ha

Re: [PERFORM] cpu bound postgresql setup.

2010-06-28 Thread Bruce Momjian
Benjamin Krajmalnik wrote: > Rajesh, > > I had a similar situation a few weeks ago whereby performance all of a > sudden decreased. > The one tunable which resolved the problem in my case was increasing the > number of checkpoint segments. > After increasing them, everything went back to its norma

Re: [PERFORM] cpu bound postgresql setup.

2010-06-24 Thread Kevin Grittner
Rajesh Kumar Mallah wrote: > Kevin Grittner wrote: >>> max_connections = 300 >> >> As I've previously mentioned, I would use a connection pool, in >> which case this wouldn't need to be that high. > > We do use connection pooling provided to mod_perl server > via Apache::DBI::Cache. If i reduc

Re: [PERFORM] cpu bound postgresql setup.

2010-06-24 Thread Alvaro Herrera
Excerpts from Rajesh Kumar Mallah's message of jue jun 24 13:25:32 -0400 2010: > What prompted me to post to list is that the server transitioned from > being IO bound to CPU bound and 90% of syscalls being > lseek(XXX, 0, SEEK_END) = YYY It could be useful to find out what file is being seek

Re: [PERFORM] cpu bound postgresql setup.

2010-06-24 Thread Benjamin Krajmalnik
age- > From: pgsql-performance-ow...@postgresql.org [mailto:pgsql-performance- > ow...@postgresql.org] On Behalf Of Rajesh Kumar Mallah > Sent: Thursday, June 24, 2010 11:27 AM > To: Kevin Grittner > Cc: pgsql-performance@postgresql.org > Subject: Re: [PERFORM] cpu bound postgresql

Re: [PERFORM] cpu bound postgresql setup.

2010-06-24 Thread Rajesh Kumar Mallah
>i do not remember well but there is a system view that (i think) >guides at what stage the marginal returns of increasing it >starts disappearing , i had set it a few years back. Sorry the above comment was regarding setting shared_buffers not effective_cache_size. On Thu, Jun 24, 2010 at 10:5

Re: [PERFORM] cpu bound postgresql setup.

2010-06-24 Thread Rajesh Kumar Mallah
On Thu, Jun 24, 2010 at 8:57 PM, Kevin Grittner wrote: > I'm not clear whether you still have a problem, or whether the > changes you mention solved your issues. I'll comment on potential > issues that leap out at me. It shall require more observation to know if the "problem" is solved. my "pro

Re: [PERFORM] cpu bound postgresql setup.

2010-06-24 Thread Kevin Grittner
I'm not clear whether you still have a problem, or whether the changes you mention solved your issues. I'll comment on potential issues that leap out at me. Rajesh Kumar Mallah wrote: > 3. we use xfs and our controller has BBU , we changed barriers=1 > to barriers=0 as i learnt that having b

Re: [PERFORM] cpu bound postgresql setup.

2010-06-24 Thread Rajesh Kumar Mallah
Dear List, 1. It was found that too many stray queries were getting generated from rouge users and bots we controlled using some manual methods. 2. We have made application changes and some significant changes have been done. 3. we use xfs and our controller has BBU , we changed barriers=1

Re: [PERFORM] cpu bound postgresql setup.

2010-06-23 Thread Kevin Grittner
Your response somehow landed in the subject line, apparently truncated. I'll extract that to the message body and reply to what made it through. Rajesh Kumar Mallah wrote: > Firstly many thanks for responding. I am concerned because the > load averages have increased and users complaining of

Re: [PERFORM] cpu bound postgresql setup.

2010-06-23 Thread Kevin Grittner
Rajesh Kumar Mallah wrote: > PasteBin for the vmstat output > http://pastebin.com/mpHCW9gt > > On Wed, Jun 23, 2010 at 8:22 PM, Rajesh Kumar Mallah > wrote: >> Dear List , >> >> I observe that my postgresql (ver 8.4.2) dedicated server has >> turned cpu bound and there is a high load average in

Re: [PERFORM] cpu bound postgresql setup.

2010-06-23 Thread Rajesh Kumar Mallah
PasteBin for the vmstat output http://pastebin.com/mpHCW9gt On Wed, Jun 23, 2010 at 8:22 PM, Rajesh Kumar Mallah wrote: > Dear List , > > I observe that my postgresql (ver 8.4.2) dedicated server has turned cpu > bound and there is a high load average in the server > 50 usually. > The server has

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread Ivan Voras
Bryan Buecking wrote: On Tue, Apr 22, 2008 at 10:55:19AM -0500, Erik Jones wrote: Are you referring to PHP's persistent connections? Do not use those. Here's a thread that details the issues with why not: http://archives.postgresql.org/pgsql-general/2007-08/msg00660.php . Thanks for tha

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread PFC
about 2300 connections in idle (ps auxwww | grep postgres | idle) [...] The server that connects to the db is an apache server using persistent connections. MaxClients is 2048 thus the high number of connections needed. Application was written in PHP using the Pear DB class.

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread Craig Ringer
Erik Jones wrote: max_connections = 2400 That is WAY too high. Get a real pooler, such as pgpool, and drop that down to 1000 and test from there. I see you mentioned 500 concurrent connections. Are each of those connections actually doing something? My guess that once you cut down on th

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread Tom Lane
Bryan Buecking <[EMAIL PROTECTED]> writes: > On Tue, Apr 22, 2008 at 10:55:19AM -0500, Erik Jones wrote: >> That is WAY too high. Get a real pooler, such as pgpool, and drop >> that down to 1000 and test from there. > I agree, but the number of idle connections dont' seem to affect > performace

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread Scott Marlowe
On Tue, Apr 22, 2008 at 10:10 AM, Bryan Buecking <[EMAIL PROTECTED]> wrote: > > I agree, but the number of idle connections dont' seem to affect > performace only memory usage. I'm trying to lessen the load of > connection setup. But sounds like this tax is minimal? Not entirely true. There ar

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread Bryan Buecking
On Tue, Apr 22, 2008 at 01:21:03PM -0300, Rodrigo Gonzalez wrote: > Are tables vacuumed often? How often is often. Right now db is vaccumed once a day. -- Bryan Buecking http://www.starling-software.com -- Sent via pgsql-performance mailing list (pgsql-performance@post

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread Rodrigo Gonzalez
Are tables vacuumed often? Bryan Buecking escribió: On Tue, Apr 22, 2008 at 10:55:19AM -0500, Erik Jones wrote: On Apr 22, 2008, at 10:31 AM, Bryan Buecking wrote: max_connections = 2400 That is WAY too high. Get a real pooler, such as pgpool, and drop that down to 1000 and

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread Bryan Buecking
On Tue, Apr 22, 2008 at 10:55:19AM -0500, Erik Jones wrote: > > Are you referring to PHP's persistent connections? Do not use those. > Here's a thread that details the issues with why not: > http://archives.postgresql.org/pgsql-general/2007-08/msg00660.php . Thanks for that article, very

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread Harald Armin Massa
Bryan, > > about 2300 connections in idle > > > (ps auxwww | grep postgres | idle) that is about 2300 processes being task scheduled by your kernel, each of them using > 1 MB of RAM and some other ressources, are you sure that this is what you want? Usual recommended design for a web applicati

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread Bryan Buecking
On Tue, Apr 22, 2008 at 10:55:19AM -0500, Erik Jones wrote: > On Apr 22, 2008, at 10:31 AM, Bryan Buecking wrote: > > >max_connections = 2400 > > That is WAY too high. Get a real pooler, such as pgpool, and drop > that down to 1000 and test from there. I agree, but the number of idle connecti

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread Bryan Buecking
On Tue, Apr 22, 2008 at 08:41:09AM -0700, Joshua D. Drake wrote: > On Wed, 23 Apr 2008 00:31:01 +0900 > Bryan Buecking <[EMAIL PROTECTED]> wrote: > > > at any given time there is about 5-6 postgres in startup > > (ps auxwww | grep postgres | grep startup | wc -l) > > > > about 2300 connections i

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread Erik Jones
On Apr 22, 2008, at 10:31 AM, Bryan Buecking wrote: Hi, I'm running into an performance problem where a Postgres db is running at 99% CPU (4 cores) with about 500 concurrent connection doing various queries from a web application. This problem started about a week ago, and has been steadily

Re: [PERFORM] CPU bound at 99%

2008-04-22 Thread Joshua D. Drake
On Wed, 23 Apr 2008 00:31:01 +0900 Bryan Buecking <[EMAIL PROTECTED]> wrote: > at any given time there is about 5-6 postgres in startup > (ps auxwww | grep postgres | grep startup | wc -l) > > about 2300 connections in idle > (ps auxwww | grep postgres | idle) > > and loads of "FATAL: sorry, t

[PERFORM] CPU bound at 99%

2008-04-22 Thread Bryan Buecking
Hi, I'm running into an performance problem where a Postgres db is running at 99% CPU (4 cores) with about 500 concurrent connection doing various queries from a web application. This problem started about a week ago, and has been steadily going downhill. I have been tweaking the config a bit, mai