Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-11-01 Thread Brian Fehrle
Update on this: We did a switchover to another machine with the same hardware, however this system was running on some older parameters we had set in the postgresql.conf file. So we went from 400 max_connections to 200 max_connections, and 160MB work_mem to 200MB work_mem. And now on this ot

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-11-01 Thread Tom Lane
Brian Fehrle writes: > Update on this: > We did a switchover to another machine with the same hardware, however > this system was running on some older parameters we had set in the > postgresql.conf file. > So we went from 400 max_connections to 200 max_connections, and 160MB > work_mem to 200

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-31 Thread David Kerr
On Thu, Oct 27, 2011 at 02:09:51PM -0600, Brian Fehrle wrote: - On 10/27/2011 01:48 PM, Scott Marlowe wrote: - >On Thu, Oct 27, 2011 at 12:39 PM, Brian Fehrle - > wrote: - >>Looking at top, I see no SWAP usage, very little IOWait, and there are a - >>large number of postmaster processes at 100% cp

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-28 Thread Radosław Smogura
On Thu, 27 Oct 2011 12:43:00 -0700, John R Pierce wrote: On 10/27/11 11:39 AM, Brian Fehrle wrote: I've got a system that has 32 cores and 128 gigs of ram. We have connection pooling set up, with about 100 - 200 persistent connections open to the database. Our applications then use these conn

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-27 Thread Alan Hodgson
On October 27, 2011 01:09:51 PM Brian Fehrle wrote: > We've restarted the postgresql cluster, so the issue is not happening at > this moment. but running a vmstat 10 had my 'cs' average at 3K and 'in' > averaging around 9.5K. Random thought, is there any chance the server is physically overheating

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-27 Thread Brian Fehrle
On 10/27/2011 01:48 PM, Scott Marlowe wrote: On Thu, Oct 27, 2011 at 12:39 PM, Brian Fehrle wrote: Looking at top, I see no SWAP usage, very little IOWait, and there are a large number of postmaster processes at 100% cpu usage (makes sense, at this point there are 150 or so queries currently e

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-27 Thread Brian Fehrle
Also, I'm not having any issue with the database restarting itself, simply becoming unresponsive / slow to respond, to the point where just sshing to the box takes about 30 seconds if not longer. Performing a pg_ctl restart on the cluster resolves the issue. I looked through the logs for any s

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-27 Thread Tom Lane
Brian Fehrle writes: > Hi all, need some help/clues on tracking down a performance issue. > PostgreSQL version: 8.3.11 > I've got a system that has 32 cores and 128 gigs of ram. We have > connection pooling set up, with about 100 - 200 persistent connections > open to the database. Our applicat

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-27 Thread Brian Fehrle
On 10/27/2011 02:27 PM, Scott Mead wrote: On Thu, Oct 27, 2011 at 2:39 PM, Brian Fehrle mailto:bri...@consistentstate.com>> wrote: Hi all, need some help/clues on tracking down a performance issue. PostgreSQL version: 8.3.11 I've got a system that has 32 cores and 128 gigs of r

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-27 Thread Scott Mead
On Thu, Oct 27, 2011 at 2:39 PM, Brian Fehrle wrote: > Hi all, need some help/clues on tracking down a performance issue. > > PostgreSQL version: 8.3.11 > > I've got a system that has 32 cores and 128 gigs of ram. We have connection > pooling set up, with about 100 - 200 persistent connections ope

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-27 Thread Brian Fehrle
On 10/27/2011 02:50 PM, Tom Lane wrote: Brian Fehrle writes: Hi all, need some help/clues on tracking down a performance issue. PostgreSQL version: 8.3.11 I've got a system that has 32 cores and 128 gigs of ram. We have connection pooling set up, with about 100 - 200 persistent connections open

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-27 Thread Scott Marlowe
On Thu, Oct 27, 2011 at 1:52 PM, Scott Marlowe wrote: > On Thu, Oct 27, 2011 at 1:48 PM, Scott Marlowe > wrote: >> OK, a few points.  1: You've got a zombie process.  Find out what's > > To expand on the zombie thing, it's quite possible that you're > managing to make a pg backend process crasho

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-27 Thread Scott Marlowe
On Thu, Oct 27, 2011 at 1:48 PM, Scott Marlowe wrote: > OK, a few points.  1: You've got a zombie process.  Find out what's To expand on the zombie thing, it's quite possible that you're managing to make a pg backend process crashout, which would cause the db to restart midday, which is bad (TM)

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-27 Thread Scott Marlowe
On Thu, Oct 27, 2011 at 12:39 PM, Brian Fehrle wrote: > Looking at top, I see no SWAP usage, very little IOWait, and there are a > large number of postmaster processes at 100% cpu usage (makes sense, at this > point there are 150 or so queries currently executing on the database). > >  Tasks: 713

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-27 Thread John R Pierce
On 10/27/11 11:39 AM, Brian Fehrle wrote: I've got a system that has 32 cores and 128 gigs of ram. We have connection pooling set up, with about 100 - 200 persistent connections open to the database. Our applications then use these connections to query the database constantly, but when a conn

[GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-27 Thread Brian Fehrle
Hi all, need some help/clues on tracking down a performance issue. PostgreSQL version: 8.3.11 I've got a system that has 32 cores and 128 gigs of ram. We have connection pooling set up, with about 100 - 200 persistent connections open to the database. Our applications then use these connection