Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-10 Thread Steve Crawford
> It seems to me that the real solution is for me to stop using the database as > an IPC system to pass somewhat time-critical data between processes. Given > the time constraints I'm working under this unfortunately was the quickest > route. At least for the first 5 minutes. :) I was wonderin

Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-10 Thread Marc Rossi
Ok, partial day results. Looks like my changes have not solved the problem, just spread it out a little more (as would be expected based on your responses). The delays are now shorter (about half) but occur more frequently (maybe 1x / minute). The params I used are: bgwriter_lru_percent = 5.

Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-10 Thread Greg Smith
On Fri, 10 Aug 2007, Marc Rossi wrote: Thanks for the heads up. The box in question is a dual cpu (xeon dual cores) with 8 gig & a pair of 10k 146gb raid 1 arrays. I have the pg_xlog dir on one array (along with the OS) & the rest of the data on the other array by itself. Yeah, that's kind

Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-10 Thread Marc Rossi
pointing a little more often with less data? (right now I hit the checkpoint_timeout). Thanks again, Marc - Original Message From: Greg Smith <[EMAIL PROTECTED]> To: Marc Rossi <[EMAIL PROTECTED]> Cc: pgsql-general@postgresql.org Sent: Friday, August 10, 2007 2:36:28 AM Subje

Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-10 Thread Greg Smith
On Thu, 9 Aug 2007, Marc Rossi wrote: as well as made changes to the bgwriter settings as shown below (taken from a post in the pgsql-performance list) bgwriter_lru_percent = 20.0 # 0-100% of LRU buffers scanned/round bgwriter_lru_maxpages = 200 # 0-1000 buffers max writ

Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-09 Thread Marc Rossi
thread for others who may experience the same problem in the future. Thanks again, Marc - Original Message From: Greg Smith <[EMAIL PROTECTED]> Cc: mr19 <[EMAIL PROTECTED]>; pgsql-general@postgresql.org Sent: Thursday, August 9, 2007 11:10:42 PM Subject: Re: [GENERAL] UPDATES ha

Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-09 Thread Greg Smith
On Thu, 9 Aug 2007, Scott Marlowe wrote: Wouldn't that be the other way around, set checkpoint_warning to 1 so it triggers every time the checkpoint happens? The log message appears if the checkpoints happen more frequently than the value, so setting to 1 would only trigger a warning if you g

Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-09 Thread Scott Marlowe
On 8/9/07, Greg Smith <[EMAIL PROTECTED]> wrote: > On Tue, 7 Aug 2007, mr19 wrote: > > > I have a process that updates ~ 1500 rows in a table once a second. Every 5 > > minutes (almost exactly) the update takes ~ 15 seconds (normally < 1). > > Lots of updates will trigger checkpoints and, if you h

Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-09 Thread Tom Lane
Greg Smith <[EMAIL PROTECTED]> writes: > Try increasing checkpoint_warning in your postgresql.conf file to its > maximum of 3600 and restart the server when you can tolerate a small > service disruption; You don't need a server restart to change checkpoint_warning --- SIGHUP ("pg_ctl reload") sh

Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-09 Thread Tom Lane
mr19 <[EMAIL PROTECTED]> writes: > I have a process that updates ~ 1500 rows in a table once a second. Every 5 > minutes (almost exactly) the update takes ~ 15 seconds (normally < 1). Checkpoints? > I have run htop/top on the machine during this time period and do not see > anything unusual. Tr

Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-09 Thread Greg Smith
On Tue, 7 Aug 2007, mr19 wrote: I have a process that updates ~ 1500 rows in a table once a second. Every 5 minutes (almost exactly) the update takes ~ 15 seconds (normally < 1). Lots of updates will trigger checkpoints and, if you have auto-vacuum turned on, regular vacuum activity--either

Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-09 Thread Steve Crawford
mr19 wrote: > I have a process that updates ~ 1500 rows in a table once a second. Every 5 > minutes (almost exactly) the update takes ~ 15 seconds (normally < 1) autovacuum_naptime perhaps? Cheers, Steve ---(end of broadcast)--- TIP 4: Have y

Re: [GENERAL] UPDATES hang every 5 minutes

2007-08-09 Thread Joseph S
Keep an eye on pg_stat_activity and pg_locks to see if any lock contention is going on. mr19 wrote: I have a process that updates ~ 1500 rows in a table once a second. Every 5 minutes (almost exactly) the update takes ~ 15 seconds (normally < 1). I have run htop/top on the machine during thi

[GENERAL] UPDATES hang every 5 minutes

2007-08-09 Thread mr19
I have a process that updates ~ 1500 rows in a table once a second. Every 5 minutes (almost exactly) the update takes ~ 15 seconds (normally < 1). I have run htop/top on the machine during this time period and do not see anything unusual. I am running postgres 8.1.8 on a FC6 box. Any type