Re: [PERFORM] Very slow checkpoints

2015-03-19 Thread Joao Junior
Would not be the case slightly decrease the shared buffer, lower your checkpoints_timeout for eg 5 minutes or decrease checkpoints_segments and set the checkpoint_completation_target to 0.5 to not mess up the next checkpoints? What logs tell me is that a checkpoint occurs immediately to the other

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Steven Jones
Hi, >> >> Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz >> avgqu-sz await r_await w_await svctm %util >> sda 0.00 0.00 0.00 5.00 0.00 2056.00 822.40 >> 0.00 0.00 0.00 0.00 0.00 0.00 >> sdb 0.00 0.00 1055.00 549.00 41166.50 22840.00 79.81 >> 5.28 3.28 4.94 0.10 0.62 100.00 > Your sdb is satura

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread didier
Hi, On Wed, Mar 18, 2015 at 12:21 PM, Steven Jones wrote: > Hi, > iostat -x snapshot: > > avg-cpu: %user %nice %system %iowait %steal %idle >0.500.002.35 15.090.00 82.05 > > Device: rrqm/s wrqm/s r/s w/srkB/swkB/s avgrq-sz > avgqu-sz

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Ilya Kosmodemiansky
On Wed, Mar 18, 2015 at 1:21 PM, Steven Jones wrote: > #autovacuum = on# Enable autovacuum subprocess? 'on' > #log_autovacuum_min_duration = -1 # -1 disables, 0 logs all actions and > #autovacuum_max_workers = 3 # max number of autovacuum > subprocesses

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Steven Jones
limit = -1      # default vacuum cost limit for                                         # autovacuum, -1 means use Regards, Steve > From: ilya.kosmodemian...@postgresql-consulting.com > Date: Wed, 18 Mar 2015 13:17:14 +0100 > Subject: Re: [PERFORM] Ve

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Steven Jones
bject: Re: [PERFORM] Very slow checkpoints > To: steven.jones1...@outlook.com > CC: pgsql-performance@postgresql.org > > On Wed, Mar 18, 2015 at 12:58 PM, Steven Jones > wrote: >> Yes BBU is on the controller; 1024Mb. It is a HP P410i controller, with >> write caching t

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Ilya Kosmodemiansky
On Wed, Mar 18, 2015 at 12:21 PM, Steven Jones wrote: > - typical DW - relatively constant periodic data loads - i.e. heavy write > - we receive large CSV files ~ 5-10Gb every 15 minutes spread out across > 5-7 minutes > - DB size is ~2.5Tb; rotating load of 30 days keeps the database stable

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Ilya Kosmodemiansky
On Wed, Mar 18, 2015 at 12:58 PM, Steven Jones wrote: > Yes BBU is on the controller; 1024Mb. It is a HP P410i controller, with write > caching turned on the controller; off on disk level. vm.dirty_background_bytes=67108864 and vm.dirty_bytes=536870912 looks resonable for 512MB BBU, you can cal

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Steven Jones
will try that as well. We don't at this stage have any graphs, but we will set it up over the next 24hrs at least. Regards, Steve > From: ilya.kosmodemian...@postgresql-consulting.com > Date: Wed, 18 Mar 2015 12:42:43 +0100 > Subject: Re:

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Ilya Kosmodemiansky
Hi Steven, On Wed, Mar 18, 2015 at 12:21 PM, Steven Jones wrote: > - system is HP blade; 128Gb RAM, 2x 8-core, 12x 10k RPM RAID1+0 (database) Have you BBU on your controller? And how your controller configured, I mean cache mode, io mode, disk write cache mode. You have 15K SAS (which form fac

Re: [PERFORM] Very slow checkpoints

2015-03-18 Thread Steven Jones
Apologies about the formatting; resending again as plain-text. Regards, Steve From: steven.jones1...@outlook.com To: pgsql-performance@postgresql.org Subject: [PERFORM] Very slow checkpoints Date: Wed, 18 Mar 2015 11:21:08 + Hi, We have a so far (to us) unexplainable issue on our

[PERFORM] Very slow checkpoints

2015-03-18 Thread Steven Jones
Hi, We have a so far (to us) unexplainable issue on our production systems after we roughly doubled the amount of data we import daily. We should be ok on pure theoretical hardware performance, but we are seeing some weird IO counters when the actual throughput of the writes is very low. The use