Re: [GENERAL] replication timeout in pg_basebackup

2014-03-11 Thread John R Pierce
On 3/11/2014 5:50 AM, Aggarwal, Ajay wrote: Thats exactly what I was thinking after all other experiments. Couple of questions: 1) why did you say that 300 seconds is the upper limit? Is this enforced by Postgres? What if I want to set it to 10 minutes? 2) whats the downside of bigger replicati

Re: [GENERAL] replication timeout in pg_basebackup

2014-03-11 Thread Aggarwal, Ajay
From: pgsql-general-ow...@postgresql.org [pgsql-general-ow...@postgresql.org] on behalf of John R Pierce [pie...@hogranch.com] Sent: Monday, March 10, 2014 9:58 PM To: pgsql-general@postgresql.org Subject: Re: [GENERAL] replication timeout in pg_basebackup On 3/9/2014 6:52 PM

Re: [GENERAL] replication timeout in pg_basebackup

2014-03-10 Thread John R Pierce
On 3/9/2014 6:52 PM, Aggarwal, Ajay wrote: Our replication timeout is default 60 seconds. If we increase the replication time to say 180 seconds, we see better results but backups still fail occasionally. so increase it to 300 seconds, or whatever. thats an upper limit, it needs to be big e

Re: [GENERAL] replication timeout in pg_basebackup

2014-03-10 Thread Aggarwal, Ajay
t: RE: [GENERAL] replication timeout in pg_basebackup I have already tried experimenting with linux dirty_ratio etc. You can only fine tune up to a limit. The backup process still fills up the buffer cache very quickly. Yes, my database is about 5-6 GB in size and will grow bigger over time. If

Re: [GENERAL] replication timeout in pg_basebackup

2014-03-10 Thread Aggarwal, Ajay
force it to use direct I/O. From: Haribabu Kommi [kommi.harib...@gmail.com] Sent: Monday, March 10, 2014 8:31 PM To: Aggarwal, Ajay Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] replication timeout in pg_basebackup On Tue, Mar 11, 2014 at 7:07 AM

Re: [GENERAL] replication timeout in pg_basebackup

2014-03-10 Thread Haribabu Kommi
On Tue, Mar 11, 2014 at 7:07 AM, Aggarwal, Ajay wrote: > Thanks Hari Babu. > > I think what is happening is that my dirty cache builds up quickly for the > volume where I am backing up. This would trigger flush of these dirty pages > to the disk. While this flush is going on pg_basebackup tries to

Re: [GENERAL] replication timeout in pg_basebackup

2014-03-10 Thread Aggarwal, Ajay
207.248 ops/sec Non-Sync'ed 8kB writes: write 202216.900 ops/sec From: Haribabu Kommi [kommi.harib...@gmail.com] Sent: Monday, March 10, 2014 1:42 AM To: Aggarwal, Ajay Cc: pgsql-general@postgresql.org Subject: Re:

Re: [GENERAL] replication timeout in pg_basebackup

2014-03-09 Thread Haribabu Kommi
On Mon, Mar 10, 2014 at 12:52 PM, Aggarwal, Ajay wrote: > Our environment: Postgres version 9.2.2 running on CentOS 6.4 > > Our backups using pg_basebackup are frequently failing with following error > > "pg_basebackup: could not send feedback packet: server closed the connection > unexpectedly

[GENERAL] replication timeout in pg_basebackup

2014-03-09 Thread Aggarwal, Ajay
Our environment: Postgres version 9.2.2 running on CentOS 6.4 Our backups using pg_basebackup are frequently failing with following error "pg_basebackup: could not send feedback packet: server closed the connection unexpectedly This probably means the server terminated abnormally