Re: [HACKERS] WAL "low watermark" during base backup

2011-09-09 Thread Tom Lane
Magnus Hagander writes: > On Fri, Sep 9, 2011 at 13:40, Dimitri Fontaine wrote: >> I'm not getting why we need the later one when we have this older one? > One of them is for the simple case. It requires a single connection to > the server, and it supports things like writing to tarfiles and > c

Re: [HACKERS] WAL "low watermark" during base backup

2011-09-09 Thread Dimitri Fontaine
Florian Pflug writes: > Couldn't we send all available WAL after each single data-file instead > of waiting for all data files to be transferred before sending WAL? +1 (or maybe not at the file boundary but rather driven by archive command with some internal hooking, as the backend needs some new

Re: [HACKERS] WAL "low watermark" during base backup

2011-09-09 Thread Florian Pflug
On Sep9, 2011, at 13:48 , Magnus Hagander wrote: > On Fri, Sep 9, 2011 at 13:40, Dimitri Fontaine wrote: >> Magnus Hagander writes: If you must have this then make pg_basebackup copy xlog files regularly during the backup. That way your backup can take forever and your primary disk

Re: [HACKERS] WAL "low watermark" during base backup

2011-09-09 Thread Magnus Hagander
On Fri, Sep 9, 2011 at 13:40, Dimitri Fontaine wrote: > Magnus Hagander writes: >>> If you must have this then make pg_basebackup copy xlog files >>> regularly during the backup. That way your backup can take forever and >>> your primary disk won't fill up. In many cases it actually will take >>>

Re: [HACKERS] WAL "low watermark" during base backup

2011-09-09 Thread Dimitri Fontaine
Magnus Hagander writes: >> If you must have this then make pg_basebackup copy xlog files >> regularly during the backup. That way your backup can take forever and >> your primary disk won't fill up. In many cases it actually will take >> forever, but at least we don't take down the primary. > > Th

Re: [HACKERS] WAL "low watermark" during base backup

2011-09-09 Thread Magnus Hagander
On Tue, Sep 6, 2011 at 22:35, Simon Riggs wrote: > On Mon, Sep 5, 2011 at 11:38 AM, Magnus Hagander wrote: >> On Sun, Sep 4, 2011 at 19:02, Simon Riggs wrote: >>> On Fri, Sep 2, 2011 at 6:52 PM, Magnus Hagander wrote: >>> Attached patch implements a "low watermark wal location" in the

Re: [HACKERS] WAL "low watermark" during base backup

2011-09-06 Thread Simon Riggs
On Mon, Sep 5, 2011 at 11:38 AM, Magnus Hagander wrote: > On Sun, Sep 4, 2011 at 19:02, Simon Riggs wrote: >> On Fri, Sep 2, 2011 at 6:52 PM, Magnus Hagander wrote: >> >>> Attached patch implements a "low watermark wal location" in the >>> walsender shmem array. Setting this value in a walsender

Re: [HACKERS] WAL "low watermark" during base backup

2011-09-05 Thread Magnus Hagander
On Sun, Sep 4, 2011 at 19:02, Simon Riggs wrote: > On Fri, Sep 2, 2011 at 6:52 PM, Magnus Hagander wrote: > >> Attached patch implements a "low watermark wal location" in the >> walsender shmem array. Setting this value in a walsender prevents >> transaction log removal prior to this point - simi

Re: [HACKERS] WAL "low watermark" during base backup

2011-09-04 Thread Simon Riggs
On Fri, Sep 2, 2011 at 6:52 PM, Magnus Hagander wrote: > Attached patch implements a "low watermark wal location" in the > walsender shmem array. Setting this value in a walsender prevents > transaction log removal prior to this point - similar to how > wal_keep_segments work, except with an abso

Re: [HACKERS] WAL "low watermark" during base backup

2011-09-03 Thread Dimitri Fontaine
Magnus Hagander writes: > Attached patch implements a "low watermark wal location" in the > walsender shmem array. Setting this value in a walsender prevents > transaction log removal prior to this point - similar to how > wal_keep_segments work, except with an absolute number rather than Cool.

Re: [HACKERS] WAL "low watermark" during base backup

2011-09-02 Thread Tom Lane
Magnus Hagander writes: > Attached patch implements a "low watermark wal location" in the > walsender shmem array. Setting this value in a walsender prevents > transaction log removal prior to this point - similar to how > wal_keep_segments work, except with an absolute number rather than > relati

Re: [HACKERS] WAL "low watermark" during base backup

2011-09-02 Thread Magnus Hagander
On Fri, Sep 2, 2011 at 20:12, Jaime Casanova wrote: > On Fri, Sep 2, 2011 at 12:52 PM, Magnus Hagander wrote: >> Attached patch implements a "low watermark wal location" in the >> walsender shmem array. Setting this value in a walsender prevents >> transaction log removal prior to this point - si

Re: [HACKERS] WAL "low watermark" during base backup

2011-09-02 Thread Jaime Casanova
On Fri, Sep 2, 2011 at 12:52 PM, Magnus Hagander wrote: > Attached patch implements a "low watermark wal location" in the > walsender shmem array. Setting this value in a walsender prevents > transaction log removal prior to this point - similar to how > wal_keep_segments work, except with an abso

[HACKERS] WAL "low watermark" during base backup

2011-09-02 Thread Magnus Hagander
Attached patch implements a "low watermark wal location" in the walsender shmem array. Setting this value in a walsender prevents transaction log removal prior to this point - similar to how wal_keep_segments work, except with an absolute number rather than relative. For now, this is set when runni