Re: base backup client as auxiliary backend process

2020-03-28 Thread Alvaro Herrera
On 2020-Jan-14, Peter Eisentraut wrote: > On 2020-01-14 07:32, Masahiko Sawada wrote: > > - Replication slot name used by this WAL receiver > > + > > + Replication slot name used by this WAL receiver. This is only set > > if a > > + permanent replication slot is set using > +

Re: base backup client as auxiliary backend process

2020-02-28 Thread Peter Eisentraut
I have set this patch to "returned with feedback" in the upcoming commit fest, because I will not be able to finish it. Unsurprisingly, the sequencing of startup actions in postmaster.c is extremely tricky and needs more thinking. All the rest worked pretty well, I thought. -- Peter Eisentr

Re: base backup client as auxiliary backend process

2020-02-22 Thread Peter Eisentraut
On 2020-02-17 18:42, Peter Eisentraut wrote: On 2020-02-03 13:47, Andres Freund wrote: Comment: - It'd be good to split out the feature independent refactorings, like the introduction of InitControlFile(), into their own commit. Right now it's hard to separate out what should just shoul

Re: base backup client as auxiliary backend process

2020-02-17 Thread Peter Eisentraut
On 2020-02-03 13:47, Andres Freund wrote: Comment: - It'd be good to split out the feature independent refactorings, like the introduction of InitControlFile(), into their own commit. Right now it's hard to separate out what should just should be moved code, and what should be behaviour

Re: base backup client as auxiliary backend process

2020-02-04 Thread Masahiko Sawada
On Mon, 3 Feb 2020 at 20:06, Andres Freund wrote: > > Hi, > > On 2020-01-11 10:52:30 +0100, Peter Eisentraut wrote: > > On 2020-01-10 04:32, Masahiko Sawada wrote: > > > I agreed that these patches are useful on its own and 0001 patch and > > > > committed 0001 > > over on -committers Robert compl

Re: base backup client as auxiliary backend process

2020-02-03 Thread Michael Paquier
On Mon, Feb 03, 2020 at 01:37:25AM -0800, Andres Freund wrote: > On 2020-01-23 15:49:37 -0500, Robert Haas wrote: >> I assume that's probably discussed on the thread that is linked here, >> but you shouldn't have to dig through the discussion thread to figure >> out what the benefits of a change li

Re: base backup client as auxiliary backend process

2020-02-03 Thread Andres Freund
Hi, Comment: - It'd be good to split out the feature independent refactorings, like the introduction of InitControlFile(), into their own commit. Right now it's hard to separate out what should just should be moved code, and what should be behavioural changes. Especially when there's stuff

Re: base backup client as auxiliary backend process

2020-02-03 Thread Andres Freund
Hi, On 2020-01-11 10:52:30 +0100, Peter Eisentraut wrote: > On 2020-01-10 04:32, Masahiko Sawada wrote: > > I agreed that these patches are useful on its own and 0001 patch and > > committed 0001 over on -committers Robert complained: On 2020-01-23 15:49:37 -0500, Robert Haas wrote: > On Tue, J

Re: base backup client as auxiliary backend process

2020-01-19 Thread Masahiko Sawada
On Thu, 16 Jan 2020 at 00:17, Peter Eisentraut wrote: > > On 2020-01-15 01:40, Masahiko Sawada wrote: > > Could you rebase the main patch that adds base backup client as > > auxiliary backend process since the previous version patch (v3) > > conflicts with the current HE

Re: base backup client as auxiliary backend process

2020-01-15 Thread Peter Eisentraut
On 2020-01-09 11:57, Alexandra Wang wrote: Back to the base backup stuff, I don't quite understand all the benefits you mentioned above. It seems to me the greatest benefit with this patch is that postmaster takes care of pg_basebackup itself, which reduces the human wait in between running the

Re: base backup client as auxiliary backend process

2020-01-15 Thread Peter Eisentraut
On 2020-01-15 01:40, Masahiko Sawada wrote: Could you rebase the main patch that adds base backup client as auxiliary backend process since the previous version patch (v3) conflicts with the current HEAD? attached -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL

Re: base backup client as auxiliary backend process

2020-01-14 Thread Masahiko Sawada
se are not > > shown > > + here. > > + > > > > Now that the slot name is shown even if it's a temp slot the above > > documentation changes needs to be changed. Other changes look good to > > me. > > committed, thanks Thank you for co

Re: base backup client as auxiliary backend process

2020-01-14 Thread Peter Eisentraut
On 2020-01-14 07:32, Masahiko Sawada wrote: - Replication slot name used by this WAL receiver + + Replication slot name used by this WAL receiver. This is only set if a + permanent replication slot is set using . Otherwise, the WAL receiver may use + a temporary replicat

Re: base backup client as auxiliary backend process

2020-01-13 Thread Masahiko Sawada
On Sat, 11 Jan 2020 at 18:52, Peter Eisentraut wrote: > > On 2020-01-10 04:32, Masahiko Sawada wrote: > > I agreed that these patches are useful on its own and 0001 patch and > > committed 0001 > > > 0002 patch look good to me. For 0003 patch, > > > > + linkend="guc-primary-slot-name"/>. Oth

Re: base backup client as auxiliary backend process

2020-01-11 Thread Peter Eisentraut
On 2020-01-10 04:32, Masahiko Sawada wrote: I agreed that these patches are useful on its own and 0001 patch and committed 0001 0002 patch look good to me. For 0003 patch, + linkend="guc-primary-slot-name"/>. Otherwise, the WAL receiver may use + a temporary replication slot (dete

Re: base backup client as auxiliary backend process

2020-01-09 Thread Masahiko Sawada
On Fri, 22 Nov 2019 at 19:22, Peter Eisentraut wrote: > > On 2019-11-15 14:52, Sergei Kornilov wrote: > >> I looked into this. It seems trivial to make walsender create and use a > >> temporary replication slot by default if no permanent replication slot > >> is specified. This is basically the lo

Re: base backup client as auxiliary backend process

2020-01-09 Thread Alexandra Wang
Hi Peter, On Fri, Nov 22, 2019 at 6:22 PM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > Here is another patch set that implements the temporary slot use on the > walreceiver side, essentially mirroring what pg_basebackup already does. > > I think this patch set might be useful on

Re: base backup client as auxiliary backend process

2019-11-22 Thread Michael Paquier
On Fri, Nov 22, 2019 at 11:21:53AM +0100, Peter Eisentraut wrote: > After thinking about this a bit more, doing the temporary slot stuff on the > walsender side might lead to too many complications in practice. > > Here is another patch set that implements the temporary slot use on the > walreceiv

Re: base backup client as auxiliary backend process

2019-11-22 Thread Peter Eisentraut
On 2019-11-15 14:52, Sergei Kornilov wrote: I looked into this. It seems trivial to make walsender create and use a temporary replication slot by default if no permanent replication slot is specified. This is basically the logic that pg_basebackup has but done server-side. See attached patch for

Re: base backup client as auxiliary backend process

2019-11-15 Thread Sergei Kornilov
Hello Could you rebase patch please? I have errors during patch apply. CFbot checks latest demonstration patch. > I looked into this. It seems trivial to make walsender create and use a > temporary replication slot by default if no permanent replication slot > is specified. This is basically the

Re: base backup client as auxiliary backend process

2019-11-09 Thread Peter Eisentraut
On 2019-11-07 05:16, Michael Paquier wrote: The current defaults of pg_basebackup have been thought so as the backups taken have a good stability and so as monitoring is eased thanks to --wal-method=stream and the use of replication slots. Shouldn't the use of a least a temporary replication slot

Re: base backup client as auxiliary backend process

2019-11-06 Thread Michael Paquier
On Mon, Oct 28, 2019 at 09:30:52AM +0100, Peter Eisentraut wrote: > On 2019-09-18 10:31, Michael Paquier wrote: >> -* Verify XLOG status looks valid. >> +* Check that contents look valid. >> */ >> - if (ControlFile->state < DB_SHUTDOWNED || >> - ControlFile->state > DB_IN_PROD

Re: base backup client as auxiliary backend process

2019-10-28 Thread Peter Eisentraut
rant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From ac34ece7665b62d542653cf12238973a1a45a18b Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Mon, 28 Oct 2019 09:23:43 +0100 Subject: [PATCH v3] Base backup client as auxiliary backend process Discussio

Re: base backup client as auxiliary backend process

2019-09-18 Thread Michael Paquier
On Fri, Aug 30, 2019 at 09:10:10PM +0200, Peter Eisentraut wrote: > > Attached is a very hackish patch to implement this. It works like this: > > > > # (assuming you have a primary already running somewhere) > > initdb -D data2 --replica > > $EDITOR data2/postgresql.conf # set primar

Re: base backup client as auxiliary backend process

2019-09-11 Thread Kyotaro Horiguchi
Hello, thanks for pinging. At Wed, 11 Sep 2019 19:15:24 -0300, Alvaro Herrera wrote in <20190911221524.GA16563@alvherre.pgsql> > On 2019-Aug-30, Peter Eisentraut wrote: > > > Attached is an updated patch for this. I have changed the initdb option > > name per suggestion. The WAL receiver is n

Re: base backup client as auxiliary backend process

2019-09-11 Thread Alvaro Herrera
On 2019-Aug-30, Peter Eisentraut wrote: > Attached is an updated patch for this. I have changed the initdb option > name per suggestion. The WAL receiver is now started concurrently with > the base backup. There is progress reporting (ps display), fsyncing. > Configuration files are not copied

Re: base backup client as auxiliary backend process

2019-08-30 Thread Peter Eisentraut
cbb2a1ae4ff5d2e80abce0798799fe73 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Fri, 30 Aug 2019 20:42:51 +0200 Subject: [PATCH v2] Base backup client as auxiliary backend process Discussion: https://www.postgresql.org/message-id/flat/61b8d18d-c922-ac99-b990-a31ba63cd...@2ndquadrant.com

Re: base backup client as auxiliary backend process

2019-07-11 Thread Kyotaro Horiguchi
Hello. At Sat, 29 Jun 2019 22:05:22 +0200, Peter Eisentraut wrote in <61b8d18d-c922-ac99-b990-a31ba63cd...@2ndquadrant.com> > Setting up a standby instance is still quite complicated. You need to > run pg_basebackup with all the right options. You need to make sure > pg_basebackup has the rig

Re: base backup client as auxiliary backend process

2019-07-11 Thread Tom Lane
Peter Eisentraut writes: > On 2019-07-11 22:20, Robert Haas wrote: >> On Thu, Jul 11, 2019 at 4:10 PM Tom Lane wrote: >>> How's the postmaster to know that it's supposed to run pg_basebackup >>> rather than start normally? Where will it get the connection information? >>> Seem to need configurat

Re: base backup client as auxiliary backend process

2019-07-11 Thread Peter Eisentraut
On 2019-07-11 22:20, Robert Haas wrote: > On Thu, Jul 11, 2019 at 4:10 PM Tom Lane wrote: >> Robert Haas writes: >>> On Thu, Jul 11, 2019 at 10:36 AM Tom Lane wrote: Gotta have config files in place already, no? >> >>> Why? >> >> How's the postmaster to know that it's supposed to run pg_bas

Re: base backup client as auxiliary backend process

2019-07-11 Thread Robert Haas
On Thu, Jul 11, 2019 at 4:10 PM Tom Lane wrote: > Robert Haas writes: > > On Thu, Jul 11, 2019 at 10:36 AM Tom Lane wrote: > >> Gotta have config files in place already, no? > > > Why? > > How's the postmaster to know that it's supposed to run pg_basebackup > rather than start normally? Where w

Re: base backup client as auxiliary backend process

2019-07-11 Thread Tom Lane
Robert Haas writes: > On Thu, Jul 11, 2019 at 10:36 AM Tom Lane wrote: >> Gotta have config files in place already, no? > Why? How's the postmaster to know that it's supposed to run pg_basebackup rather than start normally? Where will it get the connection information? Seem to need configurati

Re: base backup client as auxiliary backend process

2019-07-11 Thread Robert Haas
On Thu, Jul 11, 2019 at 10:36 AM Tom Lane wrote: > Gotta have config files in place already, no? Why? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: base backup client as auxiliary backend process

2019-07-11 Thread Tom Lane
Robert Haas writes: > On Sat, Jun 29, 2019 at 4:05 PM Peter Eisentraut > wrote: >> My idea is that the postmaster can launch a base backup worker, wait >> till it's done, then proceed with the rest of the startup. initdb gets >> a special option to create a "minimal" data directory with only a f

Re: base backup client as auxiliary backend process

2019-07-11 Thread Robert Haas
On Sat, Jun 29, 2019 at 4:05 PM Peter Eisentraut wrote: > My idea is that the postmaster can launch a base backup worker, wait > till it's done, then proceed with the rest of the startup. initdb gets > a special option to create a "minimal" data directory with only a few > files, directories, and

Re: base backup client as auxiliary backend process

2019-07-11 Thread Euler Taveira
Em sáb, 29 de jun de 2019 às 17:05, Peter Eisentraut escreveu: > > Setting up a standby instance is still quite complicated. You need to > run pg_basebackup with all the right options. You need to make sure > Attached is a very hackish patch to implement this. It works like this: > > # (ass

Re: base backup client as auxiliary backend process

2019-07-11 Thread Sergei Kornilov
Hello >>  Attached is a very hackish patch to implement this. It works like this: >> >>  # (assuming you have a primary already running somewhere) >>  initdb -D data2 --minimal >>  $EDITOR data2/postgresql.conf # set primary_conninfo >>  pg_ctl -D data2 start > > +1, very nice. How

Re: base backup client as auxiliary backend process

2019-07-04 Thread Thomas Munro
On Sun, Jun 30, 2019 at 8:05 AM Peter Eisentraut wrote: > Attached is a very hackish patch to implement this. It works like this: > > # (assuming you have a primary already running somewhere) > initdb -D data2 --minimal > $EDITOR data2/postgresql.conf # set primary_conninfo > pg_

base backup client as auxiliary backend process

2019-06-29 Thread Peter Eisentraut
rom: Peter Eisentraut Date: Sat, 29 Jun 2019 21:52:21 +0200 Subject: [PATCH v1] Base backup client as auxiliary backend process --- src/backend/access/transam/xlog.c | 14 +- src/backend/bootstrap/bootstrap.c | 9 + src/backend/postmaster/pgstat.c | 6