Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-11-13 Thread Andres Freund
Hi, On 2023-11-14 09:13:44 +0900, Michael Paquier wrote: > On Mon, Nov 13, 2023 at 03:41:44PM -0800, Andres Freund wrote: > > On 2023-11-09 12:16:52 +0900, Michael Paquier wrote: > >> On Thu, Nov 09, 2023 at 12:04:19PM +0900, Michael Paquier wrote: > >> > Sure, sorry for the confusion. By "we'd d

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-11-13 Thread Michael Paquier
On Mon, Nov 13, 2023 at 03:41:44PM -0800, Andres Freund wrote: > On 2023-11-09 12:16:52 +0900, Michael Paquier wrote: >> On Thu, Nov 09, 2023 at 12:04:19PM +0900, Michael Paquier wrote: >> > Sure, sorry for the confusion. By "we'd do nothing", I mean precirely >> > "to take no specific action rela

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-11-13 Thread Andres Freund
Hi, On 2023-11-09 12:16:52 +0900, Michael Paquier wrote: > On Thu, Nov 09, 2023 at 12:04:19PM +0900, Michael Paquier wrote: > > Sure, sorry for the confusion. By "we'd do nothing", I mean precirely > > "to take no specific action related to archive recovery and recovery > > parameters at the end

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-11-08 Thread Michael Paquier
On Thu, Nov 09, 2023 at 12:04:19PM +0900, Michael Paquier wrote: > Sure, sorry for the confusion. By "we'd do nothing", I mean precirely > "to take no specific action related to archive recovery and recovery > parameters at the end of recovery", meaning that a combination of > backup_label with no

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 01:16:58PM -0500, Robert Haas wrote: > On Tue, Oct 31, 2023 at 7:39 PM Michael Paquier wrote: As you're telling me, and I've considered that as an option as well, perhaps we should just consider the presence of a backup_label file with no .signal files as a s

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-11-08 Thread Robert Haas
On Tue, Oct 31, 2023 at 7:39 PM Michael Paquier wrote: > Point 7. of what you quote says to use one? True that this needs a > refresh, and perhaps a bit fat warning about the fact that these are > required if you want to fetch WAL from other sources than the local > pg_wal/. Perhaps there may be

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-11-05 Thread Michael Paquier
On Thu, Nov 02, 2023 at 11:03:35AM +0900, Kyotaro Horiguchi wrote: > At Wed, 1 Nov 2023 08:39:17 +0900, Michael Paquier > wrote in >> See in StartupXLOG(), around the comment "complain if we did not roll >> forward far enough to reach". This complains if archive recovery has >> been requested *

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-11-01 Thread Kyotaro Horiguchi
At Wed, 1 Nov 2023 08:39:17 +0900, Michael Paquier wrote in > See in StartupXLOG(), around the comment "complain if we did not roll > forward far enough to reach". This complains if archive recovery has > been requested *or* if we retrieved a backup end LSN from the > backup_label. Please note

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-31 Thread Michael Paquier
On Tue, Oct 31, 2023 at 08:28:07AM -0400, Robert Haas wrote: > On Mon, Oct 30, 2023 at 8:40 PM Michael Paquier wrote: >> As far as I know, there's one paragraph in the docs that implies this >> mode without giving an actual hint that this may be OK or not, so >> shrug: >> https://www.postgresql.or

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-31 Thread Robert Haas
On Mon, Oct 30, 2023 at 8:40 PM Michael Paquier wrote: > As far as I know, there's one paragraph in the docs that implies this > mode without giving an actual hint that this may be OK or not, so > shrug: > https://www.postgresql.org/docs/devel/continuous-archiving.html#BACKUP-TIPS > "As with base

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-30 Thread Michael Paquier
On Mon, Oct 30, 2023 at 12:47:41PM -0700, Andres Freund wrote: > I think the problem with these variables is that they're a really messy state > machine - something this patch doesn't meaningfully improve IMO. Okay. Yes, this is my root issue as well. We're at the stage where we should reduce th

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-30 Thread Michael Paquier
On Mon, Oct 30, 2023 at 10:32:28AM -0600, Roberto Mello wrote: > I realize the original use of "touch" is a valid shortcut for what I > suggest above, however that will be less clear for the not-so-un*x-inclined > users of Postgres, while for some it'll be downright confusing, IMHO. It > also provi

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-30 Thread Michael Paquier
On Mon, Oct 30, 2023 at 01:55:13PM -0400, Robert Haas wrote: > I would encourage some caution here. Thanks for chiming here. > In a vacuum, I'm in favor of this, and for the same reasons as you, > namely, that the huge pile of Booleans that we use to control recovery > is confusing, and it's diff

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-30 Thread Andres Freund
Hi, On 2023-10-30 16:08:50 +0900, Michael Paquier wrote: > From 26a8432fe3ab8426e7797d85d19b0fe69d3384c9 Mon Sep 17 00:00:00 2001 > From: Michael Paquier > Date: Mon, 30 Oct 2023 16:02:52 +0900 > Subject: [PATCH v4] Require recovery.signal or standby.signal when reading a > backup_file > > Histo

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-30 Thread Robert Haas
On Mon, Oct 30, 2023 at 3:09 AM Michael Paquier wrote: > I have been reviewing the patch, and applied portions of it as of > dc5bd388 and 1ffdc03c and they're quite independent pieces. After > that, the remaining bits of the patch to change the behavior is now > straight-forward. I have written

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-30 Thread Roberto Mello
On Mon, Oct 30, 2023 at 1:09 AM Michael Paquier wrote: > > I have been reviewing the patch, and applied portions of it as of > dc5bd388 and 1ffdc03c and they're quite independent pieces. After > that, the remaining bits of the patch to change the behavior is now > straight-forward. I have writt

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-30 Thread Michael Paquier
On Fri, Oct 27, 2023 at 09:31:10AM -0400, David Steele wrote: > That sounds like the right plan to me. Nice and simple. I'll tackle that in a separate thread with a patch registered for the upcoming CF of November. > I'm still +1 for the patch as it stands. I have been reviewing the patch, and a

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-27 Thread David Steele
On 10/27/23 03:22, Michael Paquier wrote: On Mon, Oct 16, 2023 at 02:54:35PM +0900, Michael Paquier wrote: On Sat, Oct 14, 2023 at 03:45:33PM -0400, David Steele wrote: On 9/28/23 19:59, Michael Paquier wrote: Another idea I had was to force the creation of recovery.signal by pg_basebackup eve

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-27 Thread Michael Paquier
On Mon, Oct 16, 2023 at 02:54:35PM +0900, Michael Paquier wrote: > On Sat, Oct 14, 2023 at 03:45:33PM -0400, David Steele wrote: >> On 9/28/23 19:59, Michael Paquier wrote: >>> Another idea I had was to force the creation of recovery.signal by >>> pg_basebackup even if -R is not used. All the repo

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-16 Thread Michael Paquier
On Mon, Oct 16, 2023 at 05:48:43PM +0200, Laurenz Albe wrote: > I don't have strong feelings either way. If you have backup_label > but no signal file, starting PostgreSQL may succeed (if the WAL > with the checkpoint happens to be in pg_wal) or it may fail with > an error message. There is no da

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-16 Thread Laurenz Albe
On Mon, 2023-10-16 at 14:54 +0900, Michael Paquier wrote: > Thanks for the review.  Yes, I am wondering if other people would > chime in here.  It doesn't feel like this has gathered enough > opinions. I don't have strong feelings either way. If you have backup_label but no signal file, starting

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-15 Thread Bowen Shi
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed It looks good to me. I have reviewed the code and tested the

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-15 Thread Michael Paquier
On Sat, Oct 14, 2023 at 03:45:33PM -0400, David Steele wrote: > On 9/28/23 19:59, Michael Paquier wrote: > OK, I have now reviewed and tested the patch and it looks good to me. I > stopped short of marking this RfC since there are other reviewers in the > mix. Thanks for the review. Yes, I am won

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-10-14 Thread David Steele
On 9/28/23 19:59, Michael Paquier wrote: On Thu, Sep 28, 2023 at 04:23:42PM -0400, David Steele wrote: So overall, +1 for Michael's patch, though I have only read through it and not tested it yet. Reviews, thoughts and opinions are welcome. OK, I have now reviewed and tested the patch and i

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-09-28 Thread Michael Paquier
On Thu, Sep 28, 2023 at 04:23:42PM -0400, David Steele wrote: > After some playing around, I find I agree with Michael on this, i.e. require > at least standby.signal when a backup_label is present. > > According to my testing, you can preserve the "independent server" > functionality by setting a

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-09-28 Thread David Steele
On 9/27/23 23:58, Kyotaro Horiguchi wrote: At Fri, 10 Mar 2023 15:59:04 +0900, Michael Paquier wrote in My apologies for the long message, but this deserves some attention, IMHO. So, any thoughts? Sorry for being late. However, I agree with David's concern regarding the unnecessary inconven

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-09-27 Thread Michael Paquier
On Thu, Sep 28, 2023 at 12:58:51PM +0900, Kyotaro Horiguchi wrote: > The attached is a quick mock-up, but providing an approximation of my > thoughts. (For example, end_of_backup_reached could potentially be > extended to the ArchiveRecoveryRequested case and we could simplify > the condition..) I

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-09-27 Thread Kyotaro Horiguchi
Sorry, it seems that I posted at the wrong position.. At Thu, 28 Sep 2023 12:58:51 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 10 Mar 2023 15:59:04 +0900, Michael Paquier > wrote in > > My apologies for the long message, but this deserves some attention, > > IMHO. > > > > So, any thou

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-09-27 Thread Kyotaro Horiguchi
At Fri, 10 Mar 2023 15:59:04 +0900, Michael Paquier wrote in > My apologies for the long message, but this deserves some attention, > IMHO. > > So, any thoughts? Sorry for being late. However, I agree with David's concern regarding the unnecessary inconvenience it introduces. I'd like to main

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-09-27 Thread Michael Paquier
On Thu, Sep 21, 2023 at 11:45:06AM +0800, Bowen Shi wrote: > First I encountered the problem " FATAL: could not find > recovery.signal or standby.signal when recovering with backup_label ", > then I deleted the backup_label file and started the instance > successfully. Doing that is equal to corr

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-09-20 Thread Bowen Shi
Thanks for the patch. I rerun the test in https://www.postgresql.org/message-id/flat/ZQtzcH2lvo8leXEr%40paquier.xyz#cc5ed83e0edc0b9a1c1305f08ff7a335 . We can discuss all the problems in this thread. First I encountered the problem " FATAL: could not find recovery.signal or standby.signal when re

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-07-19 Thread Michael Paquier
On Wed, Jul 19, 2023 at 11:21:17AM -0700, David Zhang wrote: > 1) simply start server from a base backup > > FATAL:  could not find recovery.signal or standby.signal when recovering > with backup_label > > HINT:  If you are restoring from a backup, touch > "/media/david/disk1/pg_backup1/recovery.

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-07-19 Thread David Zhang
On 2023-07-16 6:27 p.m., Michael Paquier wrote: Delete a backup_label from a fresh base backup can easily lead to data corruption, as the startup process would pick up as LSN to start recovery from the control file rather than the backup_label file. This would happen if a checkpoint updates the

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-07-16 Thread Michael Paquier
On Fri, Jul 14, 2023 at 01:32:49PM -0700, David Zhang wrote: > I believe before users can make a backup using pg_basebackup and then start > the backup as an independent Primary server for whatever reasons. Now, if > this is still allowed, then users need to be aware that the backup_label > must be

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-07-14 Thread David Zhang
I believe before users can make a backup using pg_basebackup and then start the backup as an independent Primary server for whatever reasons. Now, if this is still allowed, then users need to be aware that the backup_label must be manually deleted, otherwise, the backup won't be able to start a

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-03-12 Thread Michael Paquier
On Fri, Mar 10, 2023 at 03:59:04PM +0900, Michael Paquier wrote: > My apologies for the long message, but this deserves some attention, > IMHO. Note: A CF entry has been added as of [1], and I have added an item in the list of live issues on the open item page for 16. [1]: https://commitfest.post

Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-03-09 Thread Michael Paquier
Hi all, This is a follow-up of the point I have made a few weeks ago on this thread of pgsql-bugs about $subject: https://www.postgresql.org/message-id/Y/Q/17rpys7yg...@paquier.xyz https://www.postgresql.org/message-id/Y/v0c+3W89NBT/i...@paquier.xyz Here is a short summary of what I think is inco