Re: archive modules loose ends

2024-04-02 Thread Nathan Bossart
On Tue, Mar 26, 2024 at 02:14:14PM -0500, Nathan Bossart wrote: > On Mon, Jan 15, 2024 at 08:50:25AM -0600, Nathan Bossart wrote: >> Thanks for reviewing. I've marked this as ready-for-committer, and I'm >> hoping to commit it in the near future. > > This one probably ought to go into v17, but I

Re: archive modules loose ends

2024-03-26 Thread Nathan Bossart
On Mon, Jan 15, 2024 at 08:50:25AM -0600, Nathan Bossart wrote: > Thanks for reviewing. I've marked this as ready-for-committer, and I'm > hoping to commit it in the near future. This one probably ought to go into v17, but I wanted to do one last call for feedback prior to committing. -- Nathan

Re: archive modules loose ends

2024-01-15 Thread Nathan Bossart
On Mon, Jan 15, 2024 at 12:21:44PM +, Li, Yong wrote: > The patch looks good to me. With the context explained in the thread, > the patch is easy to understand. > The patch serves as a refactoring which pulls up common memory management > and error handling concerns into the pgarch.c. With th

Re: archive modules loose ends

2024-01-15 Thread Li, Yong
> On Nov 29, 2023, at 01:18, Nathan Bossart wrote: > > External Email > > Here is a new version of the patch with feedback addressed. > > -- > Nathan Bossart > Amazon Web Services: https://aws.amazon.com Hi Nathan, The patch looks good to me. With the context explained in the thread, the

Re: archive modules loose ends

2023-11-28 Thread Nathan Bossart
Here is a new version of the patch with feedback addressed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 3cda5bb87c82738ad6f8a082ef5dfeb49cd51392 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 28 Nov 2023 11:17:13 -0600 Subject: [PATCH v3 1/1] archiver exception

Re: archive modules loose ends

2023-11-20 Thread Nathan Bossart
things I'd like to have answered... Sorry. No worries. I appreciate the review. >> I see two main options for dealing with this. One option is to simply have >> shell_archive (and any other archive modules out there) maintain its own >> memory context like basic_archive does.

Re: archive modules loose ends

2023-11-13 Thread Andres Freund
ached patch, we no longer call the archiving callback > while we're at the bottom of the exception stack, so ERRORs no longer get > bumped up to FATALs, and any palloc'd memory won't be freed. > > I see two main options for dealing with this. One option is to simply

Re: archive modules loose ends

2023-11-13 Thread Nathan Bossart
ing callback while we're at the bottom of the exception stack, so ERRORs no longer get bumped up to FATALs, and any palloc'd memory won't be freed. I see two main options for dealing with this. One option is to simply have shell_archive (and any other archive modules out there) ma

Re: Compilation error after redesign of the archive modules

2023-03-12 Thread Michael Paquier
On Fri, Mar 10, 2023 at 05:16:53PM +0900, Michael Paquier wrote: > On Fri, Mar 10, 2023 at 01:41:07PM +0530, Sravan Kumar wrote: >> I have attached a patch that fixes the problem. Can you please review >> if it makes sense to push this patch? > > Indeed, reproduced here. I'll fix that in a bit..

Re: Compilation error after redesign of the archive modules

2023-03-10 Thread Michael Paquier
On Fri, Mar 10, 2023 at 01:41:07PM +0530, Sravan Kumar wrote: > I have attached a patch that fixes the problem. Can you please review > if it makes sense to push this patch? Indeed, reproduced here. I'll fix that in a bit.. -- Michael signature.asc Description: PGP signature

Compilation error after redesign of the archive modules

2023-03-10 Thread Sravan Kumar
Hi, With the redesign of the archive modules: 35739b87dcfef9fc0186aca659f262746fecd778 - Redesign archive modules if we were to compile basic_archive module with USE_PGXS=1, we get compilation error: []$ make USE_PGXS=1 gcc -std=gnu99 -Wall -Wmissing-prototypes -Wpointer-arith

archive modules loose ends

2023-02-17 Thread Nathan Bossart
Andres recently reminded me of some loose ends in archive modules [0], so I'm starting a dedicated thread to address his feedback. The first one is the requirement that archive module authors create their own exception handlers if they want to make use of ERROR. Ideally, there would be a ha

Re: archive modules

2022-11-15 Thread Michael Paquier
On Tue, Nov 15, 2022 at 12:57:49PM -0800, Nathan Bossart wrote: > On Tue, Nov 15, 2022 at 06:14:25PM +0100, Alvaro Herrera wrote: >> Hmm, really? It seems to me that we will have two slightly different >> behaviors in 15 and master, which may be confusing later on. I think >> it'd be better to ma

Re: archive modules

2022-11-15 Thread Nathan Bossart
On Tue, Nov 15, 2022 at 06:14:25PM +0100, Alvaro Herrera wrote: > On 2022-Nov-15, Nathan Bossart wrote: > >> On Tue, Nov 15, 2022 at 10:31:44AM +0100, Peter Eisentraut wrote: > >> > The surrounding code has changed a bit between PG15 and master, so if we >> > wanted to backpatch this, we'd need a

Re: archive modules

2022-11-15 Thread Alvaro Herrera
On 2022-Nov-15, Nathan Bossart wrote: > On Tue, Nov 15, 2022 at 10:31:44AM +0100, Peter Eisentraut wrote: > > The surrounding code has changed a bit between PG15 and master, so if we > > wanted to backpatch this, we'd need another patch from you. However, at > > this point, I'm content to just l

Re: archive modules

2022-11-15 Thread Nathan Bossart
On Tue, Nov 15, 2022 at 10:31:44AM +0100, Peter Eisentraut wrote: > I have committed this to master. Thanks! > The surrounding code has changed a bit between PG15 and master, so if we > wanted to backpatch this, we'd need another patch from you. However, at > this point, I'm content to just leav

Re: archive modules

2022-11-15 Thread Peter Eisentraut
On 05.11.22 21:51, Nathan Bossart wrote: On Fri, Nov 04, 2022 at 12:05:26PM +0900, Ian Lawrence Barwick wrote: cfbot reports the patch no longer applies [1]. As CommitFest 2022-11 is currently underway, this would be an excellent time to update the patch. Indeed. Here is a new version of the

Re: archive modules

2022-11-07 Thread Nathan Bossart
On Mon, Nov 07, 2022 at 03:20:31PM +0900, Michael Paquier wrote: > On Sat, Nov 05, 2022 at 02:08:58PM -0700, Nathan Bossart wrote: >> Perhaps we could eventually move the archive_command functionality to a >> contrib module (i.e., "shell_archive") so that users must always set >> archive_library.

Re: archive modules

2022-11-06 Thread Michael Paquier
On Sat, Nov 05, 2022 at 02:08:58PM -0700, Nathan Bossart wrote: > Such a module could define a custom GUC that accepts a shell command. I > don't think we should overload the meaning of archive_command based on the > whims of whatever archive module is loaded. Besides the potential end-user > con

Re: archive modules

2022-11-05 Thread Nathan Bossart
On Mon, Oct 17, 2022 at 02:49:51PM +0900, Michael Paquier wrote: > And, by the way, this patch would prevent the existence of archive > modules that need to be loaded but *want* an archive_command with > what they want to achieve. That does not strike me as a good idea if > we wa

Re: archive modules

2022-11-05 Thread Nathan Bossart
On Fri, Nov 04, 2022 at 12:05:26PM +0900, Ian Lawrence Barwick wrote: > cfbot reports the patch no longer applies [1]. As CommitFest 2022-11 is > currently underway, this would be an excellent time to update the patch. Indeed. Here is a new version of the patch. -- Nathan Bossart Amazon Web Se

Re: archive modules

2022-11-03 Thread Ian Lawrence Barwick
2022年10月16日(日) 16:36 Bharath Rupireddy : > > On Sat, Oct 15, 2022 at 3:13 AM Nathan Bossart > wrote: > > > > On Fri, Oct 14, 2022 at 11:51:30AM -0700, Nathan Bossart wrote: > > > On Fri, Oct 14, 2022 at 12:10:18PM +0530, Bharath Rupireddy wrote: > > >> 2) I think we have a problem - set archive_m

Re: archive modules

2022-10-18 Thread Bharath Rupireddy
for the > same effect, as there would be no point in having pgarch.c do its > thing without archive_library or archive_command where a WARNING is > issued in the default case (shell_archive with no archive_command). If it's done as said above, the corresponding check_configured_cb(

Re: archive modules

2022-10-16 Thread Michael Paquier
e_library or archive_command where a WARNING is issued in the default case (shell_archive with no archive_command). And, by the way, this patch would prevent the existence of archive modules that need to be loaded but *want* an archive_command with what they want to achieve. That does not strike me as

Re: archive modules

2022-10-16 Thread Kyotaro Horiguchi
At Fri, 14 Oct 2022 14:42:56 -0700, Nathan Bossart wrote in > As promised... As the code written, when archive library is being added while archive command is already set, archiver first emits seemingly positive message "restarting archive process because of..", then errors out after the resata

Re: archive modules

2022-10-16 Thread Bharath Rupireddy
On Sat, Oct 15, 2022 at 3:13 AM Nathan Bossart wrote: > > On Fri, Oct 14, 2022 at 11:51:30AM -0700, Nathan Bossart wrote: > > On Fri, Oct 14, 2022 at 12:10:18PM +0530, Bharath Rupireddy wrote: > >> 2) I think we have a problem - set archive_mode and archive_library > >> and start the server, then

Re: archive modules

2022-10-14 Thread Nathan Bossart
On Fri, Oct 14, 2022 at 11:51:30AM -0700, Nathan Bossart wrote: > On Fri, Oct 14, 2022 at 12:10:18PM +0530, Bharath Rupireddy wrote: >> 2) I think we have a problem - set archive_mode and archive_library >> and start the server, then set archive_command, reload the conf, see >> [3] - the archiver n

Re: archive modules

2022-10-14 Thread Nathan Bossart
On Fri, Oct 14, 2022 at 12:10:18PM +0530, Bharath Rupireddy wrote: > +ereport(ERROR, > +(errcode(ERRCODE_INVALID_PARAMETER_VALUE), > + errmsg("both archive_command and archive_library > specified"), > + errdetail("Only one of archive_command,

Re: archive modules

2022-10-13 Thread Bharath Rupireddy
On Fri, Oct 14, 2022 at 6:00 AM Michael Paquier wrote: > > On Thu, Oct 13, 2022 at 02:53:38PM -0400, Tom Lane wrote: > > Yeah, it really does not work to use GUC hooks to enforce multi-variable > > constraints. We've learned that the hard way (more than once, if memory > > serves). > > 414c2fd is

Re: archive modules

2022-10-13 Thread Michael Paquier
On Thu, Oct 13, 2022 at 02:53:38PM -0400, Tom Lane wrote: > Yeah, it really does not work to use GUC hooks to enforce multi-variable > constraints. We've learned that the hard way (more than once, if memory > serves). 414c2fd is one of the most recent ones. Its thread is about the same thing. --

Re: archive modules

2022-10-13 Thread Tom Lane
Nathan Bossart writes: > On Thu, Oct 13, 2022 at 03:25:27PM +0530, Bharath Rupireddy wrote: >> The intent here looks reasonable to me. However, why should the user >> be able to set both archive_command and archive_library in the first >> place only to later fail in LoadArchiveLibrary() per the pa

Re: archive modules

2022-10-13 Thread Nathan Bossart
On Thu, Oct 13, 2022 at 03:25:27PM +0530, Bharath Rupireddy wrote: > The intent here looks reasonable to me. However, why should the user > be able to set both archive_command and archive_library in the first > place only to later fail in LoadArchiveLibrary() per the patch? IMO, > the check_hook()

Re: archive modules

2022-10-13 Thread Bharath Rupireddy
urations, no? FWIW, I'm attaching a small patch that uses check_hook(). > That looks like the right solution to me. > > Let's put that into PG 16, and maybe we can consider backpatching it. +1 to backpatch to PG 15 where the archive modules feature was introduced. -- Bharath

Re: archive modules

2022-10-10 Thread Peter Eisentraut
On 05.10.22 20:57, Nathan Bossart wrote: On Wed, Oct 05, 2022 at 07:55:58PM +0200, Peter Eisentraut wrote: Leaving archive_command empty is an intentional configuration choice. What we are talking about here is, arguably, a misconfiguration, so it should result in an error. Okay. What do you

Re: archive modules

2022-10-05 Thread Nathan Bossart
On Wed, Oct 05, 2022 at 07:55:58PM +0200, Peter Eisentraut wrote: > Leaving archive_command empty is an intentional configuration choice. > > What we are talking about here is, arguably, a misconfiguration, so it > should result in an error. Okay. What do you think about something like the attac

Re: archive modules

2022-10-05 Thread Peter Eisentraut
On 23.09.22 18:14, Nathan Bossart wrote: On Fri, Sep 23, 2022 at 05:58:42AM -0400, Peter Eisentraut wrote: On 15.09.22 00:27, Nathan Bossart wrote: Both archive_command and archive_library are PGC_SIGHUP, so IIUC that wouldn't be sufficient. I attached a quick sketch that seems to provide the

Re: archive modules

2022-09-23 Thread Nathan Bossart
On Fri, Sep 23, 2022 at 05:58:42AM -0400, Peter Eisentraut wrote: > On 15.09.22 00:27, Nathan Bossart wrote: >> Both archive_command and archive_library are PGC_SIGHUP, so IIUC that >> wouldn't be sufficient. I attached a quick sketch that seems to provide >> the desired behavior. It's nowhere ne

Re: archive modules

2022-09-23 Thread Peter Eisentraut
On 15.09.22 00:27, Nathan Bossart wrote: Both archive_command and archive_library are PGC_SIGHUP, so IIUC that wouldn't be sufficient. I attached a quick sketch that seems to provide the desired behavior. It's nowhere near committable yet, but it demonstrates what I'm thinking. What is the ef

Re: archive modules

2022-09-22 Thread Nathan Bossart
On Wed, Sep 14, 2022 at 09:33:46PM +0200, Peter Eisentraut wrote: > Another question on this feature: Currently, if archive_library is set, > archive_command is ignored. I think if both are set, it should be an error. > Compare for example what happens if you set multiple recovery_target_xxx > set

Re: archive modules

2022-09-22 Thread Peter Eisentraut
On 17.09.22 11:49, Peter Eisentraut wrote: On 14.09.22 23:09, Nathan Bossart wrote: On Wed, Sep 14, 2022 at 09:31:04PM +0200, Peter Eisentraut wrote: Here is a patch that addresses this. My intent was to present archive_command as the built-in archive library, but I can see how this might cau

Re: archive modules

2022-09-17 Thread Peter Eisentraut
On 14.09.22 23:09, Nathan Bossart wrote: On Wed, Sep 14, 2022 at 09:31:04PM +0200, Peter Eisentraut wrote: Here is a patch that addresses this. My intent was to present archive_command as the built-in archive library, but I can see how this might cause confusion, so this change seems reasonabl

Re: archive modules

2022-09-14 Thread Nathan Bossart
On Wed, Sep 14, 2022 at 06:12:09PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> On Wed, Sep 14, 2022 at 04:47:23PM -0400, Tom Lane wrote: >>> Yeah, the objection there is only to trying to enforce such >>> interrelationships in GUC hooks. In this case it seems to me that >>> we could easily

Re: archive modules

2022-09-14 Thread Tom Lane
Nathan Bossart writes: > On Wed, Sep 14, 2022 at 04:47:23PM -0400, Tom Lane wrote: >> Yeah, the objection there is only to trying to enforce such >> interrelationships in GUC hooks. In this case it seems to me that >> we could easily check and complain at the point where we're about >> to use the

Re: archive modules

2022-09-14 Thread Nathan Bossart
On Wed, Sep 14, 2022 at 04:47:23PM -0400, Tom Lane wrote: > Peter Eisentraut writes: >> On 14.09.22 22:03, Nathan Bossart wrote: >>> I originally did it this way, but changed it based on this feedback [0]. I >>> have no problem with the general idea, but the recovery_target_* logic does >>> have

Re: archive modules

2022-09-14 Thread Nathan Bossart
On Wed, Sep 14, 2022 at 09:31:04PM +0200, Peter Eisentraut wrote: > Here is a patch that addresses this. My intent was to present archive_command as the built-in archive library, but I can see how this might cause confusion, so this change seems reasonable to me. > + > +It is important tha

Re: archive modules

2022-09-14 Thread Tom Lane
Peter Eisentraut writes: > On 14.09.22 22:03, Nathan Bossart wrote: >> I originally did it this way, but changed it based on this feedback [0]. I >> have no problem with the general idea, but the recovery_target_* logic does >> have the following note: >> >> * XXX this code is broken by design.

Re: archive modules

2022-09-14 Thread Peter Eisentraut
On 14.09.22 22:03, Nathan Bossart wrote: On Wed, Sep 14, 2022 at 09:33:46PM +0200, Peter Eisentraut wrote: Another question on this feature: Currently, if archive_library is set, archive_command is ignored. I think if both are set, it should be an error. Compare for example what happens if you

Re: archive modules

2022-09-14 Thread Nathan Bossart
On Wed, Sep 14, 2022 at 09:33:46PM +0200, Peter Eisentraut wrote: > Another question on this feature: Currently, if archive_library is set, > archive_command is ignored. I think if both are set, it should be an error. > Compare for example what happens if you set multiple recovery_target_xxx > set

Re: archive modules

2022-09-14 Thread Peter Eisentraut
Another question on this feature: Currently, if archive_library is set, archive_command is ignored. I think if both are set, it should be an error. Compare for example what happens if you set multiple recovery_target_xxx settings. I don't think silently turning off one setting by setting ano

Re: archive modules

2022-09-14 Thread Peter Eisentraut
On 14.09.22 07:25, Michael Paquier wrote: removed or recycled until they are archived. If WAL archiving cannot keep up - with the pace that WAL is generated, or if archive_command + with the pace that WAL is generated, or if archive_library fails repeatedly, old WAL files will ac

Re: archive modules

2022-09-13 Thread Michael Paquier
On Wed, Sep 14, 2022 at 06:37:38AM +0200, Peter Eisentraut wrote: > I noticed that this patch has gone around and mostly purged mentions of > archive_command from the documentation and replaced them with > archive_library. I don't think this is helpful, since people still use > archive_command and

Re: archive modules

2022-09-13 Thread Peter Eisentraut
I noticed that this patch has gone around and mostly purged mentions of archive_command from the documentation and replaced them with archive_library. I don't think this is helpful, since people still use archive_command and will want to see what the documentation has to say about it. I suggest

Re: archive modules

2022-09-10 Thread Nathan Bossart
On Sat, Sep 10, 2022 at 04:44:16PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Of course. I've marked it as ready-for-committer. > > Pushed with a bit of additional wordsmithing. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: archive modules

2022-09-10 Thread Tom Lane
Nathan Bossart writes: > Of course. I've marked it as ready-for-committer. Pushed with a bit of additional wordsmithing. regards, tom lane

Re: archive modules

2022-08-30 Thread Nathan Bossart
On Tue, Aug 30, 2022 at 09:49:20AM +0200, Benoit Lobréau wrote: > Ok done, https://commitfest.postgresql.org/39/3856/ (is that fine if I > added you as a reviewer ?) Of course. I've marked it as ready-for-committer. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: archive modules

2022-08-30 Thread Benoit Lobréau
On Tue, Aug 30, 2022 at 12:46 AM Nathan Bossart wrote: > I would advise that you create a commitfest entry for your patch so that it > isn't forgotten. > Ok done, https://commitfest.postgresql.org/39/3856/ (is that fine if I added you as a reviewer ?) and thanks for the added links in the patch

Re: archive modules

2022-08-29 Thread Nathan Bossart
On Thu, Aug 25, 2022 at 01:06:00PM -0700, Nathan Bossart wrote: > On Thu, Aug 25, 2022 at 03:29:41PM +0200, talk to ben wrote: >> Here is a patch with the proposed wording. > > Here is the same patch with a couple more links. I would advise that you create a commitfest entry for your patch so tha

Re: archive modules

2022-08-25 Thread Nathan Bossart
ptions defined + by archive modules unless special + action is taken to load them into the backend process executing the query. -- 2.25.1

Re: archive modules

2022-08-25 Thread talk to ben
function). + For example, since the archive_library is only loaded by the archiver + process, this view will not display any customized options defined by + archive modules unless special action is taken to load them into the backend + process executing the query. -- 2.37.1

Re: archive modules

2022-08-24 Thread Nathan Bossart
to a user-defined C function). For example, since the archive_library is only loaded by the archiver process, this view will not display any customized options defined by archive modules unless special action is taken to load them into the backend process executin

Re: archive modules

2022-08-24 Thread talk to ben
Nathan Bossart writes: > On one hand, it seems like folks will commonly encounter this behavior with this > module, so this feels like a natural place for such a note. Yes, I looked there first. Would this addition to the pg_settings description be better ? From 5346a8a0451e222e6592baacb994e6a0

Re: archive modules

2022-08-23 Thread Tom Lane
Nathan Bossart writes: > I don't know if it makes sense to document this in basic_archive. On one > hand, it seems like folks will commonly encounter this behavior with this > module, so this feels like a natural place for such a note. But on the > other hand, this is generic behavior for any li

Re: archive modules

2022-08-23 Thread Nathan Bossart
On Tue, Aug 23, 2022 at 04:18:52PM +0200, talk to ben wrote: > --- a/doc/src/sgml/basic-archive.sgml > +++ b/doc/src/sgml/basic-archive.sgml > @@ -68,6 +68,19 @@ basic_archive.archive_directory = > '/path/to/archive/directory' > to any archiving still in progress, but users should use extra ca

Re: archive modules

2022-08-23 Thread talk to ben
Would this addition to the documentation be ok ? I hope the english is not too broken .. From 8ea8c21413eeac8fbd37527e64820cbdca3a5d7a Mon Sep 17 00:00:00 2001 From: benoit Date: Mon, 22 Aug 2022 12:00:46 +0200 Subject: [PATCH] basic_archive parameter visibility doc patch Module parameters are on

Re: archive modules

2022-07-07 Thread Nathan Bossart
On Thu, Jul 07, 2022 at 11:48:20AM -0400, Tom Lane wrote: > Alvaro Herrera writes: >> Well, this does sound unsatisfactory. I suppose one answer would be to >> load the module in all backends, in case the user wants to look at the >> value. But that would be wasteful. Maybe we should have a war

Re: archive modules

2022-07-07 Thread Tom Lane
Alvaro Herrera writes: > Well, this does sound unsatisfactory. I suppose one answer would be to > load the module in all backends, in case the user wants to look at the > value. But that would be wasteful. Maybe we should have a warning > about it in the docs -- tell people to LOAD the library

Re: archive modules

2022-07-07 Thread Alvaro Herrera
On 2022-Jul-07, talk to ben wrote: > The modified archive module parameters are visible in pg_file_settings. > They don't show up in \dconfig+, which I understand given the query used by > the meta command, but I find a little confusing from an end user POV. Well, this does sound unsatisfactory.

Re: archive modules

2022-07-07 Thread talk to ben
(Sorry for the spam Nathan) With the list in CC and additional information : The modified archive module parameters are visible in pg_file_settings. They don't show up in \dconfig+, which I understand given the query used by the meta command, but I find a little confusing from an end user POV.

Re: archive modules

2022-07-07 Thread talk to ben
y it shows up for SHOW > commands. > Thanks for the quick answer ! That's a little surprising at first but I understand better now. Will there be a facility to check archive_library gucs later on ? It might come in handy with more guc rich archive modules.

Re: archive modules

2022-07-06 Thread Nathan Bossart
On Wed, Jul 06, 2022 at 06:21:24PM +0200, talk to ben wrote: > I am not sure why, but I can't find "basic_archive.archive_directory" in > pg_settings the same way I would find for example : > "pg_stat_statements.max". > > [local]:5656 benoit@postgres=# SELECT count(*) FROM pg_settings WHERE name >

Re: archive modules

2022-07-06 Thread talk to ben
Hi, I am not sure why, but I can't find "basic_archive.archive_directory" in pg_settings the same way I would find for example : "pg_stat_statements.max". [local]:5656 benoit@postgres=# SELECT count(*) FROM pg_settings WHERE name = 'basic_archive.archive_directory'; count --- 0 (1 row)

Re: Typo in archive modules docs

2022-02-09 Thread Nathan Bossart
On Wed, Feb 09, 2022 at 03:04:34PM +0100, Daniel Gustafsson wrote: > Spotted a typo when reading the new archive modules documentation, will apply > the attached shortly to fix. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Typo in archive modules docs

2022-02-09 Thread Daniel Gustafsson
Spotted a typo when reading the new archive modules documentation, will apply the attached shortly to fix. -- Daniel Gustafsson https://vmware.com/ typo-recyling.diff Description: Binary data

Re: archive modules

2022-02-03 Thread Nathan Bossart
On Thu, Feb 03, 2022 at 04:45:52PM -0500, Robert Haas wrote: > On Thu, Feb 3, 2022 at 4:25 PM Nathan Bossart > wrote: >> 024_archive_recovery.pl seems to do something similar. Patch attached. > > Committed. I think this is mostly an issue for pg_regress tests, as > opposed to 024_archive_recove

Re: archive modules

2022-02-03 Thread Robert Haas
On Thu, Feb 3, 2022 at 4:25 PM Nathan Bossart wrote: > 024_archive_recovery.pl seems to do something similar. Patch attached. Committed. I think this is mostly an issue for pg_regress tests, as opposed to 024_archive_recovery.pl, which is a TAP test. Maybe I'm wrong about that, but it looks to m

Re: archive modules

2022-02-03 Thread Nathan Bossart
On Thu, Feb 03, 2022 at 04:15:30PM -0500, Robert Haas wrote: > On Thu, Feb 3, 2022 at 4:11 PM Nathan Bossart > wrote: >> On Thu, Feb 03, 2022 at 04:04:33PM -0500, Robert Haas wrote: >> > So apparently we need to either skip this test when wal_level=minimal, >> > or force a higher wal_level to be

Re: archive modules

2022-02-03 Thread Robert Haas
On Thu, Feb 3, 2022 at 4:11 PM Nathan Bossart wrote: > On Thu, Feb 03, 2022 at 04:04:33PM -0500, Robert Haas wrote: > > So apparently we need to either skip this test when wal_level=minimal, > > or force a higher wal_level to be used for this particular test. Not > > sure what the existing precede

Re: archive modules

2022-02-03 Thread Nathan Bossart
On Thu, Feb 03, 2022 at 04:04:33PM -0500, Robert Haas wrote: > So apparently we need to either skip this test when wal_level=minimal, > or force a higher wal_level to be used for this particular test. Not > sure what the existing precedents are, if any. The only precedent I've found so far is test

Re: archive modules

2022-02-03 Thread Robert Haas
On Thu, Feb 3, 2022 at 2:27 PM Nathan Bossart wrote: > On Thu, Feb 03, 2022 at 02:11:18PM -0500, Robert Haas wrote: > > Committed. I'm going to be 0% surprised if the buildfarm turns pretty > > colors, but I don't know how to know what it's going to be unhappy > > about except by trying it, so her

Re: archive modules

2022-02-03 Thread Nathan Bossart
On Thu, Feb 03, 2022 at 02:11:18PM -0500, Robert Haas wrote: > Committed. I'm going to be 0% surprised if the buildfarm turns pretty > colors, but I don't know how to know what it's going to be unhappy > about except by trying it, so here goes. Thanks! I'll keep an eye on the buildfarm and will s

Re: archive modules

2022-02-03 Thread Robert Haas
On Wed, Feb 2, 2022 at 5:44 PM Nathan Bossart wrote: > > I would suggest s/if it eventually/only when it/ > > Done. Committed. I'm going to be 0% surprised if the buildfarm turns pretty colors, but I don't know how to know what it's going to be unhappy about except by trying it, so here goes. --

Re: archive modules

2022-02-02 Thread Nathan Bossart
gt; even really need documentation. A one-line comment that doesn't really > say anything non-obvious seems like the worst amount. In my quest to write well-commented code, I sometimes overdo it. I adjusted these comments in the new revision. > + > + > + There ar

Re: archive modules

2022-02-02 Thread Robert Haas
On Mon, Jan 31, 2022 at 8:36 PM Nathan Bossart wrote: > If basic_archive is to be in contrib, we probably want to avoid restarting > the archiver every time the module ERRORs. I debated trying to add a > generic exception handler that all archive modules could use, but I suspect > ma

Re: archive modules

2022-01-31 Thread Nathan Bossart
t >>> as suggested, and added shutdown support for archive modules. >> >> cfbot was unhappy with v14, so here's another attempt. One other change I >> am pondering is surrounding pgarch_MainLoop() with PG_TRY/PG_FINALLY so >> that we can also call the shutdown c

Re: archive modules

2022-01-29 Thread Nathan Bossart
On Sat, Jan 29, 2022 at 04:31:48PM -0800, Nathan Bossart wrote: > On Sat, Jan 29, 2022 at 12:50:18PM -0800, Nathan Bossart wrote: >> Here is a new revision. I've moved basic_archive to contrib, hardened it >> as suggested, and added shutdown support for archive modules. &g

Re: archive modules

2022-01-29 Thread Nathan Bossart
On Sat, Jan 29, 2022 at 12:50:18PM -0800, Nathan Bossart wrote: > Here is a new revision. I've moved basic_archive to contrib, hardened it > as suggested, and added shutdown support for archive modules. cfbot was unhappy with v14, so here's another attempt. One other change I

Re: archive modules

2022-01-29 Thread Nathan Bossart
Here is a new revision. I've moved basic_archive to contrib, hardened it as suggested, and added shutdown support for archive modules. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From f62fea53b93ba7181dfe084b4100eba59eb82aaa Mon Sep 17 00:00:00 2001 From: Nathan Bossa

Re: archive modules

2022-01-28 Thread Nathan Bossart
On Fri, Jan 28, 2022 at 03:20:41PM -0500, Robert Haas wrote: > On Fri, Jan 28, 2022 at 3:01 PM Nathan Bossart > wrote: >> I discussed the two main deficiencies I'm aware of with basic_archive >> earlier [0]. The first one is the issue with "incovenient" server crashes >> (mentioned below). > >

Re: archive modules

2022-01-28 Thread Robert Haas
On Fri, Jan 28, 2022 at 3:01 PM Nathan Bossart wrote: > I discussed the two main deficiencies I'm aware of with basic_archive > earlier [0]. The first one is the issue with "incovenient" server crashes > (mentioned below). Seems easy enough to rectify, if it's just a matter of silently-succeed-i

Re: archive modules

2022-01-28 Thread Nathan Bossart
On Fri, Jan 28, 2022 at 02:06:50PM -0500, Robert Haas wrote: > I've committed 0001 now. I don't see anything particularly wrong with > the rest of this either, but here are a few comments: Thanks! > - I wonder whether it might be better to promote the basic archiving > module to contrib (as compa

Re: archive modules

2022-01-28 Thread Robert Haas
On Thu, Jan 13, 2022 at 2:38 PM Bossart, Nathan wrote: > Here is another rebase for cfbot. I've committed 0001 now. I don't see anything particularly wrong with the rest of this either, but here are a few comments: - I wonder whether it might be better to promote the basic archiving module to co

Re: archive modules

2021-12-15 Thread Bossart, Nathan
bother with at all for the first round of archive module support. I believe it is something that could be easily added later, although it might require archive modules to adjust the archiving callback to accept and return a list of files. IMO the archive modules infrastructure is still an im

Re: archive modules

2021-11-10 Thread Bossart, Nathan
On 11/10/21, 10:42 AM, "David Steele" wrote: > OK, I haven't had to go over the patch in detail so I didn't realize the > module was not backwards compatible. I'll have a closer look soon. It's backward-compatible in the sense that you'd be able to switch archive_library to "shell" to continue us

Re: archive modules

2021-11-10 Thread David Steele
On 11/10/21 1:22 PM, Bossart, Nathan wrote: On 11/10/21, 8:10 AM, "David Steele" wrote: I would prefer this module to be in core as our standard implementation and load by default in a vanilla install. Hm. I think I disagree with putting it in contrib and with making it the default archive

Re: archive modules

2021-11-10 Thread Bossart, Nathan
On 11/10/21, 8:10 AM, "David Steele" wrote: > On 11/7/21 1:04 AM, Fujii Masao wrote: >> It's helpful if you share how much this approach reduces >> the amount of overhead. > > FWIW we have a test for this in pgBackRest. Running > `archive_command=pgbackrest archive-push ...` 1000 times via system(

Re: archive modules

2021-11-10 Thread David Steele
On 11/7/21 1:04 AM, Fujii Masao wrote: On 2021/11/03 0:03, Bossart, Nathan wrote: On 11/1/21, 9:44 PM, "Fujii Masao" wrote: What is the main motivation of this patch? I was thinking that it's for parallelizing WAL archiving. But as far as I read the patch very briefly, WAL file name is still

Re: archive modules

2021-11-06 Thread Fujii Masao
On 2021/11/03 0:03, Bossart, Nathan wrote: On 11/1/21, 9:44 PM, "Fujii Masao" wrote: What is the main motivation of this patch? I was thinking that it's for parallelizing WAL archiving. But as far as I read the patch very briefly, WAL file name is still passed to the archive callback functio

Re: archive modules

2021-11-02 Thread Bossart, Nathan
ng. But in that's probably > not typical, and if it does happen well then that particular archive > module has to be preloaded. If you know that you have several archive > modules that you want to use, you can still preload them all if for > this or any other reason you want to

Re: archive modules

2021-11-02 Thread Robert Haas
archive module has to be preloaded. If you know that you have several archive modules that you want to use, you can still preload them all if for this or any other reason you want to do that. But in a lot of cases it's not going to be necessary. In other words, if we use hooks, then you'r

Re: archive modules

2021-11-02 Thread Bossart, Nathan
e >> PGC_POSTMASTER GUCs and to use RegisterBackgroundWorker(). If we >> think that switching archive modules without restarting is more >> important, I believe we will need to take on a few restrictions. > > I guess I'm failing to understand what the problem is. You can set > GUCs o

  1   2   >