Re: Something else about Redo Logs disappearing

2020-06-16 Thread Laurenz Albe
On Tue, 2020-06-16 at 00:28 +0200, Peter wrote: > On Mon, Jun 15, 2020 at 09:46:34PM +0200, Laurenz Albe wrote: > ! On Mon, 2020-06-15 at 19:00 +0200, Peter wrote: > ! > And that is one of a couple of likely pitfalls I perceived when > ! > looking at that new API. > ! > ! That is a property of my

Re: Something else about Redo Logs disappearing

2020-06-16 Thread Peter
On Mon, Jun 15, 2020 at 09:46:34PM +0200, Laurenz Albe wrote: ! On Mon, 2020-06-15 at 19:00 +0200, Peter wrote: ! > And that is one of a couple of likely pitfalls I perceived when ! > looking at that new API. ! ! That is a property of my scripts, *not* of the non-exclusive ! backup API... Then ho

Re: Something else about Redo Logs disappearing

2020-06-16 Thread Peter
On Sun, Jun 14, 2020 at 03:05:15PM +0200, Magnus Hagander wrote: ! > You can see that all the major attributes (scheduling, error-handling, ! > signalling, ...) of a WAL backup are substantially different to that ! > of any usual backup. ! ! > This is a different *Class* of backup object, therefo

Re: Something else about Redo Logs disappearing

2020-06-15 Thread Laurenz Albe
On Mon, 2020-06-15 at 19:00 +0200, Peter wrote: > And that is one of a couple of likely pitfalls I perceived when > looking at that new API. That is a property of my scripts, *not* of the non-exclusive backup API... > We could fix that, but that will then get more complicated - and > people will

Re: Something else about Redo Logs disappearing

2020-06-15 Thread Peter
On Mon, Jun 15, 2020 at 03:19:29PM +0200, Laurenz Albe wrote: ! On Mon, 2020-06-15 at 14:50 +0200, Peter wrote: ! > ! An example: ! > ! ! > ! - Backup #1 calls "pgpre.sh" ! > ! - Backup #1 starts copying files ! > ! - Backup #2 calls "pgpre.sh". ! > ! This will cancel the first backup. ! > ! - B

Re: Something else about Redo Logs disappearing

2020-06-15 Thread Laurenz Albe
On Mon, 2020-06-15 at 14:50 +0200, Peter wrote: > ! An example: > ! > ! - Backup #1 calls "pgpre.sh" > ! - Backup #1 starts copying files > ! - Backup #2 calls "pgpre.sh". > ! This will cancel the first backup. > ! - Backup #1 completes copying files. > ! - Backup #1 calls "pgpost.sh". > ! It

Re: Something else about Redo Logs disappearing

2020-06-15 Thread Peter
On Mon, Jun 15, 2020 at 11:44:33AM +0200, Laurenz Albe wrote: ! On Sat, 2020-06-13 at 19:48 +0200, Peter wrote: ! > ! > 4. If, by misconfiguration and/or operator error, the backup system ! > ! > happens to start a second backup. in parallel to the first, ! > ! > then do I correctly assume

Re: Something else about Redo Logs disappearing

2020-06-15 Thread Laurenz Albe
On Sat, 2020-06-13 at 19:48 +0200, Peter wrote: > ! > 4. If, by misconfiguration and/or operator error, the backup system > ! > happens to start a second backup. in parallel to the first, > ! > then do I correctly assume, both backups will be rendered > ! > inconsistent while this may

Re: Something else about Redo Logs disappearing

2020-06-14 Thread Magnus Hagander
On Sat, Jun 13, 2020 at 10:13 PM Peter wrote: > On Thu, Jun 11, 2020 at 10:35:13PM +0200, Magnus Hagander wrote: > ! > Okay. So lets behave like professional people and figure how that > ! > can be achieved: > ! > At first, we drop that WAL requirement, because with WAL archiving > ! > it is alre

Re: Something else about Redo Logs disappearing

2020-06-13 Thread Peter
On Thu, Jun 11, 2020 at 10:35:13PM +0200, Magnus Hagander wrote: ! > Okay. So lets behave like professional people and figure how that ! > can be achieved: ! > At first, we drop that WAL requirement, because with WAL archiving ! > it is already guaranteed that an unbroken chain of WAL is always ! >

Re: Something else about Redo Logs disappearing

2020-06-13 Thread Peter
On Sat, Jun 13, 2020 at 01:53:28PM +0200, Laurenz Albe wrote: ! > I've never seen anybody coding bash - it is strongly shunned in the ! > Berkeley community. ! ! Strange, but then I don't move in these circles. Never mind. ! > Some Questions: ! > 1. There are explicit error messages in loc-82

Re: Something else about Redo Logs disappearing

2020-06-13 Thread Laurenz Albe
On Sat, 2020-06-13 at 05:24 +0200, Peter wrote: > ! https://github.com/cybertec-postgresql/safe-backup > > Ah, thank You, very nice. > > I've never seen anybody coding bash - it is strongly shunned in the > Berkeley community. Strange, but then I don't move in these circles. > Some Questions:

Re: Something else about Redo Logs disappearing

2020-06-13 Thread Peter
On Thu, Jun 11, 2020 at 10:53:15PM +0200, Laurenz Albe wrote: ! On Thu, 2020-06-11 at 22:35 +0200, Magnus Hagander wrote: ! > I believe somebody around that time also wrote a set of bash scripts that can be used in a pre/post-backup-job combination with the current APIs. ! ! https://github.com/cy

Re: Something else about Redo Logs disappearing

2020-06-12 Thread Stephen Frost
Greetings, * Adrian Klaver (adrian.kla...@aklaver.com) wrote: > On 6/9/20 4:15 AM, Stephen Frost wrote: > >* Adrian Klaver (adrian.kla...@aklaver.com) wrote: > >>I use pg_backrest, but it does not look promising for running on BSD: > >>https://fluca1978.github.io/2019/03/04/pgbackrest_FreeBSD.html

Re: Something else about Redo Logs disappearing

2020-06-11 Thread Laurenz Albe
On Thu, 2020-06-11 at 22:35 +0200, Magnus Hagander wrote: > I believe somebody around that time also wrote a set of bash scripts that can > be used in a pre/post-backup-job combination with the current APIs. https://github.com/cybertec-postgresql/safe-backup Yours, Laurenz Albe -- Cybertec | ht

Re: Something else about Redo Logs disappearing

2020-06-11 Thread Magnus Hagander
On Thu, Jun 11, 2020 at 10:13 PM Peter wrote: > > Okay. So lets behave like professional people and figure how that > can be achieved: > At first, we drop that WAL requirement, because with WAL archiving > it is already guaranteed that an unbroken chain of WAL is always > present in the backup (e

Re: Something else about Redo Logs disappearing

2020-06-11 Thread Stephen Frost
Greetings, * Peter (p...@citylink.dinoex.sub.org) wrote: > On Wed, Jun 10, 2020 at 08:32:22AM -0400, Stephen Frost wrote: > ! > What repo?? I seem to have missed that at first glance. > ! > ! Yes, pgbackrest has a repo, like most other tools (though they call them > ! different things... pg_baseb

Re: Something else about Redo Logs disappearing

2020-06-11 Thread Peter
On Wed, Jun 10, 2020 at 08:32:22AM -0400, Stephen Frost wrote: ! > What repo?? I seem to have missed that at first glance. ! ! Yes, pgbackrest has a repo, like most other tools (though they call them ! different things... pg_basebackup has one though it's not really ! formal). ! ! > Are You inde

Re: Something else about Redo Logs disappearing

2020-06-11 Thread Peter
On Wed, Jun 10, 2020 at 01:10:36PM +0200, Magnus Hagander wrote: ! > Just having a look at their webpage, something seems to have been updated ! > recently, they now state that they have a new postgres adapter: ! > ! > https://www.bareos.com/en/company_news/postgres-plugin-en1.html ! > Enjoy readi

Re: Something else about Redo Logs disappearing

2020-06-10 Thread Stephen Frost
Greetings, * Peter (p...@citylink.dinoex.sub.org) wrote: > On Tue, Jun 09, 2020 at 03:42:48PM -0400, Stephen Frost wrote: > ! > And then 90% of the things offered here become superfluous, because > ! > they are already handled site-wide. And then you will have to > ! > consider integration of both

Re: Something else about Redo Logs disappearing

2020-06-10 Thread Peter
On Tue, Jun 09, 2020 at 03:42:48PM -0400, Stephen Frost wrote: ! > And then 90% of the things offered here become superfluous, because ! > they are already handled site-wide. And then you will have to ! > consider integration of both pieces - and that will most likely be ! > more work and more erro

Re: Something else about Redo Logs disappearing

2020-06-10 Thread Magnus Hagander
On Wed, Jun 10, 2020 at 1:53 AM Peter wrote: > On Tue, Jun 09, 2020 at 12:34:38PM -0700, Adrian Klaver wrote: > > ! The backup solution is? > > https://www.bareos.com/ > > ! Fine rant below. Go forth and work your wonders. > > I don't need to, anymore. I did that, for about 20 years - people > I

Re: Something else about Redo Logs disappearing

2020-06-09 Thread Adrian Klaver
On 6/9/20 4:35 PM, Peter wrote: On Tue, Jun 09, 2020 at 12:34:38PM -0700, Adrian Klaver wrote: And that can very well be done properly with an incremental filesystem backup software plus some 20 lines of shellscript. Read the caveats here: https://www.postgresql.org/docs/12/backup-file.ht

Re: Something else about Redo Logs disappearing

2020-06-09 Thread Stephen Frost
Greetings, * Peter (p...@citylink.dinoex.sub.org) wrote: > On Tue, Jun 09, 2020 at 03:42:48PM -0400, Stephen Frost wrote: > ! * Peter (p...@citylink.dinoex.sub.org) wrote: > ! > This professional backup solution also offers support for postgres. > ! > Sadly, it only covers postgres up to Rel.9, an

Re: Something else about Redo Logs disappearing

2020-06-09 Thread Peter
On Tue, Jun 09, 2020 at 12:34:38PM -0700, Adrian Klaver wrote: ! The backup solution is? https://www.bareos.com/ ! Fine rant below. Go forth and work your wonders. I don't need to, anymore. I did that, for about 20 years - people I used to work for as a consultant (major banks and insurance sho

Re: Something else about Redo Logs disappearing

2020-06-09 Thread Adrian Klaver
On 6/9/20 12:02 PM, Peter wrote: On Tue, Jun 09, 2020 at 01:27:20AM -0400, Tom Lane wrote: ! Adrian Klaver writes: ! > On 6/8/20 7:33 PM, Peter wrote: ! >> That "cp" is usually not synchronous. So there is the possibility ! >> that this command terminates successfully, and reports exitcode zero

Re: Something else about Redo Logs disappearing

2020-06-09 Thread Peter
On Tue, Jun 09, 2020 at 01:27:20AM -0400, Tom Lane wrote: ! Adrian Klaver writes: ! > On 6/8/20 7:33 PM, Peter wrote: ! >> That "cp" is usually not synchronous. So there is the possibility ! >> that this command terminates successfully, and reports exitcode zero ! >> back to the Postgres, and then

Re: Something else about Redo Logs disappearing

2020-06-09 Thread Stephen Frost
Greetings, * Peter (p...@citylink.dinoex.sub.org) wrote: > This professional backup solution also offers support for postgres. > Sadly, it only covers postgres up to Rel.9, and that piece of software > wasn't touched in the last 6 or 7 years. Then it certainly doesn't work with the changes in v12

Re: Something else about Redo Logs disappearing

2020-06-09 Thread Adrian Klaver
On 6/9/20 10:55 AM, Peter wrote: On Mon, Jun 08, 2020 at 09:21:47PM -0700, Adrian Klaver wrote: ! ! On 6/8/20 7:33 PM, Peter wrote: ! > ! > Actually, the affair had some good side: as usual I was checking ! > my own designs first and looking for flaws, and indeed I found one: ! > If you do copy o

Re: Something else about Redo Logs disappearing

2020-06-09 Thread Peter
On Mon, Jun 08, 2020 at 09:21:47PM -0700, Adrian Klaver wrote: ! ! On 6/8/20 7:33 PM, Peter wrote: ! > ! > Actually, the affair had some good side: as usual I was checking ! > my own designs first and looking for flaws, and indeed I found one: ! > If you do copy out the archive logs not directly

Re: Something else about Redo Logs disappearing

2020-06-09 Thread Stephen Frost
Greetings, * Adrian Klaver (adrian.kla...@aklaver.com) wrote: > On 6/9/20 4:15 AM, Stephen Frost wrote: > >* Adrian Klaver (adrian.kla...@aklaver.com) wrote: > >>I use pg_backrest, but it does not look promising for running on BSD: > >>https://fluca1978.github.io/2019/03/04/pgbackrest_FreeBSD.html

Re: Something else about Redo Logs disappearing

2020-06-09 Thread Adrian Klaver
On 6/9/20 4:15 AM, Stephen Frost wrote: Greetings, * Adrian Klaver (adrian.kla...@aklaver.com) wrote: I use pg_backrest, but it does not look promising for running on BSD: https://fluca1978.github.io/2019/03/04/pgbackrest_FreeBSD.html That's an unfortunately ancient post, really, considering

Re: Something else about Redo Logs disappearing

2020-06-09 Thread Stephen Frost
Greetings, * Adrian Klaver (adrian.kla...@aklaver.com) wrote: > I use pg_backrest, but it does not look promising for running on BSD: > https://fluca1978.github.io/2019/03/04/pgbackrest_FreeBSD.html That's an unfortunately ancient post, really, considering that pgbackrest has now been fully rewri

Re: Something else about Redo Logs disappearing

2020-06-08 Thread Tom Lane
Adrian Klaver writes: > On 6/8/20 7:33 PM, Peter wrote: >> That "cp" is usually not synchronous. So there is the possibility >> that this command terminates successfully, and reports exitcode zero >> back to the Postgres, and then the Postgres will consider that log >> being safely away. > Which

Re: Something else about Redo Logs disappearing

2020-06-08 Thread Adrian Klaver
On 6/8/20 7:33 PM, Peter wrote: Actually, the affair had some good side: as usual I was checking my own designs first and looking for flaws, and indeed I found one: If you do copy out the archive logs not directl

Re: Something else about Redo Logs disappearing

2020-06-08 Thread Peter
Actually, the affair had some good side: as usual I was checking my own designs first and looking for flaws, and indeed I found one: If you do copy out the archive logs not directly to ta