Re: EXTERNAL: Re: Netapp SnapCenter

2020-06-29 Thread Paul Förster
Hi Stephen, > On 29. Jun, 2020, at 17:29, Stephen Frost wrote: > > I typically recommend rather strongly against using anything that won't > allow you to update to keep pace with PG minor versions, considering > they include critical security and corruption fixes (in some cases, but > even if th

Re: EXTERNAL: Re: Netapp SnapCenter

2020-06-29 Thread Stephen Frost
Greetings, * Paul Förster (paul.foers...@gmail.com) wrote: > > On 29. Jun, 2020, at 16:43, Stephen Frost wrote: > > I certainly don't blame you, particularly given all the changes > > regarding how restore is done which went into v12- obviously anything > > that hasn't been updated since before v

Re: EXTERNAL: Re: Netapp SnapCenter

2020-06-29 Thread Paul Förster
Hi Stephen, > On 29. Jun, 2020, at 16:43, Stephen Frost wrote: > > On some database platforms it basically means "stop writing", in order > to allow a backup to be taken, but that's not what happens on PG and any > documentation about using PG's start/stop definitely shouldn't be > talking about

Re: EXTERNAL: Re: Netapp SnapCenter

2020-06-29 Thread Stephen Frost
Greetings, * Paul Förster (paul.foers...@gmail.com) wrote: > > On 29. Jun, 2020, at 15:32, Stephen Frost wrote: > > > > Presumably they mean 'quiesce', except that that *isn't* what PG's > > yes, sorry, "quiece" was a typo on my part. I never fully understood what > they mean with "quiesce" an

Re: EXTERNAL: Re: Netapp SnapCenter

2020-06-29 Thread Paul Förster
Hi Stephen, > On 29. Jun, 2020, at 15:32, Stephen Frost wrote: > > Presumably they mean 'quiesce', except that that *isn't* what PG's yes, sorry, "quiece" was a typo on my part. I never fully understood what they mean with "quiesce" anyway. But then, I'm not the storage specialist in out comp

Re: EXTERNAL: Re: Netapp SnapCenter

2020-06-29 Thread Stephen Frost
Greetings, * Paul Förster (paul.foers...@gmail.com) wrote: > > On 26. Jun, 2020, at 12:29, Magnus Hagander wrote: > > I believe NetApp does atomic snapshots across multiple volumes, if you have > > them in the same consistency group. (If you don't then you're definitely in > > for a world of pa

Re: Netapp SnapCenter

2020-06-26 Thread Paul Förster
Hi Stephen, > On 22. Jun, 2020, at 19:00, Stephen Frost wrote: > ... and all tablespaces are also on that volume. Basically, anything > that PG might ever write to needs to all be included in that atomic > write. it turns out that tablespaces can be stored on different volumes if you use consi

Re: EXTERNAL: Re: Netapp SnapCenter

2020-06-26 Thread Paul Förster
Hi Magnus, > On 26. Jun, 2020, at 12:29, Magnus Hagander wrote: > I believe NetApp does atomic snapshots across multiple volumes, if you have > them in the same consistency group. (If you don't then you're definitely in > for a world of pain if you ever have to restore) > > Snapshotting multip

Re: EXTERNAL: Re: Netapp SnapCenter

2020-06-26 Thread Magnus Hagander
On Thu, Jun 25, 2020 at 5:23 PM Paul Förster wrote: > Hi Ken, > > > On 25. Jun, 2020, at 17:15, Wolff, Ken L wrote: > > There's actually a lot of good stuff in that document about Postgres in > general. I'd be curious to hear what everyone thinks, though, and > specifically about what NetApp re

Re: EXTERNAL: Re: Netapp SnapCenter

2020-06-25 Thread David G. Johnston
On Thu, Jun 25, 2020 at 8:24 AM Paul Förster wrote: > Archived WAL is another thing, but PGDATA and pg_wal should IMHO always be > located on the same volume, along with tablespaces, if any. > My understanding that having such a setup (single volume) eases administration at the cost of performan

Re: EXTERNAL: Re: Netapp SnapCenter

2020-06-25 Thread Paul Förster
Hi Ken, > On 25. Jun, 2020, at 17:15, Wolff, Ken L wrote: > There's actually a lot of good stuff in that document about Postgres in > general. I'd be curious to hear what everyone thinks, though, and > specifically about what NetApp recommends in Section 3.3 about putting data > and WAL on se

RE: EXTERNAL: Re: Netapp SnapCenter

2020-06-25 Thread Wolff, Ken L
>> On 23. Jun, 2020, at 22:59, Wolff, Ken L wrote: >> >> https://www.netapp.com/us/media/tr-4770.pdf > I will check this out. Thank you very much. There's actually a lot of good stuff in that document about Postgres in general. I'd be curious to hear what everyone thinks, though, and specif

Re: Netapp SnapCenter

2020-06-24 Thread Paul Förster
Hi Wolf, > On 23. Jun, 2020, at 22:59, Wolff, Ken L wrote: > > https://www.netapp.com/us/media/tr-4770.pdf I will check this out. Thank you very much. Cheers, Paul

Re: Netapp SnapCenter

2020-06-23 Thread Wolff, Ken L
One last follow up on this thread. I don't remember this being mentioned, so wanted to share in case if might be of help. The following is NetApps's Best Practices for PostgreSQL on NetApp ONTAP storage: https://www.netapp.com/us/media/tr-4770.pdf

Re: Netapp SnapCenter

2020-06-22 Thread Stephen Frost
Greetings, * Paul Förster (paul.foers...@gmail.com) wrote: > > On 22. Jun, 2020, at 17:46, Wolff, Ken L wrote: > > So apologies if this is a stupid question but there's obviously been a lot > > of discussion on this issue. Was a consensus ever reached on the following? > > > > If a Postgres da

Re: Netapp SnapCenter

2020-06-22 Thread Paul Förster
Hi Ken, please don't put your answer at the top. > On 22. Jun, 2020, at 17:46, Wolff, Ken L wrote: > > So apologies if this is a stupid question but there's obviously been a lot of > discussion on this issue. Was a consensus ever reached on the following? > > If a Postgres database (both dat

Re: Netapp SnapCenter

2020-06-22 Thread Wolff, Ken L
in-time at which that snapshot was taken? Thanks much. Ken -Original Message- From: Stephen Frost Sent: Monday, June 22, 2020 7:06 AM To: Paul Förster Cc: Magnus Hagander ; Wolff, Ken L (US) ; pgsql-general Subject: EXTERNAL: Re: Netapp SnapCenter Greetings, * Paul Förster (

Re: Netapp SnapCenter

2020-06-22 Thread Stephen Frost
Greetings, * Paul Förster (paul.foers...@gmail.com) wrote: > > On 22. Jun, 2020, at 13:08, Magnus Hagander wrote: > > It does not work off *that* base backup. But if you start from the *prior* > > be backup (one that did complete with a successful pg_stop_backup) then you > > can still use the

Re: Netapp SnapCenter

2020-06-22 Thread Paul Förster
Hi Magnus, > On 22. Jun, 2020, at 13:08, Magnus Hagander wrote: > > It does not work off *that* base backup. But if you start from the *prior* be > backup (one that did complete with a successful pg_stop_backup) then you can > still use the archived wal to recover to any point in time. ok, t

Re: Netapp SnapCenter

2020-06-22 Thread Paul Förster
Hi Peter, > On 22. Jun, 2020, at 13:01, Peter J. Holzer wrote: > Restore the previous backup and replay WALs from there (that assumes of > course that you are archiving WALs continuously, but if you don't, you > can't do PITR in general, so if you have that requirement you are doing > it). ok, t

Re: Netapp SnapCenter

2020-06-22 Thread Magnus Hagander
On Mon, Jun 22, 2020 at 8:02 AM Paul Förster wrote: > Hi Stephen, > > > On 22. Jun, 2020, at 07:36, Stephen Frost wrote: > > That's not the only case that I, at least, have heard of- folks aren't > > really very happy with their backups fail when they could have just as > > well completed, even

Re: Netapp SnapCenter

2020-06-22 Thread Peter J. Holzer
On 2020-06-22 08:02:06 +0200, Paul Förster wrote: > > On 22. Jun, 2020, at 07:36, Stephen Frost wrote: > > The thing about this is though that the new API avoids *other* issues, > > like what happens if the system crashes during a backup (which is an > > entirely common thing that happens, conside

Re: Netapp SnapCenter

2020-06-21 Thread Paul Förster
Hi Stephen, > On 22. Jun, 2020, at 07:36, Stephen Frost wrote: > That's not the only case that I, at least, have heard of- folks aren't > really very happy with their backups fail when they could have just as > well completed, even if they're overlapping. Sure, it's better if > backups are sched

Re: Netapp SnapCenter

2020-06-21 Thread Stephen Frost
Greetings, * Paul Förster (paul.foers...@gmail.com) wrote: > > On 21. Jun, 2020, at 21:35, Magnus Hagander wrote: > > One not uncommon case is for example being able to provision a new replica > > while a backup is running. Since replicas are provisioned starting off a > > base backup, being ab

Re: Netapp SnapCenter

2020-06-21 Thread Paul Förster
Hi Magnus, > On 21. Jun, 2020, at 21:35, Magnus Hagander wrote: > One not uncommon case is for example being able to provision a new replica > while a backup is running. Since replicas are provisioned starting off a base > backup, being able to run that concurrently is very useful. Especially i

Re: Netapp SnapCenter

2020-06-21 Thread Magnus Hagander
On Sat, Jun 20, 2020 at 9:04 AM Paul Förster wrote: > Hi Stephen, > > > On 19. Jun, 2020, at 18:02, Stephen Frost wrote: > > When it comes to 'backup mode', it's actually the case that there can be > > multiple backups running concurrently because there isn't actually a > > single 'cluster wide

Re: Netapp SnapCenter

2020-06-20 Thread Paul Förster
Hi Stephen, > On 19. Jun, 2020, at 18:02, Stephen Frost wrote: > When it comes to 'backup mode', it's actually the case that there can be > multiple backups running concurrently because there isn't actually a > single 'cluster wide backup mode', really. this is what I don't understand. Why would

Re: Netapp SnapCenter

2020-06-19 Thread Stephen Frost
Greetings, * Paul Förster (paul.foers...@gmail.com) wrote: > When I wrote our backup mode script I read the deprecation note about the > exclusive mode backup. This is why I decided to go with non-exclusive to be > ready when exclusive backup mode is finally removed some day. Yet, I don't > see

Re: Netapp SnapCenter

2020-06-18 Thread Paul Förster
Hi Ron, > On 18. Jun, 2020, at 21:30, Ron wrote: > But it's trivial to make pg_xlog a separate mount point. technically yes, but that would mean it would go to a different volume and hence destroy atomicity of the Netapp snapshots. Also, you'd have a huge administrative barrier against such th

Re: Netapp SnapCenter

2020-06-18 Thread Paul Förster
Hi Stephen, > On 18. Jun, 2020, at 21:26, Stephen Frost wrote: > If the entire database, all tablespaces, and pg_wal, are on the same > volume and the snapshot of the volume is atomic, then you don't actually > need to go through the start/stop backup- a snapshot being restored will > look just l

Re: Netapp SnapCenter

2020-06-18 Thread Ron
On 6/18/20 11:49 AM, Paul Förster wrote: Hi Magnus, On 18. Jun, 2020, at 16:19, Magnus Hagander wrote: I don't know specifically about SnapCenter, but for snapshots in general, it does require backup mode *unless* all your data is on the same disk and you have an atomic snapshot across that

Re: Netapp SnapCenter

2020-06-18 Thread Stephen Frost
Greetings, * Paul Förster (paul.foers...@gmail.com) wrote: > > On 18. Jun, 2020, at 16:19, Magnus Hagander wrote: > > I don't know specifically about SnapCenter, but for snapshots in general, > > it does require backup mode *unless* all your data is on the same disk and > > you have an atomic s

Re: Netapp SnapCenter

2020-06-18 Thread Paul Förster
Hi Magnus, > On 18. Jun, 2020, at 16:19, Magnus Hagander wrote: > I don't know specifically about SnapCenter, but for snapshots in general, it > does require backup mode *unless* all your data is on the same disk and you > have an atomic snapshot across that disk (in theory it can be on differe

Re: Netapp SnapCenter

2020-06-18 Thread Magnus Hagander
On Thu, Jun 18, 2020 at 4:07 PM Paul Förster wrote: > Hi Ken, > > > On 18. Jun, 2020, at 15:56, Wolff, Ken L wrote: > > PostgreSQL doesn’t need to be in a special mode for backups to work. > > this is curious. Doesn't the PostgreSQL cluster need to be set to backup > mode to use SnapCenter? > I

Re: Netapp SnapCenter

2020-06-18 Thread Paul Förster
Hi Ken, > On 18. Jun, 2020, at 15:56, Wolff, Ken L wrote: > PostgreSQL doesn’t need to be in a special mode for backups to work. this is curious. Doesn't the PostgreSQL cluster need to be set to backup mode to use SnapCenter? The problem is, one can't test that and get a reliable answer, becau

RE: Netapp SnapCenter

2020-06-18 Thread Wolff, Ken L
We got the following from one of our NetApp contacts regarding a similar question. I'd be interested to hear if anyone feels the steps mentioned are not accurate: There is presently no NetApp supported plugin for PostgreSQL for SnapCenter. There is a community support plugin for PostgreSQL,