Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Claudio Freire
On Tue, Mar 25, 2014 at 12:22 PM, Joshua D. Drake wrote: > On 03/25/2014 08:18 AM, Ilya Kosmodemiansky wrote: >> >> >> Joshua, >> >> that is really good point: an alternative is to use pg_basebackup >> through ssh tunnel with compression, but rsync is much simpler. > > > Or rsync over ssh. The ad

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Alan Hodgson
On Tuesday, March 25, 2014 03:48:07 PM Graeme B. Bell wrote: > Postgresql rsync backups require the DB to be shutdown during the 'second' > rsync. > > 1. rsync the DB onto the backup filesystem (produces e.g. 95-99.99% > consistent DB on the backup filesystem) 2. shut down the DB > 3. rsync the s

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Joshua D. Drake
On 03/25/2014 08:21 AM, Magnus Hagander wrote: I would say that's the one thing that rsync is *not*. pg_basebackup takes care of a lot of things under the hood. rsync is a lot more complicated, in particular in failure scenarios, since you have to manually deal with pg_start/stop_backup(). Ther

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Graeme B. Bell
Postgresql rsync backups require the DB to be shutdown during the 'second' rsync. 1. rsync the DB onto the backup filesystem (produces e.g. 95-99.99% consistent DB on the backup filesystem) 2. shut down the DB 3. rsync the shut down DB onto the backup filesystem(synchronises the last few

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
OK, agreed. Ive got your point;-) On Tue, Mar 25, 2014 at 4:40 PM, Magnus Hagander wrote: > Oh, I agree it's good that you should know both methods. I only disagree > with that the choice of rsync be made with the argument of simplicity. > Simplicity is one of the main reasons to choose the *othe

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Magnus Hagander
Oh, I agree it's good that you should know both methods. I only disagree with that the choice of rsync be made with the argument of simplicity. Simplicity is one of the main reasons to choose the *other* method (pg_basebackup), and the rsync method is for more advanced usecases. But it's definitely

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
Magnus, That is correct, but I'am afraid that such all-in-one functionality also hides from one how backup really works. Probably such sort of knowledge is so essential for a DBA, that it is better to learn both methods, at least to be able to choose correctly? But maybe it is a rhetorical questio

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
Joshua, On Tue, Mar 25, 2014 at 4:22 PM, Joshua D. Drake wrote: The advantage is that you can create backups that don't > have to be restored, just started. You can also use the differential > portions of rsync to do it multiple times a day without much issue. Are you sure, that it is a nice ide

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Joshua D. Drake
On 03/25/2014 08:18 AM, Ilya Kosmodemiansky wrote: Joshua, that is really good point: an alternative is to use pg_basebackup through ssh tunnel with compression, but rsync is much simpler. Or rsync over ssh. The advantage is that you can create backups that don't have to be restored, just s

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Magnus Hagander
I would say that's the one thing that rsync is *not*. pg_basebackup takes care of a lot of things under the hood. rsync is a lot more complicated, in particular in failure scenarios, since you have to manually deal with pg_start/stop_backup(). There are definitely reasons you'd prefer rsync over p

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
Joshua, that is really good point: an alternative is to use pg_basebackup through ssh tunnel with compression, but rsync is much simpler. On Tue, Mar 25, 2014 at 3:56 PM, Joshua D. Drake wrote: > > On 03/25/2014 05:05 AM, Claudio Freire wrote: >> >> >> On Tue, Mar 25, 2014 at 4:39 AM, David John

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread desmodemone
2014-03-25 15:56 GMT+01:00 Joshua D. Drake : > > On 03/25/2014 05:05 AM, Claudio Freire wrote: > >> >> On Tue, Mar 25, 2014 at 4:39 AM, David Johnston wrote: >> >>> Hai, Can anyone tell me the difference and performance between pgdump and pg_basebackup if I want to backup a large d

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Joshua D. Drake
On 03/25/2014 05:05 AM, Claudio Freire wrote: On Tue, Mar 25, 2014 at 4:39 AM, David Johnston wrote: Hai, Can anyone tell me the difference and performance between pgdump and pg_basebackup if I want to backup a large database. Honestly, Neither is particularly good at backing up large da

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Claudio Freire
On Tue, Mar 25, 2014 at 4:39 AM, David Johnston wrote: >> Hai, >> >> Can anyone tell me the difference and performance between pgdump and >> pg_basebackup if I want to backup a large database. >> >> Thanks > > Yes. And many of their words have been written down in the documentation in > a chapter

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread David Johnston
gianfranco caca wrote > Hai, > > Can anyone tell me the difference and performance between pgdump and > pg_basebackup if I want to backup a large database. > > Thanks Yes. And many of their words have been written down in the documentation in a chapter named "Backup and Restore". Do you have a

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
Yes, you need to set recovery_target_time in your recovery.conf while performing recovery (http://www.postgresql.org/docs/9.3/static/recovery-target-settings.html). That could be a tricky thing - depends on that exactly you need. All those transactions, which were not committed at given timestamp,

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread gianfranco caca
Hai ilya, Thanks for the respond. The database is estimated over 100gb and the workload will be high. Can we use a pg_basebackup with pitr to restore based on transaction time? Thanks On Tuesday, 25 March 2014, 15:13, Ilya Kosmodemiansky wrote: Hi gianfranco, How exactly large is your

Re: [PERFORM] pg_dump vs pg_basebackup

2014-03-25 Thread Ilya Kosmodemiansky
Hi gianfranco, How exactly large is your database and how heavy is a workload on it? Usually if you have more than ~200Gb, better to use pg_basebackup because pg_dump will take too long time. And please take in mind, that pg_dump makes dump, which is actually not the same thing as a backup. Bes

[PERFORM] pg_dump vs pg_basebackup

2014-03-24 Thread gianfranco caca
Hai, Can anyone tell me the difference and performance between pgdump and pg_basebackup if I want to backup a large database. Thanks