Re: [HACKERS] File based Incremental backup v8

2016-01-25 Thread Michael Paquier
On Tue, Jan 26, 2016 at 12:55 AM, David Fetter wrote: > On Tue, Mar 10, 2015 at 08:25:27AM +0900, Michael Paquier wrote: >> On Tue, Mar 10, 2015 at 1:50 AM, Robert Haas wrote: >> > I think there's absolutely no point in spending more time on this for >> > 9.5. At least 4 committers have looked a

Re: [HACKERS] File based Incremental backup v8

2016-01-25 Thread David Fetter
On Tue, Mar 10, 2015 at 08:25:27AM +0900, Michael Paquier wrote: > On Tue, Mar 10, 2015 at 1:50 AM, Robert Haas wrote: > > I think there's absolutely no point in spending more time on this for > > 9.5. At least 4 committers have looked at it and none of them are > > convinced by the current desig

Re: [HACKERS] File based Incremental backup v8

2015-03-09 Thread Michael Paquier
On Tue, Mar 10, 2015 at 1:50 AM, Robert Haas wrote: > I think there's absolutely no point in spending more time on this for > 9.5. At least 4 committers have looked at it and none of them are > convinced by the current design; feedback from almost half a year ago > hasn't been incorporated; obvio

Re: [HACKERS] File based Incremental backup v8

2015-03-09 Thread Robert Haas
On Sat, Mar 7, 2015 at 5:45 PM, Gabriele Bartolini wrote: >> By the way, unless I'm missing something, this patch only seems to >> include the code to construct an incremental backup, but no tools >> whatsoever to do anything useful with it once you've got it. > > As stated previously, Marco is wr

Re: [HACKERS] File based Incremental backup v8

2015-03-07 Thread Bruce Momjian
On Sun, Mar 8, 2015 at 09:26:38AM +1100, Gabriele Bartolini wrote: > Hi Bruce, > > 2015-03-08 5:37 GMT+11:00 Bruce Momjian : > >         Desirability -> Design -> Implement -> Test -> Review -> Commit > > This patch has continued in development without getting agreement on > its Des

Re: [HACKERS] File based Incremental backup v8

2015-03-07 Thread Gabriele Bartolini
Hi Robert, 2015-03-07 2:57 GMT+11:00 Robert Haas : > By the way, unless I'm missing something, this patch only seems to > include the code to construct an incremental backup, but no tools > whatsoever to do anything useful with it once you've got it. As stated previously, Marco is writing a too

Re: [HACKERS] File based Incremental backup v8

2015-03-07 Thread Gabriele Bartolini
Hi Bruce, 2015-03-08 5:37 GMT+11:00 Bruce Momjian : > > Desirability -> Design -> Implement -> Test -> Review -> Commit > > This patch has continued in development without getting agreement on > its Desirability or Design, meaning we are going to continue going back > to those points until

Re: [HACKERS] File based Incremental backup v8

2015-03-07 Thread Bruce Momjian
On Thu, Mar 5, 2015 at 11:10:08AM -0500, Robert Haas wrote: > But I agree with Fujii to the extent that I see little value in > committing this patch in the form proposed. Being smart enough to use > the LSN to identify changed blocks, but then sending the entirety of > every file anyway because

Re: [HACKERS] File based Incremental backup v8

2015-03-07 Thread Marco Nenciarini
Il 05/03/15 05:42, Bruce Momjian ha scritto: > On Thu, Mar 5, 2015 at 01:25:13PM +0900, Fujii Masao wrote: Yeah, it might make the situation better than today. But I'm afraid that many users might get disappointed about that behavior of an incremental backup after the release... >>>

Re: [HACKERS] File based Incremental backup v8

2015-03-06 Thread Robert Haas
On Fri, Mar 6, 2015 at 9:38 AM, Gabriele Bartolini wrote: > I believe the main point is to look at a user interface point of view. > If/When we switch to a block level incremental support, this will be > completely transparent to the end user, even if we start with a file-level > approach with LSN

Re: [HACKERS] File based Incremental backup v8

2015-03-06 Thread Gabriele Bartolini
Hi Robert, 2015-03-06 3:10 GMT+11:00 Robert Haas : > But I agree with Fujii to the extent that I see little value in > committing this patch in the form proposed. Being smart enough to use > the LSN to identify changed blocks, but then sending the entirety of > every file anyway because you don'

Re: [HACKERS] File based Incremental backup v8

2015-03-05 Thread Robert Haas
On Wed, Mar 4, 2015 at 11:42 PM, Bruce Momjian wrote: > On Thu, Mar 5, 2015 at 01:25:13PM +0900, Fujii Masao wrote: >> >> Yeah, it might make the situation better than today. But I'm afraid that >> >> many users might get disappointed about that behavior of an incremental >> >> backup after the r

Re: [HACKERS] File based Incremental backup v8

2015-03-04 Thread Bruce Momjian
On Thu, Mar 5, 2015 at 01:25:13PM +0900, Fujii Masao wrote: > >> Yeah, it might make the situation better than today. But I'm afraid that > >> many users might get disappointed about that behavior of an incremental > >> backup after the release... > > > > I don't get what do you mean here. Can you

Re: [HACKERS] File based Incremental backup v8

2015-03-04 Thread Fujii Masao
On Thu, Mar 5, 2015 at 1:59 AM, Marco Nenciarini wrote: > Hi Fujii, > > Il 03/03/15 11:48, Fujii Masao ha scritto: >> On Tue, Mar 3, 2015 at 12:36 AM, Marco Nenciarini >> wrote: >>> Il 02/03/15 14:21, Fujii Masao ha scritto: On Thu, Feb 12, 2015 at 10:50 PM, Marco Nenciarini wrote: >>>

Re: [HACKERS] File based Incremental backup v8

2015-03-04 Thread Marco Nenciarini
Hi Fujii, Il 03/03/15 11:48, Fujii Masao ha scritto: > On Tue, Mar 3, 2015 at 12:36 AM, Marco Nenciarini > wrote: >> Il 02/03/15 14:21, Fujii Masao ha scritto: >>> On Thu, Feb 12, 2015 at 10:50 PM, Marco Nenciarini >>> wrote: Hi, I've attached an updated version of the patch. >>>

Re: [HACKERS] File based Incremental backup v8

2015-03-03 Thread Fujii Masao
On Tue, Mar 3, 2015 at 12:36 AM, Marco Nenciarini wrote: > Il 02/03/15 14:21, Fujii Masao ha scritto: >> On Thu, Feb 12, 2015 at 10:50 PM, Marco Nenciarini >> wrote: >>> Hi, >>> >>> I've attached an updated version of the patch. >> >> basebackup.c:1565: warning: format '%lld' expects type 'long l

Re: [HACKERS] File based Incremental backup v8

2015-03-02 Thread Marco Nenciarini
Il 02/03/15 14:21, Fujii Masao ha scritto: > On Thu, Feb 12, 2015 at 10:50 PM, Marco Nenciarini > wrote: >> Hi, >> >> I've attached an updated version of the patch. > > basebackup.c:1565: warning: format '%lld' expects type 'long long > int', but argument 8 has type '__off_t' > basebackup.c:1565:

Re: [HACKERS] File based Incremental backup v8

2015-03-02 Thread Fujii Masao
On Thu, Feb 12, 2015 at 10:50 PM, Marco Nenciarini wrote: > Hi, > > I've attached an updated version of the patch. basebackup.c:1565: warning: format '%lld' expects type 'long long int', but argument 8 has type '__off_t' basebackup.c:1565: warning: format '%lld' expects type 'long long int', but

Re: [HACKERS] File based Incremental backup v8

2015-02-12 Thread Marco Nenciarini
Hi, I've attached an updated version of the patch. This fixes the issue on checksum calculation for segments after the first one. To solve it I've added an optional uint32 *segno argument to parse_filename_for_nontemp_relation, so I can know the segment number and calculate the block number corre

Re: [HACKERS] File based Incremental backup v8

2015-01-31 Thread Erik Rijkers
On Sat, January 31, 2015 15:14, Marco Nenciarini wrote: > 0001-public-parse_filename_for_nontemp_relation.patch > 0002-copydir-LSN-v2.patch > 0003-File-based-incremental-backup-v8.patch Hi, It looks like it only compiles with assert enabled. This is perhaps not yet really a problem at this stag

Re: [HACKERS] File based Incremental backup v8

2015-01-31 Thread Marco Nenciarini
Il 29/01/15 18:57, Robert Haas ha scritto: > On Thu, Jan 29, 2015 at 9:47 AM, Marco Nenciarini > wrote: >> The current implementation of copydir function is incompatible with LSN >> based incremental backups. The problem is that new files are created, >> but their blocks are still with the old LSN

Re: [HACKERS] File based Incremental backup v8

2015-01-29 Thread Andres Freund
On 2015-01-29 12:57:22 -0500, Robert Haas wrote: > The issues here are similar to those in > http://www.postgresql.org/message-id/20150120152819.gc24...@alap3.anarazel.de > - basically, I think we need to make CREATE DATABASE and ALTER > DATABASE .. SET TABLESPACE fully WAL-logged operations, or th

Re: [HACKERS] File based Incremental backup v8

2015-01-29 Thread Robert Haas
On Thu, Jan 29, 2015 at 9:47 AM, Marco Nenciarini wrote: > The current implementation of copydir function is incompatible with LSN > based incremental backups. The problem is that new files are created, > but their blocks are still with the old LSN, so they will not be backed > up because they are

[HACKERS] File based Incremental backup v8

2015-01-29 Thread Marco Nenciarini
The current implementation of copydir function is incompatible with LSN based incremental backups. The problem is that new files are created, but their blocks are still with the old LSN, so they will not be backed up because they are looking old enough. copydir function is used in: CREATE DATAB