Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-22 Thread Bruce Momjian
On Fri, Aug 22, 2014 at 12:00:19PM -0400, Robert Haas wrote: > On Wed, Aug 20, 2014 at 7:33 PM, Claudio Freire > wrote: > > On Wed, Aug 20, 2014 at 8:24 PM, Bruce Momjian wrote: > >> On Mon, Aug 18, 2014 at 04:05:07PM +0300, Heikki Linnakangas wrote: > >>> But more to the point, I thought the co

Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-22 Thread Robert Haas
On Wed, Aug 20, 2014 at 7:33 PM, Claudio Freire wrote: > On Wed, Aug 20, 2014 at 8:24 PM, Bruce Momjian wrote: >> On Mon, Aug 18, 2014 at 04:05:07PM +0300, Heikki Linnakangas wrote: >>> But more to the point, I thought the consensus was to use the >>> highest LSN of all the blocks in the file, no

Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-22 Thread Robert Haas
On Mon, Aug 18, 2014 at 4:55 PM, Heikki Linnakangas wrote: > You're not thinking evil enough ;-). Let's say that you have a table that > stores bank transfers. You can do a bank transfer to pay a merchant, get the > goods delivered to you, and then a second transfer to yourself with a > specially

Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-20 Thread Claudio Freire
On Wed, Aug 20, 2014 at 8:24 PM, Bruce Momjian wrote: > On Mon, Aug 18, 2014 at 04:05:07PM +0300, Heikki Linnakangas wrote: >> But more to the point, I thought the consensus was to use the >> highest LSN of all the blocks in the file, no? That's essentially >> free to calculate (if you have to rea

Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-20 Thread Bruce Momjian
On Mon, Aug 18, 2014 at 04:05:07PM +0300, Heikki Linnakangas wrote: > But more to the point, I thought the consensus was to use the > highest LSN of all the blocks in the file, no? That's essentially > free to calculate (if you have to read all the data anyway), and > isn't vulnerable to collisions

Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-20 Thread Heikki Linnakangas
I think this has had enough review for a WIP patch. I'm marking this as Returned with Feedback in the commitfest because: * should use LSNs instead of a md5 * this doesn't do anything useful on its own, hence would need to see the whole solution before committing * not clear how this would be e

Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-18 Thread Amit Kapila
On Mon, Aug 18, 2014 at 6:35 PM, Heikki Linnakangas wrote: > > On 08/18/2014 08:05 AM, Alvaro Herrera wrote: >> >> Marco Nenciarini wrote: >> >>> To calculate the md5 checksum I've used the md5 code present in pgcrypto >>> contrib as the code in src/include/libpq/md5.h is not suitable for large >>

Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-18 Thread Heikki Linnakangas
On 08/18/2014 07:33 PM, Alvaro Herrera wrote: Heikki Linnakangas wrote: On 08/18/2014 08:05 AM, Alvaro Herrera wrote: We already have the FNV checksum implementation in the backend -- can't we use that one for this and avoid messing with MD5? (I don't think we're looking for a cryptographic

Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-18 Thread Alvaro Herrera
Heikki Linnakangas wrote: > On 08/18/2014 08:05 AM, Alvaro Herrera wrote: > >We already have the FNV checksum implementation in the backend -- can't > >we use that one for this and avoid messing with MD5? > > > >(I don't think we're looking for a cryptographic hash here. Am I wrong?) > > Hmm. An

Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-18 Thread Arthur Silva
On Mon, Aug 18, 2014 at 10:05 AM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 08/18/2014 08:05 AM, Alvaro Herrera wrote: > >> Marco Nenciarini wrote: >> >> To calculate the md5 checksum I've used the md5 code present in pgcrypto >>> contrib as the code in src/include/libpq/md5.h is

Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-18 Thread Heikki Linnakangas
On 08/18/2014 08:05 AM, Alvaro Herrera wrote: Marco Nenciarini wrote: To calculate the md5 checksum I've used the md5 code present in pgcrypto contrib as the code in src/include/libpq/md5.h is not suitable for large files. Since a core feature cannot depend on a piece of contrib, I've moved the

Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-18 Thread Heikki Linnakangas
On 08/18/2014 03:04 AM, Marco Nenciarini wrote: Hi Hackers, This is the first piece of file level incremental backup support, as described on wiki page https://wiki.postgresql.org/wiki/Incremental_backup It is not yet complete, but I wish to share it on the list to receive comments and suggesti

Re: [HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-17 Thread Alvaro Herrera
Marco Nenciarini wrote: > To calculate the md5 checksum I've used the md5 code present in pgcrypto > contrib as the code in src/include/libpq/md5.h is not suitable for large > files. Since a core feature cannot depend on a piece of contrib, I've > moved the files > > contrib/pgcrypto/md5.c > cont

[HACKERS] [PATCH] Incremental backup: add backup profile to base backup

2014-08-17 Thread Marco Nenciarini
Hi Hackers, This is the first piece of file level incremental backup support, as described on wiki page https://wiki.postgresql.org/wiki/Incremental_backup It is not yet complete, but I wish to share it on the list to receive comments and suggestions. The point of the patch is adding an option t