Re: Use rsync's checksums to deduplicate across backups

2011-11-03 Thread Chris Dunlop
On Thu, Nov 03, 2011 at 09:34:53AM -0500, Alex Waite wrote: >> Not a direct answer, but this may do what you want: >> >>  http://gitweb.samba.org/?p=rsync-patches.git;a=blob;f=link-by-hash.diff >> >>  This patch adds the --link-by-hash=DIR option, which hard >> links received >>  files in a link fa

Re: Use rsync's checksums to deduplicate across backups

2011-11-03 Thread Carlos Carvalho
Alex Waite (alexq...@gmail.com) wrote on 2 November 2011 20:09: >Recently I learned that rsync does a checksum of every file >transferred. I thought it might be interesting to record the path and >checksum of each file in a table. On future backups, the checksum of >a file being backed up

Re: Use rsync's checksums to deduplicate across backups

2011-11-03 Thread Alex Waite
> Not a direct answer, but this may do what you want: > >  http://gitweb.samba.org/?p=rsync-patches.git;a=blob;f=link-by-hash.diff > >  This patch adds the --link-by-hash=DIR option, which hard links received >  files in a link farm arranged by MD4 file hash.  The result is that the > system >  wi

Re: Use rsync's checksums to deduplicate across backups

2011-11-03 Thread Alex Waite
> > Check out http://backuppc.sourceforge.net/, it's perl-based backup tool, > using rsync and doing exactly what you ask for. > I have looked at BackupPC before (and it is a nice piece of software), and it does hardlink across all backups, but believe it does its own checksum on top of what rs

Re: Use rsync's checksums to deduplicate across backups

2011-11-03 Thread Johannes Totz
On 03/11/2011 01:09, Alex Waite wrote: > I apologize if this has already been discussed before, but as of > yet I have been unable to find any info on the topic. > I have a very simple (and common) disk based backup system using > rsync, hard links, and a little bit of perl to glue it toget