On 05/04/2010 09:29 AM, Kyle McDonald wrote: > On 3/2/2010 10:15 AM, Kjetil Torgrim Homme wrote: >> "valrh...@gmail.com" <valrh...@gmail.com> writes: >> >> >>> I have been using DVDs for small backups here and there for a decade >>> now, and have a huge pile of several hundred. They have a lot of >>> overlapping content, so I was thinking of feeding the entire stack >>> into some sort of DVD autoloader, which would just read each disk, and >>> write its contents to a ZFS filesystem with dedup enabled. [...] That >>> would allow me to consolidate a few hundred CDs and DVDs onto probably >>> a terabyte or so, which could then be kept conveniently on a hard >>> drive and archived to tape. >>> >> it would be inconvenient to make a dedup copy on harddisk or tape, you >> could only do it as a ZFS filesystem or ZFS send stream. it's better to >> use a generic tool like hardlink(1), and just delete files afterwards >> with >> >> > There is a perl script floating around on the internet for years that > will convert copies of files on the same FS to hardlinks (sorry I don't > have the name handy). So you don't need ZFS. Once this is done you can > even recreate an ISO and burn it back to DVD (possibly merging hundreds > of CD's into one DVD (or BD!). The script can also delete the > duplicates, but there isn't much control over which one it keeps - for > backupsyou may realyl want to keep the earliest (or latest?) backup the > file appeared in.
I've used "Dirvish" http://www.dirvish.org/ and rsync to do just that...worked great! Scott > > Using ZFS Dedup is an interesting way of doing this. However archiving > the result may be hard. If you use different datasets (FS's) for each > backup, can you only send 1 dataset at a time (since you can only > snapshot on a dataset level? Won't that 'undo' the deduping? > > If you instead put all the backups on on data set, then the snapshot can > theoretically contain the dedpued data. I'm not clear on whether > 'send'ing it will preserve the deduping or not - or if it's up to the > receiving dataset to recognize matching blocks? If the dedup is in the > stream, then you may be able to write the stream to a DVD or BD. > > Still if you save enough space so that you can add the required level of > redundancy, you could just leave it on disk and chuck the DVD's. Not > sure I'd do that, but it might let me put the media in the basement, > instead of the closet, or on the desk next to me. > > -Kyle > > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
