On Mon, Jan 18, 2010 at 07:34:51PM +0100, Lassi Tuura wrote: > > Consider then, using a zpool-in-a-file as the file format, rather than > > zfs send streams. > > This is an interesting suggestion :-) > > Did I understand you correctly that once a slice is written, zfs > won't rewrite it? In other words, I can have an infinitely growing > pile of these slices, and once zfs fills one file up (or one > "raidz2" set of files), I flush it to tape/optical disk/whatever, > and zfs won't change it "ever after"? When I need more space, I just > add more slices, but old slices are effectively read-only?
No. The intention of this scheme is to have an ongoing backup pool, with a rolling set of historical snapshots collected together. I wouldn't suggest growing the pool indefinately, because you'll need an indefinite number of pool backing files online when it comes time for a distant-future restore. Even if you were never to delete old snapshots, zfs will still update some of the metadata in each file, or raidz set of files, whenever you make changes to your backup pool. You will need to sync each of these files to your backup media or offsite storage, to have a consistent pool to restore from. However, you can add new top-level vdevs (e.g. raidz set of files) as your backup pool starts to fill. Until you start to recycle space (by deleting old snapshots from the backup pool) there won't be much new data written to the original full files. Therefore, if your offsite replication of these files is efficient for incremental changes (e.g. rsync) then they will update quickly. The same will happen for changes within the existing files, of course, if they're not yet full or if you are recycling space. It's the same data, all that changes is how it's spread among the files. If you're writing to tapes or dvd's, you'll either need to rewrite the files completely every time, or layer some *other* incremental mechanism on top (perhaps your existing large-scale enterprise VTL, which you'd like to continue using, or are forced to continue using :). I don't mind rewriting tape media every time.. keep 2 or three sets and just roll between them each time. There are advantages to having a predictable cycle - knowing that you're going to need exactly N tapes this week and they will take T time to write out. You also get the benefit of all other D-to-D-to-T schemes, that this can be done at leisure asynchronously to host backup windows. For DVD's its a little more annoying, but at least the media is cheap. For these purposes, you can also consider removable hard disks as tapes. As I replace one generation of drives with the next, higher capacity, I intend for the previous ones can move to backup service. > And perhaps most importantly: has anyone actually done this for > their back-ups and has success stories on restore after media > failure? For me it's somewhere between concept and regular practice. It works, I've tinkered with it and done intermittent runs at archiving off the pool files and test imports and restores, but not written automation or made it as regular practice as I should. I've used it to drop backups of my opensolaris hosts onto enterprise backed-up servers, in non-solaris customer environments. There are some wrinkles, like you can't mount zpool files off read-only DVD media directly - you have to copy them to r/w scratch space because import wants to update metadata (is there an RFE for read-only import? I forget). This is mildly irritating at worst - copying one or two dvd's worth of files is easy, yet any more than this in your backup pool, you'll need to copy anyway for lack of many dvd readers at once. I also don't recommend files >1Gb in size for DVD media, due to iso9660 limitations. I haven't used UDF enough to say much about any limitations there. -- Dan.
pgpwXMKusfxKP.pgp
Description: PGP signature
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
