On Wed, Mar 10, 2010 at 02:54:18PM +0100, Svein Skogen wrote: > Are there any good options for encapsulating/decapsulating a zfs send > stream inside FEC (Forward Error Correction)? This could prove very > useful both for backup purposes, and for long-haul transmissions.
I used par2 for this for some time, but it is slow / cpu intensive. There are windows implementations that are rather faster, but the whole codebase seems to be largely abandoned, especially the unix one. I switched to raidz pools in files, as I've described here previously. I was interested to look at zfec, which is used within Zooko's allmydata tahoe filesystem, as an allegedly much faster implementation. I couldn't get it to work, IIRC because some dependency modules were unreachable at the time. I ran into my lack of python snake-charming skills, and lack of time. You have reminded me to go back and look again, and either find that whatever issue was at fault last time was transient and now gone, or determine what it actually was and get it resolved. In case you want to: http://allmydata.org/trac/zfec > If there are any good options for simply piping the data trough, feel > free to point me in the right direction, because my google-searches > didn't give me any real clues. Maybe my google-fu isn't up to scratch. Most of the schemes/tools I've seen, including the ones above, don't work in streaming mode. That's not been a requirement in my usage (protection against bad dvd media in backup/archives) so far. I haven't really cared since I have to stage backup copies for other reasons anyway, so there's no benefit (to me) in streaming. The LDPC ones mentioned in the thread seem to be stream oriented, and I'd like to look at them some more. Note that for streaming over the network, you'd need to use something other than tcp/ssh as transport to gain any advantage. -- Dan.
pgpQga8qdi7Yn.pgp
Description: PGP signature
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
