On Dec 5, 2009, at 11:03 AM, Mike Gerdts wrote:
On Sat, Dec 5, 2009 at 11:32 AM, Bob Friesenhahn
<bfrie...@simple.dallas.tx.us> wrote:
On Sat, 5 Dec 2009, dick hoogendijk wrote:
On Sat, 2009-12-05 at 09:22 -0600, Bob Friesenhahn wrote:
You can also stream into a gzip or lzop wrapper in order to
obtain the
benefit of incremental CRCs and some compression as well.
Can you give an example command line for this option please?
Something like
zfs send mysnapshot | gzip -c -3 > /somestorage/mysnap.gz
should work nicely. Zfs send sends to its standard output so it is
just a
matter of adding another filter program on its output. This could be
streamed over ssh or some other streaming network transfer protocol.
Later, you can do 'gzip -t mysnap.gz' on the machine where the
snapshot file
is stored to verify that it has not been corrupted in storage or
transfer.
lzop (not part of Solaris) is much faster than gzip but can be used
in a
similar way since it is patterned after gzip.
It seems as though a similar filter could be created to create and
inject an error correcting code into the stream. That is:
zfs send $snap | ecc -i > /somestorage/mysnap.ecc
ecc -o < /somestorage/mysnap | zfs receive ...
I'm not aware of an existing ecc program, but I can't imagine it
would be hard to create one. There seems to already be an
implementation of Reed-Solomon encoding in ON that could likely be
used as a starting point.
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/vdev_raidz.c
It all depends on the failure you want to protect against. If you
don't know the failure mode, you won't be very effective. For
example, to protect against a unrecoverable read on a single
disk sector, you need an ECC that can recover 512 bytes. It is
this thought process that led to the original RAID work (and is
one reason why nobody does RAID-2). By contrast, if you are
a working at the media level, then it is not uncommon to have
errors that affect a few contiguous bytes, and an ECC code can
be effective (AIUI, 40% of the bits on a modern HDD are not data).
-- richard
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
