On 1/10/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
"Dick Davies" <[EMAIL PROTECTED]> wrote on 01/10/2007 05:26:45 AM: > On 08/01/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > I think that in addition to lzjb compression, squishing blocks that contain > > the same data would buy a lot of space for administrators working in many > > common workflows. > > This idea has occurred to me too - I think there are definite > advantages to 'block re-use'. > When you start talking about multiple similar zones, I suspect > substantial space savings could > be made - and if you can re-use that saved storage to provide > additional redundancy, everyone > would be happy.
My favorite uses come to mind (I have spent a fair amount of time wishing for this feature): 1) Zones that start out as ZFS clones will tend to diverge as the system patches. This will allow them to re-converge as the patches roll through multiple zones. 2) Environments where each person starts with the same code base (hg pull http://hg.intevation.org/mirrors/opensolaris.org/onnv-gate/) then build it producing substantially similar object files. 3) Disk-based backup systems (de-duplication is a buzz word here)
That issue has already come up in the thread, SHA256 is 2^128 for random, 2^80 for targeted collisions. That is pretty darn good, but it would also make sense to perform a rsync like secondary check on match using a dissimilar crypto hash. If we hit very unlikely chance that 2 blocks match both sha256 and whatever other secondary hash I think that block should be lost (act of god). =)
Reading the full block and doing a full comparison is very cheap (given the anticipated frequency) and makes you not have to explain that the file system has a 2^512 chance of silent data corruption. As slim of a chance as that is, ZFS promises to not corrupt my data and to tell on others that do. ZFS cannot break that promise. Mike -- Mike Gerdts http://mgerdts.blogspot.com/ _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
