On Sun, May 22, 2011 at 10:20 AM, Richard Elling <richard.ell...@gmail.com> wrote: > ZFS already tracks the blocks that have been written, and the time that > they were written. So we already know when something was writtem, though > that does not answer the question of whether the data was changed. I think > it is a pretty good bet that newly written data is different :-)
Not really. There's bp rewrite (assuming that ever ships, or gets implemented elsewhere), for example. >> Then, the filesystem should make this Merkle Tree available to >> applications through a simple query. > > Something like "zfs diff" ? That works within a filesystem. And zfs send/recv works when you have one filesystem faithfully tracking another. When you have two filesystems with similar contents, and the history of each is useless in deciding how to do a bi-directional synchronization, then you need a way to diff files that is not based on intra-filesystem history. The rsync algorithm is the best high-performance algorithm that we have for determining differences between files separated by a network. My proposal (back then, and Zooko's now) is to leverage work that the filesystem does anyways to implement a high-performance remote diff that is faster than rsync for the simple reason that some of the rsync algorithm essentially gets pre-computed. >> This would enable applications—without needing any further >> in-filesystem code—to perform a Merkle Tree sync, which would range >> from "noticeably more efficient" to "dramatically more efficient" than >> rsync or zfs send. :-) > > Since ZFS send already has an option to only send the changed blocks, > I disagree with your assertion that your solution will be "dramatically > more efficient" than zsf send. We already know zfs send is much more > efficient than rsync for large file systems. You missed Zooko's point completely. It might help to know that Zooko works on a project called Tahoe Least-Authority Filesystem, which is by nature distributed. Once you lose the constraints of not having a network or having uni-directional replication only, I think you'll get it. Or perhaps you'll argue that no one should ever need bi-di replication, that if one finds oneself wanting that then one has taken a wrong turn somewhere. Nico -- _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
