On Sun, 2003-06-08 at 00:31, Jeff Kowalczyk wrote: > I'm interested in these very questions (librsync-rsync relationship, > remaining limitations of rsync, active prospects for ground-up rewrites), > Google searches for rsync info have proved a little too vague due to the > programs ubiquity. Much has certainly changed since this was written, > could some people with knowledge in these areas could update martin's > response for the state of rsync, June 2003? Thanks.
regarding librsync... It is still in sort-of-active development on SourceForge by a variety of developers... a new release is waiting in CVS for me to finally get around to releasing it, but I'm busy on a big contract at the moment so its currently on hold pending some more cygwin/win32 testing. It is in active use by projects like rdiff-backup. AFAIK, rproxy is pretty much dead, and the only version that exists depends on a very old version of libhsync. The closest thing to this available now is the http proxy "proof of concept" with xdelta, but it's radically different in many ways to the old rproxy (due to xdelta not using signatures). > On 13 Nov 2000, Jason Ozolins wrote: > http://lists.samba.org/pipermail/rsync/2000-November/003147.html > > Just a quick question: is the librsync contained within the rproxy > > source code meant to be tracking the development of the mainstream > > rsync, or is it a stripped-down thing meant only to support rproxy? > > On 13 Nov 2000, Martin Pool Responded: Here's a quick history: [...] > rsync gave rise to Josh Macdonald's XDelta, which is optimized for the > case where old and new versions are on the same machine, and so it can > generate more efficient deltas. xdelta is still under active development by Josh, and is evolving into a fancy versioning virtual file system... an ideal back-end for something like subversion. Josh tends to develop stuff with little fanfare, but his code tends to be _very_ clean. > tridge extracted the algorithm into librsync, which I renamed to libhsync > when I changed the wire format. The code currently checked in as librsync > is in my opinion not very good. It tries to make the algorithm available > at various levels to programs that would like to use it, though the only > user at the moment is rproxy. rsync doesn't use libhsync -- possibly it > never will, as we care enough about rsync performance that tighter > integration is justified. Well, if we were starting from scratch it might > be separated out, but it's not worth doing it retrospectively now. [...] This is largely still true, except libhsync changed back to librsync and now has its own project on SourceForge separate from the mostly defunct rproxy. librsync itself has no "wire format", being just a general purpose signature/delta/patch library implementing the rsync algorithm. The comments about rsync never using libhsync/librsync are still true for the foreseeable future. There are many things rsync includes that are still missing from librsync, and the rsync implementation is very tightly coupled, with many backwards compatibility issues. Even when librsync reaches the point of being as good or better than rsync at signature/delta/patch calculation, it would be a major task to "fit it into" rsync. rsync also has more active development, mostly in the form of incremental feature additions and the resulting "bugfix fire-fighting", all of which lead to an even more tangled implementation. Occasionally there are efforts to re-write and clean up sections of the code, but they are (rightly) regarded cautiously because of the breakage risk involved for little immediate gain. The librsync code in CVS is still largely "not very good". It is pretty messy and needs a good cleanup. The API is mostly OK though, and it _does_ work quite well, with no known bugs. I have some plans for a major cleanup and optimisation of the code based on my experiences with pysync. I have a patch submitted that I plan to commit after the next release that optimises and cleans up the delta calculation code quite a bit. The "next big thing" in delta calculation is probably going to be the vcdiff encoding format, which should allow a common delta format for various applications and supports "self-referencing delta's", which makes it capable of compression. According to the xdelta project this has already been implemented, and I'm keen to see Josh's code, as it could be used as the basis for a cleanup/replacement of at least the "patch" component of librsync. Possibly worth also mentioning is things like pysync which is a demonstration implementation of rsync and xdelta, as well as a wrapper for librsync. I'm kind of embarrassed though that at the moment rdiff-backup probably has a better python wrapper of librsync than pysync does. I believe there has also been some implementations of rsync in Perl (one that claims to talk to rsync, which is an amazing achievement), but I'm not up to date on those. I think someone has a Perl wrapper for librsync that was being used as a test bed for rsync 3 type development (superlifter?). For the future I can see continued support of the exising rsync code. I would also like to see librsync adopt vcdiff as it's delta format, and get a major cleanup, possibly by re-using some xdelta code. There are many common elements to the xdelta and rsync algorithms, and I see no reason why a single library couldn't support both (as pysync does). It would be nice if librsync and/or xdelta could become _the_ delta library. -- ---------------------------------------------------------------- Donovan Baarda http://minkirri.apana.org.au/~abo/ ---------------------------------------------------------------- -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
