On 13 Nov 2000, Jason Ozolins <[EMAIL PROTECTED]> wrote:
> Just a quick question: is the librsync contained within the rproxy
> source code meant to be tracking the development of the mainstream
> rsync, or is it a stripped-down thing meant only to support rproxy?
Here's a quick history:
In the beginning was rsync, which is a file transfer protocol.
At the moment I look after the day-to-day stuff, and tridge watches
the evolution.
rsync gave rise to Josh Macdonald's XDelta, which is optimized for the
case where old and new versions are on the same machine, and so it can
generate more efficient deltas.
tridge extracted the algorithm into librsync, which I renamed to
libhsync when I changed the wire format. The code currently checked
in as librsync is in my opinion not very good. It tries to make the
algorithm available at various levels to programs that would like to
use it, though the only user at the moment is rproxy. rsync doesn't
use libhsync -- possibly it never will, as we care enough about rsync
performance that tighter integration is justified. Well, if we were
starting from scratch it might be separated out, but it's not worth
doing it retrospectively now.
The problems with rsync at the moment are basically:
* Quirks of design ('triangular' TCP sockets, etc) tend to provoke
bugs in operating systems or remote shells.
* Useful features have been added in ad-hoc, and so the code is
fairly crufty in places.
* People still want even more features for special cases. To avoid
feature hell, my opinion is that we need a clean scripting or
plugin mechanism.
* rsync is optimized for transferring relatively small trees
(e.g. the rsync source tree) across slow links (e.g. 56kbps ppp).
This is fine and important, but people want to use it for different
situations (10GB, 100Mbps, 50 in parallel) where some design
decisions (e.g. traverse the whole tree up front) are no longer
optimal or even adequate.
rproxy uses the rsync algorithm to improve HTTP caching -- it's not
rsync-over-HTTP. I'm the lead developer for it, and it's in beta.
Completely unrelated to rproxy, sfr has added a small feature to
tunnel rsync through HTTP CONNECT proxies.
Therefore, some people at Linuxcare (primarily rusty, tridge and
myself) are looking at a ground-up rewrite with new code and a new
network protocol. (Of course we will have a fallback mode.) This
might be called rsync-3.0, or rsync-tng, or tsync, or something else.
This will likely be a more traditional client-server protocol,
somewhat similar to FTP and HTTP in that the client sends commands to
the server to put or get files. However, commands will be pipelined,
network-independent binary, and using only a single tcp connection.
In general we hope that there will be less special cases, and probably
that there will be less application-level intelligence in the server
and more in the client. This should be a firmer foundation for
building things such as
* implementations in different languages/platforms (Java, Win32
native, INTERCAL, ...)
* interactive rsync (like ftp(1))
* two-way rsync (controlled by the client, which could be automatic
or even have a GUI.)
* rsync as a transport for things such as CVS
Discussion about either feature requests or implementation ideas would
be very welcome. It's probably best to send them to the rsync mailing
list.
> The reason I ask is that I am thinking of extending Bob Edwards'
> rsync-based backup server architecture here at DCS, using a database to
> hold file metadata, doing binary deltas for history, and doing block
> compression on backed up data. This is a fair amount of stuff to
> change, and I was wondering which source base would be better to start
> with.
You might like to look at the XDelta work on XDFS and PCVS, or in the
longer term to work on rsync 3.0.
> If this question seems silly, just let me know and I'll look for my
> answer within the source code. :-)
--
Martin Pool, Linuxcare, Inc.
+61 2 6262 8990
[EMAIL PROTECTED], http://www.linuxcare.com/
Linuxcare. Support for the revolution.
PGP signature
