I subscribed to the list before I sent this. I've not seen either the email confirmation request or my mail to the list.
Wayne Davison wrote:
On Tue, Jun 08, 2004 at 07:37:32AM +0800, John wrote:
1. ADSL connexion at one end ot the other dropping for a while. rsync doesn't notice and mostly hangs. I have seen rsync at home still running but with no relevant files open.
There are two aspects of this: (1) Your remote shell should be setup to timeout appropriately (which is why rsync doesn't timeout by default) -- see your remote-shell's docs for how to do this; (2) you can tell rsync to timeout after a certain amount of inactivity (see --timeout).
I'm pretty sure that ssh times out properly: certainly it disconnects some time after my dialup line goes down.
I'd managed to overlook the --timeout option on rsync.
I'll look more closely for ssh hanging round next time it happens, but I'm skeptical.
1. No chance, I'd think, of it handling hard links properly if it can't see at least one of the other "copies." However, I could easily be wrong.2. rsync uses an enormous amount of virtual memory
Yes, it uses something like 80-100 bytes or so per file in the
transferred hierarchy (depending on options) plus a certain base amount
of memory. Your options are to (1) copy smaller sections of the
hierarchy at a time, (2) add more memory, or (3) help code something
better. This is one of the big areas that I've wanted to solve by
completely replacing the current rsync protocol with something better
(as I did in my rZync testbed protocol project a while back -- it
transfers the hierarchy incrementally, so it never has more than a
handful of directories in action at any one time). At some point I will
get back to working on an rsync-replacement project.
2. May require new boxes all round. Money may be an issue.
3. Yeah. My C skills a pretty rudimentary; I can barely read the stuff. Time for me to learn it was 30 or even years go, but I don't think it was invented then.
I've been known to have "unlikely" failures before:-) It could be the bloody Billion in the way. I know that if a TCP session is quiet "too long" the Billion forgets all about it.
3. rsync does not detect when its partner has vanished.
That seems unlikely unless the remote shell is still around. If the shell has terminated, the socket would return an EOF and rsync would exit. So, I'll assume (until shown otherwise) that this is a case of the remote shell still hanging around.
The fact I'm now trying a VPN using UDP should overcome that issue.
Well, I cut and pasted the commandline (with only a minor edit to disguise relevant sites). Here 'tis again:3a. It'd like to see rsync have the ability to retry in the case it's initiated the transfer.
There has been some talk of this recently. It doesn't seem like it would be too hard to do, but it's not trivial either. If someone wanted to code something up, I'd certainly appreciate the assistance. Or feel free to put an enhancement request into bugzilla. (BTW: has anyone heard from J.W. Schultz anytime recently? He seems to have dropped off the net without any explanation about 3 months ago -- I hope he's OK.)
4. [...] As best I can tell, rsync is transferring each copy fresh
instead of recognising the hard link before the transfer and getting
the destination rsync to make a new hard link.
This should not be the case if you use the -H option. (It also helps
to use 2.6.2 on both ends, as the memory-consumption was reduced
considerably from older releases.) If you're seeing a problem with
this, you should provide full details on what command you're running,
what versions you're using, and as small a test case as you can that
shows the problem.
rsync --recursive --links --hard-links --perms --owner --group --devices --times --sparse --one-file-system --rsh=/usr/bin/ssh --delete --delete-excluded --delete-after --max-delete=80 --relative --stats --numeric-ids
At the moment I'm running the standard latest versions for Woody (office) and Red Hat Linux 9 (home).
Acutally, Woody may be one fix behind latest, there as a new rsync out in the past few days that went in this morning. Nah, looks like the update went in before the last started, the rync binary it's got open isn't "deleted."
Office:
rsync version 2.5.6cvs protocol version 26
Home:
rsync version 2.5.7 protocol version 26
Is there a source of pre-built binaries? I didn't see it on the Rsync site.
..wayne..
-- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
