On 11.07.2011 16:01, Donald Pearson wrote: > I am looking to do state-full resume of rsync transfers. > > My network environment is is an unreliable and slow satellite > infrastructure, and the files I need to send are approaching 10 gigs in > size. In this network environment often times links cannot be maintained > for more than a few minutes at a time. In this environment, bandwidth is at > a premium, which is why rsync was chosen as ideal for the job. > > The problem that I am encountering while using rsync in these conditions is > that the connection between the client and server will drop due to network > instability before rsync can transfer the entire file. > > Upon retries, rsync starts from the beginning. Re-checking data that has > already been sent, as well as re-building the checksum in it's entirety > every time. Eventually I reach an impasse where the frequency of link loss > prevents rsync from ever getting any new data to the destination. > > I've been reading through the various switches in the man pages to try to > find a combination that will work. My thinking was to use a combination of > --partial and --append. With the first attempt using the --partial switch, > and subsequent attempts using both --partial and --append. The idea being > rsync would build a new "partial" file, and be able to resume building that > file while making the assumption upon subsequent retries that the existing > partial file, however large it may be, was assembled correctly and does not > need to be checked. > > However in practice rsync does not work in this way.
I think you didn't wait for the target rsync to complete, if a connection breaks, you have 2 parts left hanging. The less visible target-side is the important one here. That rsync has to "complete" before you do another try. Depending on how your connection drops it MAY hang for some time. I don't remember if rsync does "the right thing" if you just kill it, or if you have to wait for it. In the latter case "--timeout" sounds like it can be used to expedite matters. And also --inplace, with or without --append, reads like it is what you want, if you can live with it's caveats. Bis denn -- Real Programmers consider "what you see is what you get" to be just as bad a concept in Text Editors as it is in women. No, the Real Programmer wants a "you asked for it, you got it" text editor -- complicated, cryptic, powerful, unforgiving, dangerous. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
