David,
What OS is on each end?
This sounds like (maybe) classic "TCP buffers too small by default",
which limits the amount of data in-flight for an individual
TCP flow to min(senders_TCP_send_buf, receivers_TCP_recv_buf),
assuming a clean network path.
(If you're running current Linux, it should autotune those buffers
up to a max-configured-value. But many out-of-box max's are too
small, anyway).
In the old days, rsync clients had no way to ask for a specific TCP
buffer size.
So you were stuck with whatever is default for the OS.
Thankfully, rsync developers (Wayne) added a CLI option awhile ago, like:
rsync -av --sockopts=SO_SNDBUF=2000000,SO_RCVBUF=2000000 host::module
This requests ~2MB for send and receive buffer from the OS.
(If the OS has a smaller upper bound, it might silently not give you
what you ask for).
Need to do this sort of ask-for-large-buffers on both ends,
whether client or server/daemon.
If working properly, and if the OS is configured to allow what you
ask for, and if the network is reasonably clean (few losses),
and the on-path routers have reasonable (bw*delay) buffers
to help when there's an occasional loss,
you should be able to get very high single-flow rsync performance.
It's worked for me, anyway.
Not sure of the impact of in-line h/w-VPN's...
Let me know if I can help (perhaps unicast?).
Once it's resolved, it's be nice to post a note back to the
list on what the resolution is.
But it sounds suspiciously like TCP-buffer-limited, from what
you've said so far...
(although CA->germany 440/500 doesn't sound bad...)
Another diagnostic approach would be to verify that you can
get good single-flow performance with iperf (specifying reasonable buffers)
or similar.
If you can't get good performance with iperf, then you won't
be able to with single-flow rsync either...
If iperf is tripping over some loss-rate on the path, then you *could*
look at more aggressive TCP stacks - but let's save that for later.
Larry
--
At 5:01 PM -0700 8/10/06, David Halliday wrote:
Rsync speed seems to be effected more by latency than bandwidth, why is
this?
I have been happily rsyncing all over the world for some time, but today I
had cause to sync 40GB of data to all our remote sites causing me to look
closer at the statistics.
We effectively pull data using.
rsync -az --delete --force hqserver::filesystem /export/sync/filesystem
Our network is a 20Meg+ T3 at HQ and a T1/E1 at each remote site. All
running Hardware VPN over internet.
When rsync is the only network user I am only getting a fraction of the
available bandwith utilized. With the above command running at each site I
used 'show int' on the VPN router to get a breakdown of bandwidth use on a
site to site basis.
Total Outbound Usage 5354000 bits/sec (out of 20Meg) at California
From CA to Arizona 1071000bits/sec (Out of 1.5Meg)
From CA to Texas 838000bits/sec (Out of 1.5Meg)
From CA to Boston 672000bits/sec (Out of 1.5Meg)
From CA to France 429000bits/dec (Out of 2.0Meg)
From CA to UK 312000bits/sec (Out of 2.0Meg)
From CA to Germany 444000bits/dec (Out of 500K)
From CA to Korea 452000bits/dec (Out of 1.5Meg)
From CA to Taiwan 395000bits/dec (Out of 1.5Meg)
From CA to Singapore 340000bits/dec (Out of unknown)
Finally, when I run several bandwidth using apps (such as ftp, scp etc)
concurrently I can get close to line speed at any site all sites.
From this I conclude a single rsync at a site can not utilize all available
bandwidth. It is also interesting to note that the bandwidth the rsync is
able to utilize is directly proportional to the distance (latency) to the
remote site.
Now the question, Is there any way to optimize these transfers so that a
single rsync can use the entire bandwidth available?
I considered trying different block sizes but most references I found
suggested that I leave it to rsync to find the optimum size.
Dave.
--
To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
