Rsync speed seems to be effected more by latency than bandwidth, why is this?
I have been happily rsyncing all over the world for some time, but today I had cause to sync 40GB of data to all our remote sites causing me to look closer at the statistics. We effectively pull data using. rsync -az --delete --force hqserver::filesystem /export/sync/filesystem Our network is a 20Meg+ T3 at HQ and a T1/E1 at each remote site. All running Hardware VPN over internet. When rsync is the only network user I am only getting a fraction of the available bandwith utilized. With the above command running at each site I used 'show int' on the VPN router to get a breakdown of bandwidth use on a site to site basis. Total Outbound Usage 5354000 bits/sec (out of 20Meg) at California >From CA to Arizona 1071000bits/sec (Out of 1.5Meg) >From CA to Texas 838000bits/sec (Out of 1.5Meg) >From CA to Boston 672000bits/sec (Out of 1.5Meg) >From CA to France 429000bits/dec (Out of 2.0Meg) >From CA to UK 312000bits/sec (Out of 2.0Meg) >From CA to Germany 444000bits/dec (Out of 500K) >From CA to Korea 452000bits/dec (Out of 1.5Meg) >From CA to Taiwan 395000bits/dec (Out of 1.5Meg) >From CA to Singapore 340000bits/dec (Out of unknown) Finally, when I run several bandwidth using apps (such as ftp, scp etc) concurrently I can get close to line speed at any site all sites. >From this I conclude a single rsync at a site can not utilize all available bandwidth. It is also interesting to note that the bandwidth the rsync is able to utilize is directly proportional to the distance (latency) to the remote site. Now the question, Is there any way to optimize these transfers so that a single rsync can use the entire bandwidth available? I considered trying different block sizes but most references I found suggested that I leave it to rsync to find the optimum size. Dave. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
