there are also some issues with using ssh as your transport that can slow down transfers over high-bandwidth links.
http://www.psc.edu/networking/projects/hpn-ssh/ You can tune TCP as well - see www.web100.org for lots and lots of info. Most of this stuff is written to deal with extremely high-bandwidth transfers but the prinicples will probably still apply in your situation - the ratio of bandwidth to latency is still the problem. Most of us just don't notice it on our 1.5Mb links. On Sun, Nov 07, 2004 at 07:08:07AM +1300, Jason Haar wrote: > Wayne Davison wrote: > > >On Fri, Oct 29, 2004 at 07:03:43PM -0700, Rudy Moore wrote: > > > > > >>I noticed in the feature list that rsync pipelines file transfers to > >>minimize latency - does this only affect transfer of large numbers of > >>files? > >> > >> > > > >It should be true of any transfer, but I'd estimate that the round-trip > >time gets factored in twice per transfer, so it would get amortized the > >more files you include in the transfer. > > > > > > > > I don't think "pipeline" means anything special here: rsync treats TCP > transfers the same way FTP or HTTP does - it just lets TCP stream it as > fast as it can. (compare with NFS or SMB which turns the data into > "blocks" and doesn't allow TCP to stream it - resulting in TERRIBLE > throughput on high-latency links compared with HTTP/rsync) > > >>Are there configuration steps that I can use to improve throughput? > >> > >> > High-latency means you'll never fill the pipe with a single > transmission. If you can, split the rsync job into sub-directories and > run separate rsync jobs on those at the same time. You might find (all > depends on your latency) that if you have a 1.5Mbs pipe, then you get a > max of 0.3Mbs for a single rsync job. But if you run 4 rsync jobs > simultaneously, you get 1.2Mbs. YMMV > > Jason > > -- > To unsubscribe or change options: > http://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html danno -- dan pritts - systems administrator - internet2 734/352-4953 office 734/834-7224 mobile -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
