On Mon, 2004-01-12 at 22:15, jw schultz wrote: > On Mon, Jan 12, 2004 at 06:22:04PM -0500, Andrew Boyce-Lewis wrote: > > Hi, I am running rsync version 2.5.7 (stock distro on redhat linux, ES > > and 9) to rsync a directory with ~300k files in it from a machine on a > > 10Mbit internet tap to a machine with a 100Mbit internet tap. The > > problem is that I am only getting about 500Kbps during the transfer. I > > have tested link speed by scping a file between the two systems, > > performance was approximately 7.5Mbit/sec. Ping time between the two > > locations is about 2.5 ms (only 20 miles apart) > > > > This is obviously a problem. I have also tried rsyncing just a few files > > It is? Rsync and scp do two completely different things and > their performance is not comparable.
Not comparable, but seriously... 62 KB/sec vs 960KB/sec? I have done similar operations over gig-e with much better performance, on almost exactly the same hardware. The transfers that I am having rsync do are of whole, large files (say 5 - 15 MB each), so I would think that scp and rsync (in this case) would actually be somewhat comparable. > > > (say 400) to rule out problems related to huge file lists, however I had > > the same problem. > > > > Is this a known problem? Am I missing something really obvious? > > > > I have tried testing drive performance in each machine and have found it > > to be acceptable. > > But how is the filesystem performance at doing what rsync > does to it? Raw sequential throughput is irrelevant. Well, with out good hard data to back this up, the drives at both ends are ultra 160 scsi raids, and hdparm suggests about 40MB/sec write (yes, I know hdparm is a bad benchmark). I have seen no signs of any disk performance problems in other applications. > > > The line that I am using to do the rsync is the following: > > > > rsync -avvu --exclude-from=/root/sync_list.exclude --delete -e ssh > > --stats /data/ dest:/data/ > > Try --whole-file; if that improves things you are disk bound. > > Drop to just one -v; -vv is for debugging and adversely > affects network buffering. I will give both a try this evening. > > It probably isn't your exclude file but a large exclude file > would add overhead. the exclude file has only two entries. > > 2.5.7 is a security release with minimal changes from 2.5.6 > which is almost a year old. There have been a lot of > changes since then most of which are in 2.6.0. Since 2.6.0 > there have been a number of changes related to performance > including one that dramatically improves the network > performance. All of this you would have known if you had > read the list archives or checked NEWS. I did read the archives and I did read the NEWS. I am aware that 2.6.0 is newer, but I am also aware that the performance problems that I am having are abnormal. The fact of the matter is that 2.6.0 has not been widely adpoted yet. I am not opposed to running 2.6.0, but I would at least like to try to figure out whats wrong with 2.5.7, especially since the problem that I am having is NOT in the known bugs and is NOT addressed in the archives. -Andrew > > > -- > ________________________________________________________________ > J.W. Schultz Pegasystems Technologies > email address: [EMAIL PROTECTED] > > Remember Cernan and Schmitt
signature.asc
Description: This is a digitally signed message part
-- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
