On Mon, Jun 14, 2004 at 11:03:12PM -0700, Craig Barratt wrote: > "Wallace Matthews" writes: > > > I copy the 29 Gig full backup back into fedor//test/Kibbutz and issue > > the command "time rsync -avv --rsh=rsh --stats --block-size=181272 > > /test/Kibbutz/Kbup_1 fedor://test/Kibbutz" and it CRAWLS during delta > > generation/transmittal at about 1 Megabyte per second. > > > > I have repeated the experiment 3 times; same result each time. > > > > The only thing that is different is --block-size= option. First, > > time it isnt specified and I get a predictable answer. Second > > time, I give it a block size that is about 1/2 of square root of > > (29 Gig) and that is ok. But, explicitly give it something that > > is approximately the square root of the 29 Gig and it CRAWLS. > > > > When I cancel the command, the real time is 86 minutes and the > > user time is 84 minutes. This is similar to the issue I reported > > on Friday that Chris suggested I remove the --write-batch= option > > and that seemed to fix the CRAWL. > > If I understand the code correctly, map_ptr() in filio.c maintains > a sliding window of data in memory. The window starts 64K prior > to the desired offset, and the window length is 256K. So your > block-size of 181272 occupies most of the balance of the window. > > Each time you hit the end of the window the data is memmoved > and the balance needed is read. With such a large block size > there will be a lot of memmoves and small reads. > > I doubt this issue explains the dramatic reduction in speed, but > it might be a factor. Perhaps there is a bug with large block > sizes? >
I agree that this shouldn't account for such a slow-down. Perhaps Wally can test this hypothesis by using a block size exactly equal to the window size (256k?) and one less (256k-1?) and one more (256k+1?). > And, yes, your observation about the number of matching blocks > needs to be explored. I agree. > > Craig > -- > To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
