Attached is a patch that makes window strides constant when files are
walked with a constant block size. In these cases, it completely
avoids all memmoves.
In my simple local test of rsyncing 57MB of 10 local files, memmoved
bytes went from 18MB to zero.
I haven't tested this for a big variety of file cases. I think that this
will always reduce the memmoves involved with walking a large file, but
perhaps there's some case I'm not seeing.
Also, the memmove cost if obviously neglegible compared to real disk
i/o, so you have pretty much no chance of measuring the difference
unless your files are already starting in cache.
Also, with the new new caps on window size, the worst case
memmoves are quite a bit smaller than they used to be, so the benefit of
avoiding them is comensurately reduced. Therefore, in order to measure
the difference in terms of actually time to completion, you'd need to be
walking through a lot of cached data.
I don't have enough RAM (I'm at 192MB) to really measure this difference
well. If you do, feedback from testing is especially welcome. [glances
in wally's direction] :)
Overall, I think this should never hurt performance, but with large
datasets and much memory, it should improve performance.
-chris
Index: fileio.c
===================================================================
RCS file: /cvsroot/rsync/fileio.c,v
retrieving revision 1.15
diff -u -r1.15 fileio.c
--- fileio.c 20 Jul 2004 21:35:52 -0000 1.15
+++ fileio.c 2 Aug 2004 02:31:02 -0000
@@ -23,6 +23,7 @@
#include "rsync.h"
extern int sparse_files;
+int total_bytes_memmoved=0;
static char last_byte;
static int last_sparse;
@@ -182,8 +183,7 @@
/* nope, we are going to have to do a read. Work out our desired window */
if (offset > 2*CHUNK_SIZE) {
- window_start = offset - 2*CHUNK_SIZE;
- window_start &= ~((OFF_T)(CHUNK_SIZE-1)); /* assumes power of 2 */
+ window_start = offset;
} else {
window_start = 0;
}
@@ -212,6 +212,7 @@
read_offset = read_start - window_start;
read_size = window_size - read_offset;
memmove(map->p, map->p + (map->p_len - read_offset), read_offset);
+ total_bytes_memmoved += read_offset;
} else {
read_start = window_start;
read_size = window_size;
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
