Re: rsync'ing large files

2004-04-23 Thread Peter L. Wargo
On Fri, 23 Apr 2004, Wayne Davison wrote: > Rsync doesn't call sqrt -- it just does a simple little integer > estimation. It is probable that using a hardware sqrt call would be > faster than rsync's estimator, but taking only 0.0041 seconds per > file instead of 0.0232 seconds per file(*

Re: rsync'ing large files

2004-04-23 Thread Wayne Davison
On Thu, Apr 22, 2004 at 03:21:10PM -0600, Peter L. Wargo wrote: > IIRC, the G5 has hardware sqrt. Rsync doesn't call sqrt -- it just does a simple little integer estimation. It is probable that using a hardware sqrt call would be faster than rsync's estimator, but taking only 0.0041 seconds p

Re: rsync'ing large files

2004-04-23 Thread Peter L. Wargo
On Fri, 23 Apr 2004, Paul Slootman wrote: > I don't think that the overhead of one sqrt() per file will make much > difference either way... It might over tens of thousands of files... :-) -Pete -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting,

Re: rsync'ing large files

2004-04-23 Thread Paul Slootman
On Thu 22 Apr 2004, Peter L. Wargo wrote: > On Thu, 22 Apr 2004, Wayne Davison wrote: > > > * Per-file dynamic block size is now sqrt(file length). The > Oooo... I need to go compile 2.6 on a G5 running OS X (unless somebody > beat me to it). IIRC, the G5 has hardware sqrt. This could be q

Re: rsync'ing large files

2004-04-22 Thread Peter L. Wargo
On Thu, 22 Apr 2004, Wayne Davison wrote: > * Per-file dynamic block size is now sqrt(file length). The > per-file checksum size is determined according to an algorithm > provided by Donovan Baarda which reduces the probability of rsync > algorithm corrupting data and fallin

Re: rsync'ing large files

2004-04-22 Thread Wayne Davison
On Thu, Apr 22, 2004 at 11:23:18AM -0400, Brian Cuttler wrote: > I can't address the algoritm questions but I'll tell you that we > had a tremendous improvement is speed when we switched to a newer > version of rsync. Yes, rsync 2.6.0 has some big improvements in the checksum processing for large

Re: rsync'ing large files

2004-04-22 Thread Brian Cuttler
I can't address the algoritm questions but I'll tell you that we had a tremendous improvement is speed when we switched to a newer version of rsync. We are using it (in this case) to rsync our oracle files to a separate partition on the system cpu. > I'm using rsync to copy some large (>1GB) ora

rsync'ing large files

2004-04-22 Thread Jeffrey Layton
I'm using rsync to copy some large (>1GB) oracle datafiles. I've noticed that sometimes it transfers some of the files twice. Some earlier posts to this list that I saw in the archives seemed to indicate that this is a problem with the rsync algorithm itself when dealing with large files. Some of