Re: rsync performance on large files strongly depends on file's (dis)similarity

2014-04-11 Thread Thomas Knauth
Maybe an alternative explanation is that a high degree of similarity allows to skip more bytes on the sender. For each matched block, the sender can does not need to compute any checksums, weak or strong, for the next S bytes, where S is the block size. As the number of matched blocks decreases, i

Re: Rsync performance with large exchange database files

2014-02-10 Thread Kevin Korb
ence. > > Maybe this is normal and I've just not noticed it on these other > servers since they have a much smaller amount of data to backup? > Still seems like some thing is wrong. I wouldn't expect the speed > difference to be that huge. > > > > -- Original Mes

Re: Rsync performance with large exchange database files

2014-02-10 Thread bruce
Korb" To: rsync@lists.samba.org Sent: 2/10/2014 10:57:08 AM Subject: Re: Rsync performance with large exchange database files -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 3.1.0 will probably help some. What are the specs of the FreeBSD system? I have found that ZFS on FreeBSD is extremel

Re: Rsync performance with large exchange database files

2014-02-10 Thread Kevin Korb
/2014 > 10:57:08 AM Subject: Re: Rsync performance with large exchange > database files > >> -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 >> >> 3.1.0 will probably help some. >> >> What are the specs of the FreeBSD system? I have found that ZFS >> o

Re: Rsync performance with large exchange database files

2014-02-10 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 3.1.0 will probably help some. What are the specs of the FreeBSD system? I have found that ZFS on FreeBSD is extremely RAM hungry. In my experience 8GB of RAM is the minimum if dedup is disabled and 16BG of RAM for when dedup is enabled. Also, a ca

Re: Rsync performance with large exchange database files

2014-02-10 Thread bruce
-- Original Message -- From: br...@sqls.net To: rsync@lists.samba.org Sent: 2/10/2014 8:38:06 AM Subject: Rsync performance with large exchange database files I'm using a mixture of FreeBSD w/ ZFS+snapshots and rsync to backup all the servers at my day job. This works pretty good overa

Re: Rsync performance with very large files

2010-01-08 Thread Carlos Carvalho
Eric Cron (ericc...@yahoo.com) wrote on 8 January 2010 12:20: >We're having a performance issue when attempting to rsync a very large file. >Transfer rate is only 1.5MB/sec. My issue looks very similar to this one: > >http://www.mail-archive.com/rsync@lists.samba.org/msg17812.html > >I

Re: rsync performance

2006-08-23 Thread Tim Moore
Try using major subdirectories one level lower. Example sequence from my automated home machine backup: rsync -ax --stats / /rsync_backup/ rsync -ax --stats /usr/ /rsync_backup/usr/ rsync -ax --stats /home/ /rsync_backup/home/ Compare to "rsync -a --stats / /backup/" Ralf Fassel wrote: We're

Re: Rsync Performance

2004-07-22 Thread Jason Haar
On Thu, Jul 22, 2004 at 07:33:21PM -0700, Wayne Davison wrote: > On Fri, Jul 23, 2004 at 02:15:04PM +1200, Jason Haar wrote: > > is there any intention of a "new improved" "--partial" option whereby > > any failed uploads are kept as temp files > > I had been contemplating whether we need a new op

Re: Rsync Performance

2004-07-22 Thread Wayne Davison
On Fri, Jul 23, 2004 at 02:15:04PM +1200, Jason Haar wrote: > is there any intention of a "new improved" "--partial" option whereby > any failed uploads are kept as temp files I had been contemplating whether we need a new option for this or not. One idea would be to change the behavior when --par

Re: Rsync Performance

2004-07-22 Thread Jason Haar
On Wed, Jul 21, 2004 at 02:30:04PM -0700, Wayne Davison wrote: > That's what the --partial option indicates, though it does move the > partial file into place awaiting the next transfer (it does not auto- > resume). ..and would crash the box if that was an OS file... This has been discussed befor

Re: Rsync Performance

2004-07-21 Thread Wayne Davison
On Fri, Jul 16, 2004 at 12:36:45PM -0700, Joe Eckstrom wrote: > we would definitely want the ability to resume an incomplete download. That's what the --partial option indicates, though it does move the partial file into place awaiting the next transfer (it does not auto- resume). > The central r

Re: rsync performance question

2004-01-09 Thread jim
jw schultz writes: On Thu, Jan 08, 2004 at 01:05:25PM -0500, Rick Frerichs wrote: Hello, I seem to be having a performance problem with rsync. ... If I do a transfer (either way) with ftp, I get about 500 Kbytes/sec. Using rsync to do the same transfer (either way) I only get about 50 Kbytes

Re: rsync performance question

2004-01-08 Thread jw schultz
On Thu, Jan 08, 2004 at 01:05:25PM -0500, Rick Frerichs wrote: > Hello, > > I seem to be having a performance problem with rsync. > I have done some testing of rsync and ftp. If I do > a transfer (either way) with ftp, I get about 500 Kbytes/sec. > Using rsync to do the same transfer (either way)

Re: Rsync Performance In Windows

2003-06-17 Thread jw schultz
On Tue, Jun 17, 2003 at 12:08:43PM -0500, Chris McKeever wrote: > Greger.. > I replaced the rsync.exe with the one from your link, it relieved the > windows CPU some (from 100% to 98% with flucuation to 100%). > > I also took the advice of using the -u switch. From the man: > > -u, --update

RE: Rsync Performance In Windows

2003-06-17 Thread Chris McKeever
; Cc: [EMAIL PROTECTED] > > Subject: Re: Rsync Performance In Windowsv All you need to run rsync properly is rsync.exe, cygwin1.dll and cygpopt- 0.dll. There is no difference in performance if you do a full cygwin install or not. However, I suggest you try and download the version I compil

RE: Rsync Performance In Windows

2003-06-16 Thread Chris McKeever
Thanks for your reply! > -Original Message- > From: Greger Cronquist [mailto:[EMAIL PROTECTED] > Sent: Monday, June 16, 2003 2:42 PM > To: _Chris McKeever_ > Cc: [EMAIL PROTECTED] > Subject: Re: Rsync Performance In Windows > > > Did you compile from the s

RE: Rsync Performance In Windows

2003-06-16 Thread _Chris McKeever_
om: Chris McKeever > Sent: Monday, June 16, 2003 4:03 PM > To: 'Lapo Luchini'; _Chris McKeever_; rsync > Subject: RE: Rsync Performance In Windows > > > Thanks for your response... > > > > > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA1 &

RE: Rsync Performance In Windows

2003-06-16 Thread Chris McKeever
Thanks for your response... > > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > _Chris McKeever_ wrote: > > >The linux machine connecting to the windows rsync daemon > has a very low > >performance hit when the session is running (see below). > However, the > >windows machine, which

Re: Rsync Performance In Windows

2003-06-16 Thread Lapo Luchini
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 _Chris McKeever_ wrote: >The linux machine connecting to the windows rsync daemon has a very low >performance hit when the session is running (see below). However, the >windows machine, which has a much faster CPU hits a CPU usage of 100%. > rsync CPU

Re: Rsync Performance In Windows

2003-06-16 Thread Greger Cronquist
Did you compile from the sources, or did you grab the cygwin binary? I suggest you build it from the sources and apply the "craigb-perf.diff" patch which is in the patches directory in the 2.5.6 distribution. This patch makes all the difference for Windows (system calls cost a lot under cygwin)

Re: Rsync performance increase through buffering

2003-01-14 Thread Dave Dykstra
Craig, I'd like to get your patch into the 2.5.6 patches directory. Could you please make sure it applies cleanly onto version 2.5.6pre1 (see news on http://rsync.samba.org home page if you haven't been following the mailing list) and repost it? Thanks, - Dave Dykstra On Wed, Dec 11, 2002 at 03:

Re: Rsync performance increase through buffering

2002-12-12 Thread Greger Cronquist
I use a win2k<->win2k rsync with daemon, and that patch makes a *big* difference! Especially the file list transfer and syncing big files with small changes goes a lot faster with the patch. Unfortunatly the rsync deadlocks after all the files have been transferred, so that needs to be fixed first

Re: Rsync performance increase through buffering

2002-12-11 Thread Dave Dykstra
On Sun, Dec 08, 2002 at 11:48:57PM -0800, Craig Barratt wrote: > I've been studying the read and write buffering in rsync and it turns > out most I/O is done just a couple of bytes at a time. This means there > are lots of system calls, and also most network traffic comprises lots > of small packe