RE: How to speed up rsync when haveing lots of files

2009-03-03 Thread Boniforti Flavio
> You should not use -z at ALL with large numbers of small > files. The increased latency of the > compression/decompression will far exceed any time saved in > transmission. You're not on a 300 baud modem, I assume? > -z should be used when you are sending large files that are > compressib

Re: How to speed up rsync when haveing lots of files

2009-03-03 Thread lewis butler
On 3-Mar-2009, at 22:55, Daniel.Li wrote: -z is apparently affect the performance when CPU has a lower frequency, like 200MHz or so. When doing rsync, 100% cpu occupied, which limits network performance. You should not use -z at ALL with large numbers of small files. The increased latency

Re: How to speed up rsync when haveing lots of files

2009-03-03 Thread Daniel.Li
On Tue, 2009-03-03 at 21:26 -0700, lewis butler wrote: > On 3-Mar-2009, at 20:33, Daniel.Li wrote: > > On Tue, 2009-03-03 at 08:58 -0500, Charles Marcus wrote: > >> On 3/3/2009 8:52 AM, m...@bortal.de wrote: > > unfortunatelly rsync is beeing REALLY slow and produces a high > > load when

Re: How to speed up rsync when haveing lots of files

2009-03-03 Thread lewis butler
On 3-Mar-2009, at 20:33, Daniel.Li wrote: On Tue, 2009-03-03 at 08:58 -0500, Charles Marcus wrote: On 3/3/2009 8:52 AM, m...@bortal.de wrote: unfortunatelly rsync is beeing REALLY slow and produces a high load when we try to sync lots of files (>250 000 small files). What version of rsync

Re: How to speed up rsync when haveing lots of files

2009-03-03 Thread Ryan Malayter
On Tue, Mar 3, 2009 at 9:33 PM, Daniel.Li wrote: >> Upgrade to 3.0.5 (on both ends) > > OK, besides this, is there any other way to improve the network > performance? some thing like change the option or what? This is a well-known performance issue in rsync that was specifically addressed in v3.0

Re: How to speed up rsync when haveing lots of files

2009-03-03 Thread Daniel.Li
On Tue, 2009-03-03 at 08:58 -0500, Charles Marcus wrote: > On 3/3/2009 8:52 AM, m...@bortal.de wrote: > >>> unfortunatelly rsync is beeing REALLY slow and produces a high load when > >>> we try to sync lots of files (>250 000 small files). > > >> What version of rsync are you using? > > > # rsync

DO NOT REPLY [Bug 6151] --safe-links can be fooled by adding extra slashes to the path

2009-03-03 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=6151 way...@samba.org changed: What|Removed |Added Status|NEW |RESOLVED Resolution|

Re: Treat files that were modified locally and remote

2009-03-03 Thread Wayne Davison
On Sat, Feb 28, 2009 at 01:31:32PM +0100, Kurt wrote: > One way to identify such files would be to do a dry-run upsync, then a > dry-run downsync, find files that would have been sync'ed in both dry-runs, > and then prompt the user for some action. Assuming that you mean to use the -u option, th

Re: rsync --files-from

2009-03-03 Thread Wayne Davison
On Thu, Feb 26, 2009 at 07:55:22PM -0700, Manoj Joseph wrote: > Is there a way of passing a list of source _and_ destination paths?? Only for a single-file transfer. So, to do this with rsync, you'd need to transfer each file separately, possibly using the lighter daemon protocol (which could be

Re: How to speed up rsync when haveing lots of files

2009-03-03 Thread Charles Marcus
On 3/3/2009 8:52 AM, m...@bortal.de wrote: >>> unfortunatelly rsync is beeing REALLY slow and produces a high load when >>> we try to sync lots of files (>250 000 small files). >> What version of rsync are you using? > # rsync --version > rsync version 2.6.9 protocol version 29 Upgrade to 3.0.

Re: How to speed up rsync when haveing lots of files

2009-03-03 Thread m...@bortal.de
# rsync --version rsync version 2.6.9 protocol version 29 Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others. Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles, inplace, IPv6, ACLs, 64-bit system inums

RE: On which side these errors are?

2009-03-03 Thread Boniforti Flavio
> It would help if you gave the command used. > The opendir refers to a module name "Profili", but without > knowing if you're syncing to or from the module I can't say > whether it's source or destination. Yes, of course you're right. The module "profili" is the remote server which I'm syncing

Re: On which side these errors are?

2009-03-03 Thread Paul Slootman
On Tue 03 Mar 2009, Boniforti Flavio wrote: > > rsync: opendir "client05" (in Profili) failed: Permission denied (13) > rsync: delete_file: rmdir(client02/SendTo) failed: Permission denied > (13) > > Am I correct if I say that the "opendir" is related to the source I'm > trying to rsync, but the