On Tue, 12 Feb 2008 10:52:00 +0100, Giuliano Gavazzi <[EMAIL PROTECTED]>
wrote:
On 12Feb, 2008, at 09:53 , Martin Strand wrote:
I'm trying to use rsync to get a live backup of 2 million files, about
50 gb, max depth of 5 directory levels.
I'm on a gigabit lan so I'm passing -W, but it's still incredibly slow.
What else can I do to speed things up?
Perhaps there's a good way to filter out files older than X so only
newer files are checked?.
Will rsync 3.0.0 make a big difference for large trees?
I say yes, although you did not quantify your "slow".
The following was not on a gigabit, but on a firewire (tcp/ip over
firewire), and I suspect that it would have worked better on a gigabit
(better drivers). It was reported some time ago using rsync-3.0.0pre7:
a new tranfer of about 30GB worth of data in about 2 hours:
Number of files: 407317
Total file size: 34860439736 bytes
Total transferred file size: 34860261591 bytes
Literal data: 34860261591 bytes
Matched data: 0 bytes
File list size: 10391700
File list generation time: 0.003 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 35084947192
Total bytes received: 7332354
sent 35084947192 bytes received 7332354 bytes 4927652.82 bytes/sec
A subsequent re-rsync took about 5 minutes (little data changes).
I would say that you can look at about the same sort of figures, or even
a bit less, with better hardware. This was after all just a PowerBook G4
backed up to an iMac Intel.
Giuliano
I just tried it with rsync 3.0.0pre8 and it was significantly faster.
Thanks!
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
