Georg Limbach via rsync wrote: > Hello, > > I have a problem while rsyncing a directory with perhaps 1.000.000 > files. Before this I generate some files locally and add them to temp > files. So this files should be excluded from syncing. > > My rsync call looks like this: > > rsync --exclude='tmp/' \ > --exclude-from=/tmp/tmp.GF7SsFPnS3 \ > --exclude-from=/tmp/tmp.8SjJNCHyaI \ > --exclude-from=/tmp/tmp.CxZXEoPjgV \ > --exclude-from=/tmp/tmp.G3g2iMo4bs \ > --exclude-from=/tmp/tmp.H9KJYPMfMS \ > --exclude-from=/tmp/tmp.PNi7cJaREP \ > --exclude-from=/tmp/tmp.S4N9H4lsU7 \ > --exclude-from=/tmp/tmp.a5Zlgh6pUK \ > --exclude-from=/tmp/tmp.eiUlMluAe8 \ > --exclude-from=/tmp/tmp.ma0S1YSewc \ > --exclude-from=/tmp/tmp.sLR95oVbVD \ > --exclude-from=/tmp/tmp.zbfeLpezMX \ > -ax --info=progress2 \ > -e 'ssh -x -T -o Compression=no' \ > '/srv/project/shared/uploads/' \ > 'project@host:/srv/project/shared/uploads/' > > In every temp file are 1000s of lines with files like this: > > orders/scan/file/2234480/scan.pdf > > When I started rsync it only should transfer perhaps 1000 files and > delete 20 ones. But it takes 15 minutes to do that. I think the big > count of excluded files tear down the compare speed of files. > > What can I do to speed up this process? > > Thanks for your advise! > > Georg
I found a big speed up when I starting providing a list of candidate files to rsync with the --files-from option. If you have some way of identifying which files might need to be rsynced that's quicker than rsync itself checking everything, it can make a big difference. That's not why I did it, but it was a nice bonus. cheers, raf -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
