Perry Smith wrote:
This is my first time to really use rsync. I did small tests to get the
arguments like I wanted and then kicked off the big rsync about 2 and a half
hours ago. So far, it has not copied over any files.
-----
Is it really making progress? Or will it take this long to really start
copying files over each day I start it?
I expect the total amount copied to be about 400G and about 4 million files
-----
This appears to be a classic case of using a hammer to drive in a screw.
Um... rsync was designed to save network bandwidth by running on the
host and
doing file-stat intensive stuff ON the local hard disk (by running on
the server and
on your client).
But your usage case does very badly because rsync needs direct access on
both end ---
THEN it optimizes the stuff transferred to minimize the amount needing
to be copied
over the network.... But you are not getting ANY benefit because it
will do all of those
stats over the network via NFS which is notoriously slow in many or most
cases (especially
with lots of stat calls).
Your copy job would already be done if you did it with 'tar' and just
copied over everything.
On the receiving end tell tar not to overwrite newer stuff. Yes it
will waste more network
bandwidth, but it would very likely, already be done.
As you have described the problem, there is no real reason to use rsync,
as it is unable to
optimize network bandwidth because all the stats are remote.
Even "cp -au src/. dst/." will likely be faster than trying to use
rsync.... talk about
tool abuse! ;-)
For rsync to do a reasonable job, you really need to tell whoever owns
that server to put
rsync ON that server so it can access the files locally, then it could
do what it does best
and build up a list of differences so it only needs to transfer the
changed stuff.
Certainly, even if you have rsync on the remote end -- for the 1st
transfer, if you need to transfer
most of the files, it would be better just to create a tar on the remote
end, compress it, and copy
that locally.
How is it that you have so much data on a server you don't have any
ability to run a
local 'job' on? It really sounds like an impediment to you getting
your work done.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
