-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 It is still the same number of stat calls. Doesn't really matter if you split them up.
Can you rsync to the NFS server directly? On 01/13/2014 05:34 PM, Perry Smith wrote: > Ok. I can get the Mac up to version 3 but I'm wondering if I need > to rethink my whole strategy. Since the source is on NFS, doing a > stat on all the files each run may cost me too much time. > > I might need to split it into smaller pieces and then rotate > through the pieces via a script. Do you have any suggestions for > this type of situation? > > Perry > > On Jan 13, 2014, at 4:08 PM, Kevin Korb <k...@sanitarium.net> > wrote: > >> Signed PGP part On 01/13/2014 05:05 PM, Perry Smith wrote: >>> A friend and I noticed the --times or --archive flag. I have >>> not stopped it yet but I'll add that flag (probably --times). >>> >>> This is the first time so it must be #2. >>> >>> The side issuing the command is a Mac using rsync version >>> 2.6.9 protocol version 29. The other side is AIX using rsync >>> version 3.1.0 protocol version 31 (that I built myself). >> >> Yes, if either end is version 2 then rsync will have to index >> the entire tree on both systems before it starts copying >> anything. >> >>> I don't mind recompiling rsync on the Mac side if you think >>> that would improve things. >> >> I have no Mac experience but that is the way it is everywhere >> else. >> >>> I was trying to find some type of scratch file or something >>> but could not. I'm curious, where is the index kept? >> >> There is no index kept. Rsync has no memory between runs which >> is why copying the timestamps is important. >> >> When I say indexing files I really mean it is going through the >> tree and doing a stat() on everything so it will have a list of >> existing files and timestamps to compare with the other end. >> Rsync v3 does this too but it does it incrementally while it is >> also copying stuff. >> >>> Thank you for your help Perry >>> >>> On Jan 13, 2014, at 2:49 PM, Kevin Korb <k...@sanitarium.net> >>> wrote: >>> >>>> Signed PGP part First, don't run rsync without either --times >>>> or --archive. Without that rsync won't copy timestamps and >>>> it won't be able to tell what is changed when you run it >>>> again. >>>> >>>> Second, if rsync isn't copying anything then there are 2 >>>> reasons... 1. You already have most of the files copied and >>>> it is going through them looking for a file that needs >>>> updating 2. You are using rsync version 2 where all files had >>>> to be indexed before it copied anything. >>>> >>>> On 01/13/2014 03:06 PM, Perry Smith wrote: >>>>> This is my first time to really use rsync. I did small >>>>> tests to get the arguments like I wanted and then kicked >>>>> off the big rsync about 2 and a half hours ago. So far, it >>>>> has not copied over any files. >>>>> >>>>> The command I used is: >>>>> >>>>> rsync \ --relative \ --recursive \ --copy-links \ >>>>> host:/glob/that/matches/about/eighty/./directories \ >>>>> /local/target/dir >>>>> >>>>> The list of directories are all full of symbolic links >>>>> that point off to NFS mounted file systems. I don't expect >>>>> it to complete today but I do have to stop it each day at >>>>> the end of the work day. But it worries me that it has yet >>>>> to copy over any files. >>>>> >>>>> Is it really making progress? Or will it take this long >>>>> to really start copying files over each day I start it? >>>>> >>>>> I expect the total amount copied to be about 400G and about >>>>> 4 million files. >>>>> >>>>> It is possible to break this up into pieces if that would >>>>> help. >>>>> >>>>> Thank you for your help and advice, Perry >>>>> >>>>> >>>>> >>>> >>>> -- >>>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >>>> >>>> >> >>>> Kevin Korb Phone: (407) 252-6853 >>>> Systems Administrator Internet: FutureQuest, Inc. >>>> ke...@futurequest.net (work) Orlando, Florida >>>> k...@sanitarium.net (personal) Web page: >>>> http://www.sanitarium.net/ PGP public key available on web >>>> site. >>>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >>>> >>>> >>>> >> >>>> - -- >>>> Please use reply-all for most replies to avoid omitting the >>>> mailing list. To unsubscribe or change options: >>>> https://lists.samba.org/mailman/listinfo/rsync Before >>>> posting, read: >>>> http://www.catb.org/~esr/faqs/smart-questions.html >>> >> >> -- >> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >> >> Kevin Korb Phone: (407) 252-6853 >> Systems Administrator Internet: FutureQuest, Inc. >> ke...@futurequest.net (work) Orlando, Florida >> k...@sanitarium.net (personal) Web page: >> http://www.sanitarium.net/ PGP public key available on web site. >> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >> > >> - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Florida k...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlLUbFcACgkQVKC1jlbQAQd6VwCfe8wnRBnfp02Bs5yq93LAJgNF ASIAn0PWLh1yvfn49j6Pm4vR29eJCo0G =ezKS -----END PGP SIGNATURE----- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
