The NFS server is off somewhere else, locked down. secure, blah blah. Doing it via a script that rotates is the same number of stat calls but it would start at a different place each day.
If I start it day 1 and it gets 25% through the stat calls, on day 2, will rsync start where it left off or start back at the beginning? I figure since it does not save context, I would start back at the beginning. So if I rotate, it would start at a different point. On Jan 13, 2014, at 4:44 PM, Kevin Korb <k...@sanitarium.net> wrote: > Signed PGP part > It is still the same number of stat calls. Doesn't really matter if > you split them up. > > Can you rsync to the NFS server directly? > > On 01/13/2014 05:34 PM, Perry Smith wrote: > > Ok. I can get the Mac up to version 3 but I'm wondering if I need > > to rethink my whole strategy. Since the source is on NFS, doing a > > stat on all the files each run may cost me too much time. > > > > I might need to split it into smaller pieces and then rotate > > through the pieces via a script. Do you have any suggestions for > > this type of situation? > > > > Perry > > > > On Jan 13, 2014, at 4:08 PM, Kevin Korb <k...@sanitarium.net> > > wrote: > > > >> Signed PGP part On 01/13/2014 05:05 PM, Perry Smith wrote: > >>> A friend and I noticed the --times or --archive flag. I have > >>> not stopped it yet but I'll add that flag (probably --times). > >>> > >>> This is the first time so it must be #2. > >>> > >>> The side issuing the command is a Mac using rsync version > >>> 2.6.9 protocol version 29. The other side is AIX using rsync > >>> version 3.1.0 protocol version 31 (that I built myself). > >> > >> Yes, if either end is version 2 then rsync will have to index > >> the entire tree on both systems before it starts copying > >> anything. > >> > >>> I don't mind recompiling rsync on the Mac side if you think > >>> that would improve things. > >> > >> I have no Mac experience but that is the way it is everywhere > >> else. > >> > >>> I was trying to find some type of scratch file or something > >>> but could not. I'm curious, where is the index kept? > >> > >> There is no index kept. Rsync has no memory between runs which > >> is why copying the timestamps is important. > >> > >> When I say indexing files I really mean it is going through the > >> tree and doing a stat() on everything so it will have a list of > >> existing files and timestamps to compare with the other end. > >> Rsync v3 does this too but it does it incrementally while it is > >> also copying stuff. > >> > >>> Thank you for your help Perry > >>> > >>> On Jan 13, 2014, at 2:49 PM, Kevin Korb <k...@sanitarium.net> > >>> wrote: > >>> > >>>> Signed PGP part First, don't run rsync without either --times > >>>> or --archive. Without that rsync won't copy timestamps and > >>>> it won't be able to tell what is changed when you run it > >>>> again. > >>>> > >>>> Second, if rsync isn't copying anything then there are 2 > >>>> reasons... 1. You already have most of the files copied and > >>>> it is going through them looking for a file that needs > >>>> updating 2. You are using rsync version 2 where all files had > >>>> to be indexed before it copied anything. > >>>> > >>>> On 01/13/2014 03:06 PM, Perry Smith wrote: > >>>>> This is my first time to really use rsync. I did small > >>>>> tests to get the arguments like I wanted and then kicked > >>>>> off the big rsync about 2 and a half hours ago. So far, it > >>>>> has not copied over any files. > >>>>> > >>>>> The command I used is: > >>>>> > >>>>> rsync \ --relative \ --recursive \ --copy-links \ > >>>>> host:/glob/that/matches/about/eighty/./directories \ > >>>>> /local/target/dir > >>>>> > >>>>> The list of directories are all full of symbolic links > >>>>> that point off to NFS mounted file systems. I don't expect > >>>>> it to complete today but I do have to stop it each day at > >>>>> the end of the work day. But it worries me that it has yet > >>>>> to copy over any files. > >>>>> > >>>>> Is it really making progress? Or will it take this long > >>>>> to really start copying files over each day I start it? > >>>>> > >>>>> I expect the total amount copied to be about 400G and about > >>>>> 4 million files. > >>>>> > >>>>> It is possible to break this up into pieces if that would > >>>>> help. > >>>>> > >>>>> Thank you for your help and advice, Perry > >>>>> > >>>>> > >>>>> > >>>> > >>>> -- > >>>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ > >>>> > >>>> > >> > >>>> > Kevin Korb Phone: (407) 252-6853 > >>>> Systems Administrator Internet: FutureQuest, Inc. > >>>> ke...@futurequest.net (work) Orlando, Florida > >>>> k...@sanitarium.net (personal) Web page: > >>>> http://www.sanitarium.net/ PGP public key available on web > >>>> site. > >>>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ > >>>> > >>>> > >>>> > >> > >>>> > -- > >>>> Please use reply-all for most replies to avoid omitting the > >>>> mailing list. To unsubscribe or change options: > >>>> https://lists.samba.org/mailman/listinfo/rsync Before > >>>> posting, read: > >>>> http://www.catb.org/~esr/faqs/smart-questions.html > >>> > >> > >> -- > >> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ > >> > >> > Kevin Korb Phone: (407) 252-6853 > >> Systems Administrator Internet: FutureQuest, Inc. > >> ke...@futurequest.net (work) Orlando, Florida > >> k...@sanitarium.net (personal) Web page: > >> http://www.sanitarium.net/ PGP public key available on web site. > >> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ > >> > > > >> > -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. ke...@futurequest.net (work) > Orlando, Florida k...@sanitarium.net (personal) > Web page: http://www.sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >
signature.asc
Description: Message signed with OpenPGP using GPGMail
-- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
