-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The answer to the original question "What is it doing?" is usually best answered by rsync it self with -vvv
Perry Smith skrev 2014-01-14 00:19: > Yea. Shouldn't be hard to split up. The hard part is some type of > dependable rotation. > > You mention "pause"... I have to disconnect so I assume that would > "abort" the transfer. But that triggered another question: would > daemon mode help in this situation? (I assume not. The daemon > probably folks and the child does all the work and dies when the > connection is lost.) > > Perry > > On Jan 13, 2014, at 4:55 PM, Kevin Korb <k...@sanitarium.net> > wrote: > >> Signed PGP part If you have to abort it then I suppose that makes >> sense. Otherwise you could throttle or pause it. >> >> If you do have to split it up then it shouldn't be difficult. >> Your original command was specifying multiple sources using a >> glob of some kind so you would just need to alter that. >> >> On 01/13/2014 05:51 PM, Perry Smith wrote: >>> The NFS server is off somewhere else, locked down. secure, >>> blah blah. >>> >>> Doing it via a script that rotates is the same number of stat >>> calls but it would start at a different place each day. >>> >>> If I start it day 1 and it gets 25% through the stat calls, on >>> day 2, will rsync start where it left off or start back at the >>> beginning? I figure since it does not save context, I would >>> start back at the beginning. >>> >>> So if I rotate, it would start at a different point. >>> >>> On Jan 13, 2014, at 4:44 PM, Kevin Korb <k...@sanitarium.net> >>> wrote: >>> >>>> Signed PGP part It is still the same number of stat calls. >>>> Doesn't really matter if you split them up. >>>> >>>> Can you rsync to the NFS server directly? >>>> >>>> On 01/13/2014 05:34 PM, Perry Smith wrote: >>>>> Ok. I can get the Mac up to version 3 but I'm wondering if >>>>> I need to rethink my whole strategy. Since the source is >>>>> on NFS, doing a stat on all the files each run may cost me >>>>> too much time. >>>>> >>>>> I might need to split it into smaller pieces and then >>>>> rotate through the pieces via a script. Do you have any >>>>> suggestions for this type of situation? >>>>> >>>>> Perry >>>>> >>>>> On Jan 13, 2014, at 4:08 PM, Kevin Korb >>>>> <k...@sanitarium.net> wrote: >>>>> >>>>>> Signed PGP part On 01/13/2014 05:05 PM, Perry Smith >>>>>> wrote: >>>>>>> A friend and I noticed the --times or --archive flag. >>>>>>> I have not stopped it yet but I'll add that flag >>>>>>> (probably --times). >>>>>>> >>>>>>> This is the first time so it must be #2. >>>>>>> >>>>>>> The side issuing the command is a Mac using rsync >>>>>>> version 2.6.9 protocol version 29. The other side is >>>>>>> AIX using rsync version 3.1.0 protocol version 31 (that >>>>>>> I built myself). >>>>>> >>>>>> Yes, if either end is version 2 then rsync will have to >>>>>> index the entire tree on both systems before it starts >>>>>> copying anything. >>>>>> >>>>>>> I don't mind recompiling rsync on the Mac side if you >>>>>>> think that would improve things. >>>>>> >>>>>> I have no Mac experience but that is the way it is >>>>>> everywhere else. >>>>>> >>>>>>> I was trying to find some type of scratch file or >>>>>>> something but could not. I'm curious, where is the >>>>>>> index kept? >>>>>> >>>>>> There is no index kept. Rsync has no memory between >>>>>> runs which is why copying the timestamps is important. >>>>>> >>>>>> When I say indexing files I really mean it is going >>>>>> through the tree and doing a stat() on everything so it >>>>>> will have a list of existing files and timestamps to >>>>>> compare with the other end. Rsync v3 does this too but it >>>>>> does it incrementally while it is also copying stuff. >>>>>> >>>>>>> Thank you for your help Perry >>>>>>> >>>>>>> On Jan 13, 2014, at 2:49 PM, Kevin Korb >>>>>>> <k...@sanitarium.net> wrote: >>>>>>> >>>>>>>> Signed PGP part First, don't run rsync without >>>>>>>> either --times or --archive. Without that rsync >>>>>>>> won't copy timestamps and it won't be able to tell >>>>>>>> what is changed when you run it again. >>>>>>>> >>>>>>>> Second, if rsync isn't copying anything then there >>>>>>>> are 2 reasons... 1. You already have most of the >>>>>>>> files copied and it is going through them looking for >>>>>>>> a file that needs updating 2. You are using rsync >>>>>>>> version 2 where all files had to be indexed before it >>>>>>>> copied anything. >>>>>>>> >>>>>>>> On 01/13/2014 03:06 PM, Perry Smith wrote: >>>>>>>>> This is my first time to really use rsync. I did >>>>>>>>> small tests to get the arguments like I wanted and >>>>>>>>> then kicked off the big rsync about 2 and a half >>>>>>>>> hours ago. So far, it has not copied over any >>>>>>>>> files. >>>>>>>>> >>>>>>>>> The command I used is: >>>>>>>>> >>>>>>>>> rsync \ --relative \ --recursive \ --copy-links \ >>>>>>>>> host:/glob/that/matches/about/eighty/./directories >>>>>>>>> \ /local/target/dir >>>>>>>>> >>>>>>>>> The list of directories are all full of symbolic >>>>>>>>> links that point off to NFS mounted file systems. >>>>>>>>> I don't expect it to complete today but I do have >>>>>>>>> to stop it each day at the end of the work day. But >>>>>>>>> it worries me that it has yet to copy over any >>>>>>>>> files. >>>>>>>>> >>>>>>>>> Is it really making progress? Or will it take >>>>>>>>> this long to really start copying files over each >>>>>>>>> day I start it? >>>>>>>>> >>>>>>>>> I expect the total amount copied to be about 400G >>>>>>>>> and about 4 million files. >>>>>>>>> >>>>>>>>> It is possible to break this up into pieces if >>>>>>>>> that would help. >>>>>>>>> >>>>>>>>> Thank you for your help and advice, Perry >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >>>>>>>> >>>>>>>> >>>>>> >>>>>>>> >>>> >>>>>>>> >> >>>>>>>> Kevin Korb Phone: (407) 252-6853 >>>>>>>> Systems Administrator Internet: FutureQuest, Inc. >>>>>>>> ke...@futurequest.net (work) Orlando, Florida >>>>>>>> k...@sanitarium.net (personal) Web page: >>>>>>>> http://www.sanitarium.net/ PGP public key available >>>>>>>> on web site. >>>>>>>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>>>> >>>> >>>>>>>> >> >>>>>>>> - -- >>>>>>>> Please use reply-all for most replies to avoid >>>>>>>> omitting the mailing list. To unsubscribe or change >>>>>>>> options: >>>>>>>> https://lists.samba.org/mailman/listinfo/rsync >>>>>>>> Before posting, read: >>>>>>>> http://www.catb.org/~esr/faqs/smart-questions.html >>>>>>> >>>>>> >>>>>> -- >>>>>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >>>>>> >>>>>> >>>> >>>>>> >> >>>>>> Kevin Korb Phone: (407) 252-6853 >>>>>> Systems Administrator Internet: FutureQuest, Inc. >>>>>> ke...@futurequest.net (work) Orlando, Florida >>>>>> k...@sanitarium.net (personal) Web page: >>>>>> http://www.sanitarium.net/ PGP public key available on >>>>>> web site. >>>>>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >>>>>> >>>>> >>>>>> >>>> >>>>>> >> >>>>>> - -- >>>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >>>> >>>> >> >>>> Kevin Korb Phone: (407) 252-6853 >>>> Systems Administrator Internet: FutureQuest, Inc. >>>> ke...@futurequest.net (work) Orlando, Florida >>>> k...@sanitarium.net (personal) Web page: >>>> http://www.sanitarium.net/ PGP public key available on web >>>> site. >>>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >>>> >>> >>>> >> >>>> - -- >> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >> >> Kevin Korb Phone: (407) 252-6853 >> Systems Administrator Internet: FutureQuest, Inc. >> ke...@futurequest.net (work) Orlando, Florida >> k...@sanitarium.net (personal) Web page: >> http://www.sanitarium.net/ PGP public key available on web site. >> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ >> > >> > > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (Darwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJS1R6VAAoJEOB3m+uCZjRD8IEIAKYaUWLUpzJesJ2HOegaldeX bO1uVorHap5+54HzgmdyJbT6cxdbYWf88Oa2qo2sX5bN5fyuM5DuykA46fnyo+50 pxtD5tG87gC4n847Uor3UrCWk+L8W1VPxmdSmk8137kKldj1nD8vpqH26DNXsEVa jI9XJ/W7glQ+/biW5CuwM7HlbwxC489NQnM++xw6luPw3FJpqboQSSyW0YLiOQzz iLj6W25Wi7uyQYyVC4PGkcBSgvuOmZZBUYoM/+mFE+/yz832hk4Mg2v8oSM336jJ TQDG8zSKp3SlXQChA3A/kueknkXgdI4rVAMbJ411at0SfSK/GyCEIW9HPZ+0lSI= =wXIQ -----END PGP SIGNATURE----- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
