Hi Bill, I see you mention that you have tried the find | xargs rsync option without a lot of luck so i will just put this here in case it is different to what you have tried but we are using this quite sucessfully
cd <source>; find . -maxdepth 1 -mindepth 1 ! -path './.*' -print0 | xargs -0 -n1 -P<no_of_threads> -I% rsync -irlt % <destination>/. I am however keen to look at the parsync linked by Nic. Thanks for the link Nick Evans On 16 February 2017 at 16:01, Nic Baxter via luv-main <[email protected]> wrote: > > > On 16/02/17 12:12, Bill Yang via luv-main wrote: > > Hi there > > > I need to transfer 200+ TB data from one storage server (Red Hat Linux > based) to another (FreeBSD). I am planning to use rsync with multiple > threads in a script. There are a number of suggestions on the Internet > (find + xargs + rsync), but none of them worked well so far. I also need a > reliable way to check whether all files/directories from the source server > have been copied to the destination server. Any suggestions/help would be > appreciated. > > This may help. http://moo.nac.uci.edu/~hjm/parsync/ > scripts running on top of rsync with some crude load balancing and > throttling. > > _______________________________________________ > luv-main mailing list > [email protected] > https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main > >
_______________________________________________ luv-main mailing list [email protected] https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
