On Wed, Aug 8, 2012 at 8:10 PM, Darac Marjal <mailingl...@darac.org.uk> wrote: > On Wed, Aug 08, 2012 at 03:14:50PM +0800, lina wrote: >> Hi, >> >> It's a bit big data to transfer, around 1.1 T, >> >> from one server to another server. >> >> I checked that rsync is faster than scp, >> but in my situations rsync has elapsed for 1 hour, I guess the network >> is also a problem, >> >> Here I wish to know are there some tools (better default) can use for >> fast transferring, regardless the security reason, my data is just >> some data, no need special security care. >> > > In addition to the suggestions mentioned by other people, consider the > compressability of your data. I don't believe it's possible to > definitely predict the trade-offs here (it depends on how well the data > compresses, basically, but also on how your CPUs compare to the network > bandwidth, but you may find that spending some time compressing the data > reduces the overall time. > > In that case try, the following. On the receiver: > > $ nc -l -p 12345 | $COMP -d | pv > outfile > > and on the sender: > > $ pv infile | $COMP | nc receiver 12345 > > where $COMP is your preferred streaming compressor (gzip, bzip2 and xz > should all work nicely here). By the way, "pv" (package: pv) is a useful > pipeline-viewer and will show you progress (on the sending side) as well > as throughput levels.
Thank you, neither receiver nor sender has the pv installed. I will spend sometime to figure the nc out first. Best regards, > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iQIcBAEBCgAGBQJQIlc9AAoJEKB7YbRsd8TGRkkP/iQLhUKprzjOhVAPv8A4rRoI > QYktoGPfRedswG4ivs0EaUVySUuK4sjmFuSj84vLh3WRLmsQUtDpiFbOUnlJS1Uj > BrtKGEe73uMZQHrO6l/HtnOvUj/X5hxKPLI5ZFId3AvDIiiDOtZxVbLgjVEfrFnV > LeV5YKVwks0O9uGxTOwUItXjSF2LUG2W96gKDqVAOvRLSKweG6mKtWALjDs4QYTw > PPfpuvXxdEr0CscqUehph2iY/27MwPW6yNbC91o0uxRAj6/fcqnnJr3bR9+Vyxxk > HtP6G+niroSoF8EFV4ETtZQXubIZuMkp7FhaxRjynLkVFbskJ4pRIXgL789qOX38 > LJxHhPDNpn3F009rYaawz7t1ySv+sobicbI38hEQynCv9hZ9BiX0nIriTB2DXJ/A > 8RLkpKGSlUXbbCA8UJ35wBwVBdaVEhLGACOXsE3zLbjcjsEDRtiHJIwdeHfpJbK4 > xV1seAxkgKljPvTf+k+GEuRyh60CIduYx9yLG5icUV/HFVz/21azpqELBH2kWdpq > WKREKD5ymmzofFLQtS8QI8I5UH6ZerHfptj/r5E5ff4BcFNmQ7ys7q52vkV3HPeI > QBJ3GMuey/iNruQJXs7xU6P+BdXq+TyG9XK0LF4/4nk6mPSMot/Vqc5ShWsmeYkW > tHddx04MGvI0OHc04vOo > =a8qg > -----END PGP SIGNATURE----- > -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAG9cJm=cpg0uxfu38atmhkt7ku-5rfkvfwbgvb4wsj0h45z...@mail.gmail.com
