> From: Asif Iqbal [mailto:vad...@gmail.com] > > currently to speed up the zfs send| zfs recv I am using mbuffer. It > moves the data > lot faster than using netcat (or ssh) as the transport method
Yup, this works because network and disk latency can both be variable. So without buffering, your data stream must instantaneously go the speed of whichever is slower: The disk or the network. But when you use buffering, you're able to go as fast as the network at all times. You remove the effect of transient disk latency. > that is why I thought may be transport it like axel does better than > wget. > axel let you create multiple pipes, so you get the data multiple times > faster > than with wget. If you're using axel to download something from the internet, the reason it's faster than wget is because your data stream is competing against all the other users of the internet, to get something from that server across some WAN. Inherently, all the routers and servers on the internet will treat each data stream fairly (except when explicitly configured to be unfair.) So when you axel some file from the internet using multiple threads, instead of wget'ing with a single thread, you're unfairly hogging the server and WAN bandwidth between your site and the remote site. Slowing down everyone else on the internet who are running with only 1 thread each. Assuming your zfs send backup is going local, on a LAN, you almost certainly do not want to do that. If your zfs send is going across the WAN ... maybe you do want to multithread the datastream. But you better ensure it's encrypted. _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
