On Mon, Jun 20, 2011 at 1:51 PM, Walter Dnes <waltd...@waltdnes.org> wrote: > On Mon, Jun 20, 2011 at 08:25:57AM -0700, Mark Knecht wrote >> Hi, >> Is split an appropriate program to use to break a single 10GB file >> into 100 100MB files to transfer over the net using rsync, and then >> use cat to reassemble? >> >> Is there some better way to do this? > > That's what split was written for. I can't think of anything better. > BTW, what type of data is the 10 gig file? If it's text, then consider > using zip or bzip2 on each of the fragments before transferring. If > it's an already compressed binary format, then don't waste time > attempting further compression. > > -- > Walter Dnes <waltd...@waltdnes.org>
Hi Walter. It's a Virtualbox VM exported from virtualbox-4.0.8. It consumes about 40GB on disk, compressed by Virtualbox to about 10GB by their 'Export Appliance' feature. I actually did try compressing the exported 10GB file with gzip & bzip2 before splitting it. None of those provided any compression. I didn't try the spilt outputs as I figured they are just binary chunks and wouldn't compress either. Thanks for the info. Cheers, Mark
