On Monday 20 Jun 2011 22:06:59 Mark Knecht wrote: > On Mon, Jun 20, 2011 at 1:51 PM, Walter Dnes <waltd...@waltdnes.org> wrote: > > On Mon, Jun 20, 2011 at 08:25:57AM -0700, Mark Knecht wrote > > > >> Hi, > >> Is split an appropriate program to use to break a single 10GB file > >> into 100 100MB files to transfer over the net using rsync, and then > >> use cat to reassemble? > >> > >> Is there some better way to do this? > > > > That's what split was written for. I can't think of anything better. > > BTW, what type of data is the 10 gig file? If it's text, then consider > > using zip or bzip2 on each of the fragments before transferring. If > > it's an already compressed binary format, then don't waste time > > attempting further compression. > > > > -- > > Walter Dnes <waltd...@waltdnes.org> > > Hi Walter. > > It's a Virtualbox VM exported from virtualbox-4.0.8. It consumes about > 40GB on disk, compressed by Virtualbox to about 10GB by their 'Export > Appliance' feature. > > I actually did try compressing the exported 10GB file with gzip & > bzip2 before splitting it. None of those provided any compression. I > didn't try the spilt outputs as I figured they are just binary chunks > and wouldn't compress either. > > Thanks for the info.
You could try: wget -c -t 0 to retry (forever) should the transfer fail for some reason and not bother with splitting the file. -- Regards, Mick
signature.asc
Description: This is a digitally signed message part.
