Rsync is great. (What is cp? What is tar? Since I started using rsync I forgot how to use those tools.) I sync over 1,000,000 files (50G) to multiple destinations every day. Over medium speed links it takes about 3 hrs.
The default mode of operation creates a hidden (.file) on the destination while transferring. This is only done one file at a time so there is very little overhead unless you have a 10G filesystem with a single 9G file -- even then I think there are rsync flags to deal with this situation... If you run into a file count limitation then split it up do do the rsync on a dir-by-dir basis. There are scripts in the mailing list archives. I don't have enough memory/time to easily do the whole tree at once -- I do each top-level subdir individually. eric > Duane Meyer wrote: > > This is a simple question. How much file system overhead is there with > this system? Is it only as large as the largest file transfered or > could you potentially (even if configured correctly) end up with > double what you started out with on the sending or receiving end? > > The reason I need to be sure is that I have a file system that's > literally several hundred thousand files. Complete it's around > 25-30GB.. these are sparse files so there's lots of room for > compression. ..tar'd and compressed it's 3-4GB. > > I just need to be sure there will be enough room if no single file is > larger than about 200K and I have about 5GB of space left of the > partition. > > It's just taking way too long to tar and compress the complete > directory, transfer than uncompress and untar.. the last run took > around 24 hours front to back. I'm hoping to cut that to around 4-5 > hours. :) (probably optimistic, but who knows!) > > Anybody know of limitations on file count or size I might run into > here? > > Any help is appreciated for this rsync newbie. > > Regards, > Duane Meyer
