Re: Syncing large amounts of data

2003-02-13 Thread Steve Bonds
A good solution for the "lots of little files" problem is syncing below the filesystem layer (usually either at the logical volume or disk level.) This allows you to avoid the stat() system call on loads of files while looking for the ones to send. As was already suggested, if you can avoid the s

Re: Syncing large amounts of data

2003-02-12 Thread jw schultz
On Wed, Feb 12, 2003 at 01:13:45AM -0600, Adam Herbert wrote: > I need some suggestions. Here's my setup: > > 800GB of Data > 14,000,000+ Files > No changes just additions > Files range in size from 30k - 190k > > The files are laid out in a tree fashion like: > > BASE >