Re: deep vs. shallow directory structure

2010-07-05 Thread Paul Slootman
On Mon 05 Jul 2010, Shai Ayal wrote: > > More directories probably won't help much. > > Thanks. Will more directories make the performance worse? Not necessarily, it depends on things like how fast scanning the directories is in relation to how many files actually need to be transferred, for exa

Re: deep vs. shallow directory structure

2010-07-05 Thread Shai Ayal
On Mon, Jul 5, 2010 at 1:45 PM, Paul Slootman wrote: > On Sat 03 Jul 2010, Shai Ayal wrote: > >> My question (finally): how to optimize rsync's speed? - how deep should the >> directory structure be? What would be faster to rsync - 1 directory with 2M >> files, or 16 directories with 125K files ea

Re: deep vs. shallow directory structure

2010-07-05 Thread Paul Slootman
On Sat 03 Jul 2010, Shai Ayal wrote: > My question (finally): how to optimize rsync's speed? - how deep should the > directory structure be? What would be faster to rsync - 1 directory with 2M > files, or 16 directories with 125K files each, or 256 directories with 8K > files each, Smaller d

deep vs. shallow directory structure

2010-07-03 Thread Shai Ayal
Hi all, I am using rsync to sync a directory tree of ~2 million files. I am using rsyncrypto with filename encryption to encrypt the files before syncing. When encrypting filenames, rsyncrypto assigned each file a filename which is made of hex digits. It also has the option to put the files in di