I agree that something isn't scaling too well. Sync works ok with directories < 1 GB ( ~17 000 files, 1300 folders) less than a minute..
There are ~ 500 000 (half a million) files, and ~74 000 folders. When I browsed the sync / copy code I found that there was a lot of synchronized code blocks. I doubt that it works much more faster with paraller tasks (not tried, yet..). -- -Antti- 2008/1/25, Dominique Devienne <[EMAIL PROTECTED]>: > > On Jan 25, 2008 1:02 AM, Antti Luoma <[EMAIL PROTECTED]> wrote: > > BTW, very large in this case means ~17 GB :) > > Well, transferring 17GB @ 100Mb/s takes in theory 1350 seconds (less > than 23 minutes). > > You indicated a total time for the sync of 56667816 seconds, which is > about 300 bytes / seconds. > > Even accounting for non-ideal network conditions and directory > traversal, there's still a huge gap between 100Mb/s and 2.4Kb/s > (40,000x different in speed...). > > Something doesn't quite compute here ;-) How many files and dirs do > you have in these 17GB? It's possible some of the data-structure used > in sync don't scale well for a very large number of files... > > Assuming the dir you sync has a few well known top level dirs, you > could also sync those instead of the "root" sync dir, and put several > sync in <parallel>. --DD > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
