Hi there, Last week on IRC I had a short discussion with Bert about the file handling overhead on Windows. That prompted me to run some tests on my notebook and this is what I found.
Opening or creating files and folders is not too expensive even on NTFS. Creating and deleting folders is about as expensive as creating or deleting empty files. Gains from using multiple worker threads are measurable but scalability is limited - at least as long as we operate on the same volume. And even on Windows, c/o speeds >2000 files / sec are technically feasible. All tests are being executed repeatedly for 100 different files in a single folder, no virus scanner active, 2 core 2.4GHz CPU; Windows XP 32 bits. Results in detail shown as runs per second: Test\#Threads 1 2 gain (1) 5,639 6,798 21% (2) 9,663 11,818 22% (3) 3,943 5,292 34% (4) 4,461 6,331 42% (5) 4,460 5,842 31% (6) 6,936 7,272 5% (7) 8,305 9,661 16% (1) ... create and close empty files (2) ... delete these empty files (3) ... create, write 12kB and close files (4) ... rename these files (5) ... delete these files (6) ... create empty directories (7) ... delete these empty directories Test (1) with varying number of files per folder, 1 thread: 10 6,307 100 5,639 1000 5,083 10000 4,319 -- Stefan^2.
