Hello Aknin,
Thursday, April 19, 2007, 7:20:26 AM, you wrote:
|
> |
Hi Robert, thanks for the information. I understand from your words that you're more worried about overall filesystem size rather than the number of files, yes? Is the number of files something I should or should not worry about? i.e., what are the differences (in stability, recoverability, performance, manageability... etc) between a 25TB filesystem with 2^35 files and a 25TB filesystem with 1,000 files, each 25GB? |
If you are ok with your application having to access lot of small files then it's not an issue except backup.
Really depends how you want to do your backup. Lot of small files is bad, very bad for classical backup solutions.
In terms of many small files I see no problem with stability, recoverability or performance (depends on app and workload).
Now the difference in a scenario you asked is that if you want to backup 1000 files, depending what file system you use and how you created those files you're probably going to read them mostly sequentially on physical layer. Also it's very cheap in most cases to check 1000 files if they changed instead of millions.
As I wrote - if your app/workload is happy with many small files then fine.
But you'll definitely have a problem with a backup.
|
> |
Also, if it's possible to ask without stepping out of any of your customers' NDAs, can you at least say what's the average filesize you have on some of your multi-tera volumes (is 10K a small file? is 100K? 1K?) |
I'm afraid I can't :(
But I can say that to me anything below 512KB is a small file (starting from few bytes).
Also a file size distribution is such that I have mostly small files and large files, the rest 10% is somewhere between.
--
Best regards,
Robert Milkowski mailto:[EMAIL PROTECTED]
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
