On Thu, 21 Aug 2008, Brent Jones wrote: > I have been experimenting with ZFS on a test box, preparing to present it to > management. > One thing I cannot test right now is our real-world application load. We > write to CIFS shares currently in small files. > We write about 250,000 files a day, in various sizes (1KB to 500MB). Some > directories get a lot of individual files (sometimes 50,000 or more) in a > single directory.
As a test case, I created a million files in one directory and do not notice that zfs itself is slower for that directory. Time to access files directly by name in that directory is similar to small directories. Some applications definitely have problems with huge directories since they want to obtain all the directory info at once and sort it or build GUI screens. The GNOME file browser will declare (after a few minutes) that the huge directory is empty without displaying an error message. > We spoke to a Sun storage person, and he said our scenario is unique > in our file count, and we could run into severe performance > problems. Has anyone seen how ZFS behaves under such file counts? > Currently NTFS handles it reasonably well (Explorer doesn't like > large directories, but our applications bypass that). If your system has sufficient RAM installed (for caching), I don't see how the raw file count is going to make much difference. The main thing that you don't want to do is to bring the pool too close to full. Always over-provision the storage space in your pool to keep it happy. This avoids performance-robbing fragmentation and assures that zfs can easily find fresh blocks. You will find that zfs is superb at writing new files, and is less good at updating existing files in place. Bob ====================================== Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
