On 8/21/07, Matthew Ahrens <[EMAIL PROTECTED]> wrote: > Brandorr wrote: > > Is ZFS efficient at handling huge populations of tiny-to-small files - > > for example, 20 million TIFF images in a collection, each between 5 > > and 500k in size? > > Do you mean efficient in terms of space used? If so, then in general it is > quite efficient. Eg, files < 128k space is rounded up to only a multiple of > 512 bytes. Around 1k of metadata is consumed per file. > > There are however a few cases where it will not be optimal. Eg, 129k files > will use up 256k of space. However, you can work around this problem by > turning on compression.
You answer part of my question perfectly. (Regarding space utilization.) The other was related to performance, but someone else has answered that. > > I am asking because I could have sworn that I read somewhere that it > > isn't, but I can't find the reference. > > If you find it, let us know. It turns out, what I read was related to the fact that RAID-Z was a suboptimal volume layout for the "large amounts of small files" use case. (I you still want, I can probably find it, as I think it was an opensolaris.org discussion thread.) (Richard reminded me of this.) One issue, the person looking at doing this has 8 x 750GB drives, so some sort of parity based raid striping will be required. Being that this will be suboptimal for any file system, I think the performance impact can be dealt with. I'd like to thank everyone for their responses. Based on the discussion we've had and my reviews of the XFS and ReiserFS file systems, I feel very confident recommending ZFS as a superior alternative. (ZFS data integrity, scalability and compression are the key winners here.). (Now I just need to research compatibility between Linux and Solaris NFS and whether his 3Ware card works with OpenSolaris. Thanks, Brian P.S. - Is there a ZFS FAQ somewhere? -- - Brian Gupta http://opensolaris.org/os/project/nycosug/ _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
