On Tue, Oct 14, 2008 at 12:31 AM, Gray Carper <[EMAIL PROTECTED]> wrote: > Hey, all! > > We've recently used six x4500 Thumpers, all publishing ~28TB iSCSI targets > over ip-multipathed 10GB ethernet, to build a ~150TB ZFS pool on an x4200 > head node. In trying to discover optimal ZFS pool construction settings, > we've run a number of iozone tests, so I thought I'd share them with you and > see if you have any comments, suggestions, etc. > > First, on a single Thumper, we ran baseline tests on the direct-attached > storage (which is collected into a single ZFS pool comprised of four raidz2 > groups)... > > [1GB file size, 1KB record size] > Command: iozone -i o -i 1 -i 2 -r 1k -s 1g -f /data-das/perftest/1gbtest > Write: 123919 > Rewrite: 146277 > Read: 383226 > Reread: 383567 > Random Read: 84369 > Random Write: 121617 > > [8GB file size, 512KB record size] > Command: > Write: 373345 > Rewrite: 665847 > Read: 2261103 > Reread: 2175696 > Random Read: 2239877 > Random Write: 666769 > > [64GB file size, 1MB record size] > Command: iozone -i o -i 1 -i 2 -r 1m -s 64g -f /data-das/perftest/64gbtest > Write: 517092 > Rewrite: 541768 > Read: 682713 > Reread: 697875 > Random Read: 89362 > Random Write: 488944 > > These results look very nice, though you'll notice that the random read > numbers tend to be pretty low on the 1GB and 64GB tests (relative to their > sequential counterparts), but the 8GB random (and sequential) read is > unbelievably good. > > Now we move to the head node's iSCSI aggregate ZFS pool... > > [1GB file size, 1KB record size] > Command: iozone -i o -i 1 -i 2 -r 1k -s 1g -f > /volumes/data-iscsi/perftest/1gbtest > Write: 127108 > Rewrite: 120704 > Read: 394073 > Reread: 396607 > Random Read: 63820 > Random Write: 5907 > > [8GB file size, 512KB record size] > Command: iozone -i 0 -i 1 -i 2 -r 512 -s 8g -f > /volumes/data-iscsi/perftest/8gbtest > Write: 235348 > Rewrite: 179740 > Read: 577315 > Reread: 662253 > Random Read: 249853 > Random Write: 274589 > > [64GB file size, 1MB record size] > Command: iozone -i o -i 1 -i 2 -r 1m -s 64g -f > /volumes/data-iscsi/perftest/64gbtest > Write: 190535 > Rewrite: 194738 > Read: 297605 > Reread: 314829 > Random Read: 93102 > Random Write: 175688 > > Generally speaking, the results look good, but you'll notice that random > writes are atrocious on the 1GB tests and random reads are not so great on > the 1GB and 64GB tests, but the 8GB test looks great across the board. > Voodoo! ;> Incidentally, I ran all these tests against the ZFS pool in disk, > raidz1, and raidz2 modes - there were no significant changes in the results. > > So, how concerned should we be about the low scores here and there? Any > suggestions on how to improve our configuration? And how excited should we be > about the 8GB tests? ;> > > Thanks so much for any input you have! > -Gray > --- > University of Michigan > Medical School Information Services > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
Your setup sounds very interesting how you export iSCSI to another head unit, can you give me some more details on your file system layout, and how you mount it on the head unit? Sounds like a pretty clever way to export awesomely large volumes! Regards, -- Brent Jones [EMAIL PROTECTED] _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
