On 7/3/2010 2:22 PM, Roy Sigurd Karlsbakk wrote:
------------------------------------------------------------------------
>To summarise, putting 28 disks in a single vdev is nothing you
would do if you want performance. You'll end >up with as many
IOPS a single drive can do. Split it up into smaller (<10
disk) vdevs and try again. If you need >high performance, put
them in a striped mirror (aka RAID1+0)
>A little addition - for 28 drives, I guess I'd choose four
vdevs with seven drives each in raidz2. You'll loose >space,
but it'll be four times faster, and four times safer. Better
safe than sorry....
Ok... so we've rebuilt the pool as 14 pairs of mirrors, each pair
having one disk in each of the two JBODs. Now we're getting about
500-1000 IOPS (according to zpool iostat) and 20-30MB/sec in
random read on a big database. Does that sounds right?
At first, it seems low. Can you try to benchmark that with bonnie++ or
iozone? You should be getting tenfold of that performance even for
database usage
Vennlige hilsener / Best regards
roy
Actually, that sound pretty much what should be expected. 7200RPM SATA
drives aren't going to be able to do more than 100 IOPS or so each under
ideal conditions. So, the best case for your setup would be 1400 IOPS.
(each pair of mirrors is going to have slightly more IOPS than a single
disk). So, you're reporting 40-70% of maximum, which is well within
expected range.
Similarly with the read speeds. If you're reading only small amounts
each time, then your aggregate throughput isn't going to be anywhere
near the theoretical streaming read. I've found that getting more than
5-10MB/s out of a single 7200RPM drive when doing random reads isn't
possible. I would possibly expect your aggregate total to be more than
20-30MB/s, but it's still entirely possible, especially if most of your
reads are small (which only span a couple of the mirrored pairs, rather
than the full "stripe" width).
So, on the whole, I think that's about the best you can expect, given
that your workload is NOT an optimal one for the types of disk you have.
Depending on your table layout and internal data design/schema, it might
actually be to your benefit to split your disks into multiple POOLS, and
put different DB tables on different pools. I'd suggest trying it out
with 3 pools of mirrors, and see what that gets you.
And, of course, as for all database work, you need to get your DB
indexes into RAM (or on a very fast SSD). Having them on disk is going
to severely penalize your performance.
--
Erik Trimble
Java System Support
Mailstop: usca22-123
Phone: x17195
Santa Clara, CA
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
