On Jan 5, 2013, at 9:42 AM, Russ Poyner <rpoy...@engr.wisc.edu> wrote:
> I'm configuring a box with 24x 3Tb consumer SATA drives, and wondering about
> the best way to configure the pool. The customer wants capacity on the cheap,
> and I want something I can service without sweating too much about data loss.
> Due to capacity concerns raid 10 is probably out, which leaves various raidz
> choices
You should consider space, data dependability as measured by Mean Time to Data
Loss (MTTDL), and performance.
For MTTDL[1] model, let's use 700k hours MTBF for the disks and 168 hours for
recovery (48 hours logistical + 120 hours resilver of a full disk)
For performance, lets hope for 7,200 rpms and about 80 IOPS for small, random
reads
with 100% cache miss.
>
> 1. A stripe of four 6 disk raidz2
Option 1
space ~= 4 * (6 - 2) * 3TB = 48 TB
MTTDL[1] = 8.38e+5 years or 0.000119% Annualized Failure Rate (AFR)
small, random read performance, best of worst case = 4 * (6/4) * 80
IOPS = 480 IOPS
> 2. A stripe of two 11 disk raidz3 with 2 hot spares.
Option 2
space ~= 2 * (11 - 3) * 3TB = 48 TB
MTTDL[1] = 3.62e+7 years or 0.000003% AFR
small, random read performance, best of worst case = 2 * (11/8) * 80
IOPS = 220 IOPS
Option 2a (no hot spares)
space ~= 2 * (12 - 3) * 3TB = 54 TB
MTTDL[2] = 1.90e+7 years or 0.000005% AFR
small, random read performance, best of worst case = 2 * (12/9) * 80
IOPS = 213 IOPS
>
> Other, better ideas?
There are thousands of permutations you could consider :-)
For 24-bay systems with double parity or better, we also see a 3x8-disk as a
common configuration. Offhand, I'd say we see more 4x6-disk and 3x8-disk
configs than any configs with more than 10 disks per set.
>
> My questions are
>
> A. How long will resilvering take with these layouts when the disks start
> dying?
It depends on the concurrent workload. By default resilvers are throttled and
give
way to other workload. In general, for double or triple parity RAID, you don't
need
to worry too much on a per-disk basis. The conditions you need to worry about
are
where the failure cause is common to all disks, such as a controller, fans,
cabling,
or power because they are more likely than a triple failure of disks (as
clearly shown
by the MTTDL[1] model results above)
>
> B. Should I prefer hot spares or additional parity drives, and why?
In general, addional parity is better than hot spares. You get more performance
and better data dependability.
>
> The box is a supermicro with 36 bays controlled through a single LSI 9211-8i.
> There is a separate intel 320 ssd for the OS. The purpose is to backup data
> from the customer's windows workstations. I'm leaning toward using BackupPC
> for the backups since it seems to combine good efficiency with a fairly
> customer-friendly web interface.
Sounds like a good plan.
-- richard
>
> I'm running FreeBSD 9, after having failed to get the plugin jail working in
> FreeNAS, also for personal reasons I find csh easier to use than the FreeNAS
> web interface. My impression is that FreeBSD combines a mature OS with the
> second oldest/best (after Illumos) free implementation of zfs.
>
> Thanks in advance
> Russ Poyner
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss@opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
--
richard.ell...@richardelling.com
+1-760-896-4422
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
