Ian Collins wrote:
On 08/11/10 05:16 AM, Terry Hull wrote:
So do I understand correctly that really the "Right" thing to do is
to build
a pool not only with a consistent strip width, but also to build it with
drives on only one size? It also sounds like from a practical point of
view that building the pool full-sized is the best policy so that the
data
can be spread relatively uniformly across all the drives from the very
beginning. In my case, I think what I will do is to start with the 16
drives in a single pool and when I need more space, I'll create a new
pool
and manually move the some of the existing data to the new pool to
spread
the IO load.
That is what I have done when Thumpers fill up!
The other issue here seems to be RAIDZ2 vs RAIDZ3. I assume there is
not a
significant performance difference between the two for most loads, but
rather I choose between them based on how badly I want the array to stay
intact.
The real issue is how long large capacity drives take to resilver and
is the risk of loosing a second drive during that window high enough
to cause concern. In a lot of situations with 2TB drives, it is.
Of course your redundancy requirements may be such that you are okay
with taking that risk because you have a complete backup elsewhere that
you can restore from in case a second drive is lost during that recovery
window. Evaluate your requirements and see if it makes more sense to
dedicate more drives to make a RAIDZ3 vs RAIDZ2 (or by the same argument
RAIDZ2 vs RAIDZ1), than it does to dedicate those additional drives to
backup.
Remember that redundancy provided by RAID is not a replacement for
backup. (I get the feeling that a little bit of redundancy (RAIDZ1 or
RAIDZ2) with a little bit of backup (stripe or RAIDZ1) is the best
compromise overall, with forces such these pushing things one way or the
other: outage tolerance, time to recover from backup, and the amount of
pain induced by a complete system failure in case a non-backed up RAIDZ3
falls apart.) Running a RAIDZ3 with no backup seems like one is saying
"I really really want to protect against individual drive failures, but
if lightning strikes or the water main upstairs breaks I'll give up."
Running a main pool with a non-redundant stripe and two backups says "I
can tolerate losing my day-to-day changes to the pool since the last
backup was run, but I really don't want to lose the bulk of my stuff
which the backup protects." Running a main pool with a little bit of
redundancy (e.g. RAIDZ1) and a backup with redundancy says "I don't like
downtime nor data loss of my day-to-day changes. I'm going to guard
against single drive failures which is the most likely case, but resort
to my more time-intensive to recover backups if I hit a bad streak and
get two+ simultaneous drive failures during a long resilver. My backups
have redundancy as well because if I need to fall back upon them, then
it is because I really really need them."
E.g.with 24 x 1TB drives, some options include:
3 x 3x1TB RAIDZ1, 6TB usable for data, 9 drives + 2x (separate pools,
not striped) 7x1TB RAIDZ1 6TB usable for backup, 14 drives + 1 hot
spare. Low strength main pool. Two complete low strength backups. Hot
spare.
3 x 4x1TB RAIDZ2, 6TB usable for data, 12 drives + 2x (separate pools,
not striped) 6x1TB stripe (no redundancy) 6TB usable for backup, 12
drives. Strong main pool. Two complete but fragile backups.
3 x 4x1TB RAIDZ1, 9TB usable for data, 12 drives + 1x 12x1TB RAIDZ3 9TB
usable for backup, 12 drives. Low strength main pool. Strong backup.
3 x 5x1TB RAIDZ2, 9TB usable for data ,15 drives + 1x 9x1TB stripe (no
redundancy) 9TB usable for backup, 9 drives. Strong main pool. Fragile
backup.
3 x 6x1TB RAIDZ3, 9TB usable for data ,18 drives + 1x 6x1TB stripe (no
redundancy) 6TB usable for backup, 6 drives. Very strong main pool.
Fragile, possibly incomplete backup (depending on compression and
whether part of the directory structure is not backed up)
4 x 3x1TB RAIDZ1, 8TB usable for data, 12 drives + 1x 12x1TB RAIDZ3 9TB
usable for backup, 12 drives + 0 hot spares
4 x 3x1TB RAIDZ1, 8TB usable for data, 12 drives + 1x 10x1TB RAIDZ2 8TB
usable for backup, 10 drives + 2 hot spares. Low strength main pool.
Strong backup. Hot spares.
4 x 4x1TB RAIDZ1, 12TB usable for data, 16 drives + 1x 12x1TB stripe (no
redundancy) 12TB usable for backup, 8 drives + 0 hot spares. Low
strength main pool. Fragile backup.
4 x 4x1TB RAIDZ2, 8TB usable for data ,16 drives + 1x 8x1TB stripe (no
redundancy) 8TB usable for backup, 8 drives + 0 hot spares. Strong main
pool. Fragile backup.
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
