I made a mistake in calculating the mttdl-drop for adding stripes - it
should have read:
2 disks: space=500 GB, mttdl=760.42 years, iops=158
4 disks: space=1000 GB, mttdl=380 years, iops=316
6 disks: space=1500 GB, mttdl=*253* years, iops=474
8 disks: space=2000 GB, mttdl=*190* years, iops=632
So, in my conclusion, it should have read
1. is less expensive per addition (its always just two disks)
2. not limited in number of stripes (a raidz should only hold up to 8
data disks)
3. *drops mttdl much less quickly (in fact, you'd need 12 stripes
before hitting the 8+1 mttdl)*
4. increases performance (adding disks to a raidz set has no impact)
5. increases space more slowly (the only negative - can you live with
it?)
Sorry!
Kent
Kent Watsen wrote:
I think I have managed to confuse myself so i am asking outright hoping for a straight answer.
Straight answer:
ZFS does not (yet) support adding a disk to an existing raidz set
- the only way to expand an existing pool is by adding a stripe.
Stripes can either be mirror, raid5, or raid6 (raidz w/ single or
double parity) - these striped pools are also known as raid10,
raid50, and raid60 respectively. Each stripe in a pool may be
different in both size and type - essentially, each offers space
at a resiliency rating. However, since apps can't control which
stripe their data is written to, all stripes in a pool generally
have the same amount of parity. Thus, in practice, stripes differ
only in size, which can be achieved either by using larger disks
or by using more disks (in a raidz). When stripes are of
different size, ZFS will, in time, consume all the space each
stripe offers - assuming data-access is completely balanced,
larger stripes effectively have more I/O. Regarding matching the
amount of parity in each stripe, note that a 2-disk mirror has the
same amount of parity as RAID5 and a 3-disk mirror has the same
parity as RAID6.
So, if the primary goal is to grow a pool over time by adding as few
disks as possible each time while having 1 bit of parity, you need to
plan on each time adding two disks in a mirrored configuration. Thus
your number of disks would grow like this: 2, 4, 6, 8, 10, etc.
But since folks apparently want to be able to just add disks to a
RAIDZ, lets compare that to adding 2-disk mirror stripes in terms of
impact to space, resiliency, and performance. In both cases I'm
assuming 500GB disks having a MTBF of 4 years,7,200 rpm, and 8.5 ms
average read seek.
Lets first consider adding disks to a RAID5:
Following the ZFS best-practice rule of (N+P), where N={2,4,8} and
P={1,2}, the disk-count should grow as follows: 3, 5, 9. That is,
you would start with 3, add 2, and then add 4 - note: this would
be the limit of the raidz expansion since ZFS discourages N>8.
So, the pool's MTTDL would be:
3 disks: space=1000 GB, mttdl=760.42 years, iops=79
5 disks: space=2000 GB, mttdl=228.12 years, iops=79
9 disks: space=4000 GB, mttdl=63.37 years, iops=79
Now lets consider adding 2-disk mirror stripes:
We already said that the disks would grow by twos: 2, 4, 6, 8, 10,
etc. - so the pool's MTTDL would be:
2 disks: space=500 GB, mttdl=760.42 years, iops=158
4 disks: space=1000 GB, mttdl=380 years, iops=316
6 disks: space=1500 GB, mttdl=190 years, iops=474
8 disks: space=2000 GB, mttdl=95 years, iops=632
So, adding 2-disk mirrors:
1. is less expensive per addition (its always just two disks)
2. not limited in number of stripes (a raidz should only hold up to
8 data disks)
3. drops mttdl at about the same rate (though the raidz is dropping
a little faster)
4. increases performance (adding disks to a raidz set has no impact)
5. increases space more slowly (the only negative - can you live
with it?)
Highly Recommended Resources:
http://blogs.sun.com/relling/entry/zfs_raid_recommendations_space_performance
http://blogs.sun.com/relling/entry/raid_recommendations_space_vs_mttdl
Hope that helps,
Kent
------------------------------------------------------------------------
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
