On Thu, 2010-06-03 at 12:22 -0400, Dennis Clarke wrote:
> > If you're clever, you'll also try to make sure each side of the mirror
> > is on a different controller, and if you have enough controllers
> > available, you'll also try to balance the controllers across stripes.
>
> Something like this ?
>
> # zpool status fibre0
> pool: fibre0
> state: ONLINE
> status: The pool is formatted using an older on-disk format. The pool can
> still be used, but some features are unavailable.
> action: Upgrade the pool using 'zpool upgrade'. Once this is done, the
> pool will no longer be accessible on older software versions.
> scrub: none requested
> config:
>
> NAME STATE READ WRITE CKSUM
> fibre0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c2t16d0 ONLINE 0 0 0
> c5t0d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c5t1d0 ONLINE 0 0 0
> c2t17d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c5t2d0 ONLINE 0 0 0
> c2t18d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c2t20d0 ONLINE 0 0 0
> c5t4d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c2t21d0 ONLINE 0 0 0
> c5t6d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c2t19d0 ONLINE 0 0 0
> c5t5d0 ONLINE 0 0 0
> spares
> c2t22d0 AVAIL
>
> errors: No known data errors
That looks like a good configuration to me!
>
> However, unlike the bad old days of SVM ( DiskSuite or Solstice Disksuite
> or Online Disk Suite etc ) I have no idea what algorithm is used to pick
> the hot spare in the event of a failure. I mean, if I had more than one
> hotspare there of course. Also, I think the weird order of controllers is
> a user mistake on my part. Some of them have c5 listed first and others
> have c2 listed first. I don't know if that matters at all however.
I don't think the order matters. It certainly won't make a difference
for write, since you have to use both sides of the mirror. It *could*
make a difference for reading... but I suspect that zfs will try to
sufficiently balance things out that any difference in ordering will be
lost in the noise.
The hotspare replacement shouldn't matter all that much... except when
you're using them to upgrade to bigger drives. (Then just use a single
hot spare to force the selection.)
>
> I can add mirrors on the fly but I can not ( yet ) remove them. I would
> imagine that the algorithm to remove data from vdevs would be fairly
> gnarly.
Indeed. However, you shouldn't need to remove vdevs. With redundancy,
you can resilver to a hot spare, and then remove the drive, but
ultimately, you can't condense the data onto fewer drives. You have to
accept when you configure your array that, modulo hot spares, your pool
will always consume the same number of spindles.
>
> The item that I find somewhat confusing is how to apply multi-path fibre
> devices to a stripe of mirrors. Consider these :
>
>
> This is not disk redundency but rather fibre path redundency. When I drop
> these guys into a ZPool it looks like this :
>
> NAME STATE READ WRITE CKSUM
> fp0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c4t20000004CFBFD4BDd0s0 ONLINE 0 0 0
> c4t20000004CFA4D3A1d0s0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c4t20000004CFA4D2D9d0s0 ONLINE 0 0 0
> c4t20000004CFA4D2C7d0s0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c4t20000004CFA4D655d0s0 ONLINE 0 0 0
> c4t20000004CF9B63D0d0s0 ONLINE 0 0 0
The above configuration looks good to me.
>
> So the manner in which any given IO transaction gets to the zfs filesystem
> just gets ever more complicated and convoluted and it makes me wonder if I
> am tossing away performance to get higher levels of safety.
If you're using multipathing, then you get path load balancing
automatically, and you can pretty much ignore the controller balancing
issue, as long as you use the mpxio (scsi_vhci) path. mpxio should take
care of ensuring that I/O is balanced across ports for you; you just
need to make sure that you are balancing *spindles* properly.
-- Garrett
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
