On Tue, Nov 01, 2011 at 10:24:06PM +0200, Alexander Motin wrote: > On 01.11.2011 19:50, Dennis K?gel wrote: > > Not sure if replying on-list or off-list makes more sense... > > Replying on-list could share experience to other users. > > > Anyway, some first impressions, on stable/9: > > > > The lab environment here is a EMC VNX / Clariion SAN, which has two Storage > > Processors, connected to different switches, connected to two isp(4)s on > > the test machine. So at any time, the machine sees four paths, but only two > > are available (depending on which SP owns the LUN). > > > > 580# camcontrol devlist > > <DGC VRAID 0531> at scbus0 target 0 lun 0 (da0,pass0) > > <DGC VRAID 0531> at scbus0 target 1 lun 0 (da1,pass1) > > <DGC VRAID 0531> at scbus1 target 0 lun 0 (da2,pass2) > > <DGC VRAID 0531> at scbus1 target 1 lun 0 (da3,pass3) > > <COMPAQ RAID 1(1VOLUME OK> at scbus2 target 0 lun 0 (da4,pass4) > > <COMPAQ RAID 0 VOLUME OK> at scbus2 target 1 lun 0 (da5,pass5) > > <hp DVD D DS8D3SH HHE7> at scbus4 target 0 lun 0 (cd0,pass6) > > > > I miss the ability to "add" disks to automatic mode multipaths, but I (just > > now) realized this only makes sense when gmultipath has some kind of path > > checking facility (like periodically trying to read sector 0 of each > > configured device, this is was Linux' devicemapper-multipathd does). > > In automatic mode other paths supposed to be detected via metadata > reading. If in your case some paths are not readable, automatic mode > can't work as expected. By the way, could you describe how your > configuration supposed to work, like when other paths will start > working?
Without knowledge of the particular Clariion SAN Dennis is working with, I've seen some so-called active/active RAID controllers force a LUN fail over from one controller to another (taking it offline for 3 seconds in the process) because the LUN received an I/O down a path to the controller that was formerly taking the standby role for that LUN (and it was per-LUN, so some would be owned by one controller and some by the other). During the controller switch, all I/O to the LUN would fail. Thankfully that particular RAID model where I observed this behaviour hasn't been sold in several years, but I would tend to expect such behaviour at the lower end of the storage market with the higher end units doing true active/active configurations. (and no, I won't name the manufacturer on a public list) This is exactly why Linux ships with a multipath configuration file, so it can describe exactly what form of brain damage the controller in question implements so it can work around it, and maybe even document some vendor-specific extensions so that the host can detect which controller is taking which role for a particular path. Even some controllers that don't have pathological behaviour when they receive I/O down the wrong path have sub-optimal behaviour unless you choose the right path. NetApp SANs in particular typically have two independant controllers with a high-speed internal interconnect, however there is a measurable and not-insignificant penalty for sending the I/O to the "partner" controller for a LUN, across the internal interconnect (called a "VTIC" I believe) to the "owner" controller. I've been told, although I have not measured this myself, that it can add several ms to a transaction, which when talking about SAN storage is potentially several times what it takes to do the same I/O directly to the controller that owns it. There's probably a way to make the "partner" controller not advertise the LUN until it takes over in a failover scenario, but every NetApp I've worked with is set (by default I believe) to advertise the LUN out both controllers. Gary _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
