On Jan 21, 2011, at 6:41 PM, Edward Morbius <dredmorb...@gmail.com> wrote:
> We've been wrestling with this for ... rather longer than I'd care to admit. > > Host / initiator systems are a number of real and virtualized CentOS 5.5 > boxes. Storage arrays / targets are Dell MD3220i storage arrays. > > CentOS is not a Dell-supported configuration, and we've had little helpful > advice from Dell. There's been some amount of FUD in that Dell don't seem to > know what Dell's own software installation (the md3 > > Dell doesn't seem to have much OS experience generally. > > Their docs are pretty inconsistent. I've noted omissions, terminology > differences, and procedural differences among the "Owner's Manual", > "Deployment Guide", a professional services "Remote Services Installation > Agreement" service description. Some of the multipathing guidance we've had > comes from their EqualLogic line of storage servers. > > Questions: > > 1: Is there anyone out there running this configuration and are you satisfied > with it? > > 2: We get a set of error messages on the initator at target login. These > appear to be benign, and web research suggests it's the result of a driver > configuration issue in trying to send instructions to a > > http://comments.gmane.org/gmane.linux.iscsi.open-iscsi/5970 > http://bit.ly/gguLl7 > > MD3000i boxes have 1 controller that can execute IO and one that > cannot execute READ/WRITE IO until a special command is sent. In > older kernels, layers like the partition scanning and udev and > hal would send down IO to those disabled paths and we would see > IO errors like in your link. > > In newer kernels we have device handler modules (for MD3000i you > would want scsi_dh_rdac (so do "modprobe scsi_dh_rdac" and then > lsmod to see if it is there)) that will detect if the path is > not active and if so it will either not send the IO or it will > not print a error message since we expect the IO to fail. > > We'd prefer *not* seeing spurious I/O errors that we don't have to sift > through looking for real storage issues. > > 3: The MD32xxi series is a dual-controller array with multiple ports on each > controller. Multiple targets can be logged into from an initiator, with the > pathways aggregated by the Linux multipath (device-mapper-multipath) system. > There's very little clear documentation on multipath. In particular, any way > to trigger alerting from multipath events / failures, or iscsi session > actions, would be helpful. The MD32xxi series only supports reporting from a > GUI management utility (MDSM) which would be at best problematic to run in a > server environment. The other question is: is multipathing typical of iSCSI > configuation? Little of the iSCSI docs I've found discusses multipath > configurations at all: > > http://www.open-iscsi.org/ > http://www.cuddletech.com/articles/iscsi/index.html (good but very dated) > http://www.cyberciti.biz/tips/rhel-centos-fedora-linux-iscsi-howto.html (no > mention of multipathing) > > 4: Dell suggests a shutdown procedure including a flush of multipathing paths > ("multipath -F" -- in the "Owner's Manual"): > > 3 Flush the Device Mapper multipath maps list to remove any old or > modified mappings > # multipath F > > Presumably this would go into one of the init scripts -- perhaps the > /etc/init.d/multipath script, as part of the "stop" sequence (after the > multipath daemon is killed). Anyone done this or know why the practice is > recommended? > > Based on our experience with Dell I would NOT recommend this configuration > for others. But we're stuck with it, and any help in getting things > configured would be very helpful and gratefully received. > > I'm also hoping to get clearance to release docs we've generated, though > that's the subject of some internal negotiation. You need the RDAC kernel module installed, this handles asymmetric multipathing to these devices. You can get this from Dell's site. Once this is installed you need to setup dm-multipath, look for multipathd.conf in /etc, get the product id and vendor id from dmesg after making an initial connection via open-iscsi and use that in the mutipath config. Your going to need to use path utility 'rdac' in the config instead of tur. Google is your friend here. -Ross
_______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
