The scsi-target-utils package distributed with RHEL/CentOS 5.3 is okay for simple stuff, but definitely out of date. And it completely fails when attempting to boot diskless systems via iSCSI over Infiniband. Spent quite a bit of time tonight attempting to figure out why BoIB 2.0.0 with a ConnectX card refused to communicate with my apparently working (export connectable/useable on other machines) iSCSI volume. Replaced tgtadm with the enterprise iSCSI target, and boom – instant satisfaction.

First, download the appropriate drivers for your FC HBA. I’m currently using a QLogic card, and you should be, too. If you’re not, your mileage and sanity may vary. At any rate, a quick trip to the QLogic site provided me with qla2xxx-v8.02.23_4-dist.tgz, which I shoved into /opt.

The thing of primary importance are the Linux rdac drivers. The rdac drivers they provide are out of date and will not compile against any modern kernel. You can snag the updated rdac drivers from: http://lsi.com/rdac/rdac-LINUX-09.02.C5.13-source.tar.gz

Those, too, go into /opt.

Now on with the awesome:

cd /opt
tar -zxf qla2xxx*.tgz
cd qlogic
./drvrsetup
cd qla2xxx-8*
./extras/build.sh install
cd /opt
tar -zxf rdac-LINUX-09.02.C5.13-source.tar.gz
cd linuxrdac-09.02.C5.13
make
make install
y

Open up /boot/grub/grub.conf.

Copy your existing Grub entry and paste it at the bottom of your file. This is your fallback. For the primary entry, modify the initrd line to point to mpp-YOURKERNELVERSION.img. It should look similar to the following:

initrd /mpp-2.6.18-128.1.14.el5.img

Of course, if your kernel is a different version, the numbers will be different.

Open up /etc/multipath.conf and do some pasting:

blacklist {
        devnode "*"
}
devices {
        device {
                vendor SUN
                product CSM200_R
                product_blacklist "Universal Xport"
                features "1 queue_if_no_path"
                path_checker rdac
                hardware_handler "1 rdac"
                path_grouping_policy group_by_prio
                prio "rdac"
                failback immediate
        }
}

This may need to be modified depending on what you want/need; I’m rather new to playing with the 6140 myself so I can’t really make any recommendations. This multipath.conf was actually provided to me by our contacts at Ibrix. Ibrix, as a business, is just pure awesome. They’re on the level of Zeus in terms of willingness to assist potential customers. Absolutely amazing.

Anyhow, make sure you have multipathd starting on boot:

chkconfig multipathd on

Reboot, and you should be able to mount exports from your 6140 on your RHEL/CentOS 5.3 system, and do proper failover, without things freaking the hell out.

Set new UUID on XFS Partition, RHEL/CentOS:

xfs admin -U generate /dev/sdb1

Set new UUID on XFS Partition, SLES/OpenSUSE:

xfs_admin -U generate /dev/sdb1

Set new UUID on ext3 Partition and Display New UUID (RHEL/CentOS, SLES/OpenSUSE):

tune2fs /dev/sdb1 -U random
dumpe2fs -h /dev/sdb1 | grep UUID

Fun, no?