Seriously, though, I’ve never had any of these weird goddamned problems with AMD-based systems. Well, whatever.

If you randomly get the ‘Timer ISR/1: Time went backwards’ nonsense, reboot your Dom0, enter BIOS, and make sure you turn off all of Intel’s power saving nonsense. For a Sun Fire x4150, specifically, kill the following crap:

Advanced >
CPU Configuration > Intel(R) SpeedStep(tm) tech
Advanced > CPU Configuration > Intel(R) C-STATE tech

With an x4150, you may end up with a weird hardware clock issue – specifically, /dev/rtc being completely inaccessible. hwclock –debug will return errno=19.

If you’re already infuriated to the point of wanting to put a server through a wall, you can do the following:

Edit /etc/rc.sysinit and /etc/init.d/halt, comment out all lines referring to hwclock
Install and configure ntp
Edit /etc/sysconfig/ntpd, add the following to the first line of options: -x -g

-x will force ntpdate to run first on startup of ntpd, ensuring your eventual completely borked time will be set before ntpd proper starts. -g will ensure that ntpd can continue to mangle your time, even if somehow your drift is several days off the mark.

Would that Xen had a more up-to-date kernel, or that RedHat hadn’t abandoned it in favor of KVM. Or would that KVM was ready for prime time. :p

Thankfully, the cables leading to the drive backplane are just barely capable of reaching the middle slot. Helps to pull them and reroute them – you can get them to fit snugly but not ridiculously tight.

Blah.

Linux

36    37   | 38    39   | 40    41   | 42    43   | 44    45   | 46    47
sdab  sdaf | sdt   sdx  | sdar  sdav | sdaj  sdan | sdl   sdp  | sdd   sdh
-----------+------------+------------+------------+------------+-----------
24    25   | 26    27   | 28    29   | 30    31   | 32    33   | 34    35
sdaa  sdae | sds   sdw  | sdaq  sdau | sdai  sdam | sdk   sdo  | sdc   sdg
-----------+------------+------------+------------+------------+-----------
12    13   | 14    15   | 16    17   | 18    19   | 20    21   | 22    23
sdz   sdad | sdr   sdv  | sdap  sdat | sdah  sdal | sdj   sdn  | sdb   sdf
-----------+------------+------------+------------+------------+-----------
00    01   | 02    03   | 04    05   | 06    07   | 08    09   | 10    11
sdy   sdac | sdq   sdu  | sdao  sdas | sdag  sdak | sdi   sdm  | sda   sde
 BOOT DEV

Solaris

36    37   | 38    39   | 40    41   | 42    43   | 44    45   | 46    47
c3t3  c3t7 | c2t3  c2t7 | c5t3  c5t7 | c4t3  c4t7 | c1t3  c1t7 | c0t3  c0t7
-----------+------------+------------+------------+------------+-----------
24    25   | 26    27   | 28    29   | 30    31   | 32    33   | 34    35
c3t2  c3t6 | c2t2  c2t6 | c5t2  c5t6 | c4t2  c4t6 | c1t2  c1t6 | c0t2  c0t6
-----------+------------+------------+------------+------------+-----------
12    13   | 14    15   | 16    17   | 18    19   | 20    21   | 22    23
c3t1  c3t5 | c2t1  c2t5 | c5t1  c5t5 | c4t1  c4t5 | c1t1  c1t5 | c0t1  c0t5
-----------+------------+------------+------------+------------+-----------
00    01   | 02    03   | 04    05   | 06    07   | 08    09   | 10    11
c3t0  c3t4 | c2t0  c2t4 | c5t0  c5t4 | c4t0  c4t4 | c1t0  c1t4 | c0t0  c0t4
 BOOT DEV

OpenSolaris

36    37    | 38    39    | 40    41    | 42    43    | 44    45   | 46    47
c11t3 c11t7 | c10t3 c10t7 | c13t3 c13t7 | c12t3 c12t7 | c8t3  c8t7 | c7t3  c7t7
------------+-------------+-------------+-------------+------------+-----------
24    25    | 26    27    | 28    29    | 30    31    | 32    33   | 34    35
c11t2 c11t6 | c10t2 c10t6 | c13t2 c13t6 | c12t2 c12t6 | c8t2  c8t6 | c7t2  c7t6
------------+-------------+-------------+-------------+------------+-----------
12    13    | 14    15    | 16    17    | 18    19    | 20    21   | 22    23
c11t1 c11t5 | c10t1 c10t5 | c13t1 c13t5 | c12t1 c12t5 | c8t1  c8t5 | c7t1  c7t5
------------+-------------+-------------+-------------+------------+-----------
00    01    | 02    03    | 04    05    | 06    07    | 08    09   | 10    11
c11t0 c11t4 | c10t0 c10t4 | c13t0 c13t4 | c12t0 c12t4 | c8t0  c8t4 | c7t0  c7t4
 BOOT DEV

Nexenta

36    37   | 38    39   | 40    41   | 42    43   | 44    45   | 46    47
c4t3  c4t7 | c3t3  c3t7 | c6t3  c6t7 | c5t3  c5t7 | c1t3  c1t7 | c0t3  c0t7
-----------+------------+------------+------------+------------+-----------
24    25   | 26    27   | 28    29   | 30    31   | 32    33   | 34    35
c4t2  c4t6 | c3t2  c3t6 | c6t2  c6t6 | c5t2  c5t6 | c1t2  c1t6 | c0t2  c0t6
-----------+------------+------------+------------+------------+-----------
12    13   | 14    15   | 16    17   | 18    19   | 20    21   | 22    23
c4t1  c4t5 | c3t1  c3t5 | c6t1  c6t5 | c5t1  c5t5 | c1t1  c1t5 | c0t1  c0t5
-----------+------------+------------+------------+------------+-----------
00    01   | 02    03   | 04    05   | 06    07   | 08    09   | 10    11
c4t0  c4t4 | c3t0  c3t4 | c6t0  c6t4 | c5t0  c5t4 | c1t0  c1t4 | c0t0  c0t4
  BOOT DEV

…s’alot of disks.

First, download the appropriate drivers for your FC HBA. I’m currently using a QLogic card, and you should be, too. If you’re not, your mileage and sanity may vary. At any rate, a quick trip to the QLogic site provided me with qla2xxx-v8.02.23_4-dist.tgz, which I shoved into /opt.

The thing of primary importance are the Linux rdac drivers. The rdac drivers they provide are out of date and will not compile against any modern kernel. You can snag the updated rdac drivers from: http://lsi.com/rdac/rdac-LINUX-09.02.C5.13-source.tar.gz

Those, too, go into /opt.

Now on with the awesome:

cd /opt
tar -zxf qla2xxx*.tgz
cd qlogic
./drvrsetup
cd qla2xxx-8*
./extras/build.sh install
cd /opt
tar -zxf rdac-LINUX-09.02.C5.13-source.tar.gz
cd linuxrdac-09.02.C5.13
make
make install
y

Open up /boot/grub/grub.conf.

Copy your existing Grub entry and paste it at the bottom of your file. This is your fallback. For the primary entry, modify the initrd line to point to mpp-YOURKERNELVERSION.img. It should look similar to the following:

initrd /mpp-2.6.18-128.1.14.el5.img

Of course, if your kernel is a different version, the numbers will be different.

Open up /etc/multipath.conf and do some pasting:

blacklist {
        devnode "*"
}
devices {
        device {
                vendor SUN
                product CSM200_R
                product_blacklist "Universal Xport"
                features "1 queue_if_no_path"
                path_checker rdac
                hardware_handler "1 rdac"
                path_grouping_policy group_by_prio
                prio "rdac"
                failback immediate
        }
}

This may need to be modified depending on what you want/need; I’m rather new to playing with the 6140 myself so I can’t really make any recommendations. This multipath.conf was actually provided to me by our contacts at Ibrix. Ibrix, as a business, is just pure awesome. They’re on the level of Zeus in terms of willingness to assist potential customers. Absolutely amazing.

Anyhow, make sure you have multipathd starting on boot:

chkconfig multipathd on

Reboot, and you should be able to mount exports from your 6140 on your RHEL/CentOS 5.3 system, and do proper failover, without things freaking the hell out.

Enter the eLOM CLI via the Srv Mgt port.

cd /SP/network
set IPAddress=172.16.200.6
set Gateway=172.16.200.1

Once done, you should be able to access the web management GUI via the IP entered. In this example, 172.16.200.6.

iLOM:

Enter the iLOM CLI via the Srv Mgt port.

cd /SP/network
set pendingipaddress=172.16.200.6
set pendingipgateway=172.16.200.1
set commitpending=true

Once done, you should be able to access the web management GUI via the IP entered. In this example, 172.16.200.6.