Had a client with a WordPress blog, using PageNavi. It suddenly broke, badly – pagination worked for specific categories, but not for the main feed (eg, index/landing page which shows all posts), or for the archives. It started off with late pages (in the 18+ range) returning page not found errors, and progressed to the point where the second page started showing the same.

Searching for info on PageNavi is like herding cats – Google will give you nothing of use, but page after page of results insisting that you just have to use PageNavi. So, this is why I get the big cheezburgrz:

The solution ended up being simple. Under the ‘Reading’ settings, the ‘Blog pages show at most’ setting was set to a ridiculously high number. Despite this, the individual pages weren’t showing more than maybe 5 posts at a time. Reduced the number to 5, and suddenly, pagination via PageNavi worked up to the final page.

Problem is, I can’t migrate their docroots. Well, I can grab most of the docroots – but curiously, there’s a number of files which are owned by apache:apache and -rx to anything but, wull, Apache.

The virtual server in question has no root access, no sudo, and no way to configure either through their control panel. Last time I had the misfortune of dealing with GoDaddy’s support… Let’s just say, I don’t have three days for another non-resolution.

What to do? Wull, it is a Drupal site…

<?php

?>

Suck it, Trebek. Suck it long, and suck it hard.

(Please note that your mileage may vary depending on how insane the host you’re trying to rescue a client from is. :p)

Apache being Apache, can do something really interesting with virtual host setups. If you access the server by a domain that isn’t explicitly mapped to a virtual host, it’ll default to the first virtual host it encountered in its configuration files. This is pretty neat, extremely useful, and – annoying in this case, as attempting to access the site via https is possible (since the domain is pointing to the system) but going to a place it should not be – the third-party application, which is the only SSL vhost set up.

Could’ve set up another vhost for SSL for the main site(s); the problem with that is Apache’s handling of SSL (IP based) and the fact that it’d require yet-another-expense – this would require a valid SSL certificate, as it’s hard to argue that going to https://mycoolsite.foo isn’t customer-facing, even if unintentional.

Solution is simple: quick cluster of rewrite rules to push anything not matching the domain of the third-party, SSL-using application, to non-https:

RewriteEngine On
RewriteCond %{HTTP_HOST} !^crazyapplication.mycoolsite.foo$ [NC]
RewriteRule (.*) http://%{HTTP_HOST}%{REQUEST_URI}

Naturally, replace crazyapplication.mycoolsite.foo with whatever.

So, what happens after the fact is this: Aside from the single domain we want covered by SSL, we’re simply redirecting https to http.

There are other ways to do this, of course, but tactical use of mod_rewrite generally avoids the backing oneself into a corner syndrome.

Yes, clearly.

RHEL/CentOS: Networking is beyond solid, albeit best configured manually in terms of setup. Xen is absolutely archaic, but at least comes with a dom0 kernel – though it seems plenty finicky around scsi devices and VNC.

Fedora: …Hung at the splash screen post-install. On a Sun Fire x4150. Seriously? Points for the upcoming Xenified kernel in Fedora 13, though.

Ubuntu: Networking seems easy enough to configure, and works, as long as you’re not going super-crazy with abstract interconnects. Xen is absolutely archaic, and there’s not even a dom0 kernel. The answer is to pull a Debian kernel (lol) or compile your own kernel, manually patching with OpenSuSE(!) patches. Yeah, no thanks.

OpenSuSE: Xen is up to date, and has a potentially super-up-to-date dom0 kernel. Networking is curiously annoying to deal with via YaST – but the real problem is, right out of the disc, I apparently can’t do ethernet bonding with Xen due to module differences.

It’s 2010.

I’m just trying to run a solid, up-to-date Xen dom0 with simple, bonded ethernet, without having to a) compile Xen myself, or b) compile a dom0 kernel myself.

It’s pointedly not 1995.

Really, if I wanted to compile common software myself, I’d step into a police box and at least get a kick-ass theme song out of it.

Well, OpenSuSE still seems like the best solution for those who are interested in non-broken, easy-to-manage 3.4.x XEN. Time to fiddle – I have no doubt that bonding works and that I’ve just missed the obvious in the installation process.

Seriously, though, I’ve never had any of these weird goddamned problems with AMD-based systems. Well, whatever.

If you randomly get the ‘Timer ISR/1: Time went backwards’ bullshit, reboot your Dom0, enter BIOS, and make sure you turn off all of Intel’s power saving nonsense. For a Sun Fire x4150, specifically, kill the following crap:

Advanced > CPU Configuration > Intel(R) SpeedStep(tm) tech
Advanced > CPU Configuration > Intel(R) C-STATE tech

With an x4150, you may end up with a weird hardware clock issue – specifically, /dev/rtc being completely inaccessible. hwclock –debug will return errno=19.

If you’re already infuriated to the point of wanting to put a server through a wall, you can do the following:

Edit /etc/rc.sysinit and /etc/init.d/halt, comment out all lines referring to hwclock
Install and configure ntp
Edit /etc/sysconfig/ntpd, add the following to the first line of options: -x -g

-x will force ntpdate to run first on startup of ntpd, ensuring your eventual completely borked time will be set before ntpd proper starts. -g will ensure that ntpd can continue to mangle your time, even if somehow your drift is several days off the mark.

Would that Xen had a more up-to-date kernel, or that RedHat hadn’t abandoned it in favor of KVM. Or would that KVM was ready for prime time. :p

This can be tossed in a .bashrc or /etc/bashrc or somewhere. :p

…Except, manually comparing files shows no difference between files in random sampling. Aside from the dos format. Yes, a file saved in a dos format will be revealed by diff as having differences. Folks, be careful what you’re editing with/how you’re saving files if you use Windows. :p

…That or make sure you have dos2unix installed (yum -y install dos2unix on real(tm) Linux; apt-get install tofrodos for those of you who are heretics) and then execute the following:

find . -type f \( -name '*.css' -o -name '*.js' -o -name '*.php' -o -name '*.module' -o -name '*.inc' -o -name '*.pl' -o -name '*.sh' \) -exec dos2unix {} \;

…That’ll run some fun happy dos2unix conversion while skipping anything that shouldn’t be touched. Hell, it’s probably skipping things that should be touched, but hey, these are the filetypes I needed to cover.

RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^/?(.*)$ http://TARGETDOMAIN/$1 [L]

Drop this sucker in, swap out ‘TARGETDOMAIN’ with where you want your 404′d files redirected to. Bah.

Server:

First, install the EPEL repository. If this article is dated, be sure to click the install link to grab more up to date instructions from their FAQ. But, for now:

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm

BTW, EPEL is a nice repository filled with all sorts of interesting software that RHEL/CentOS should have, but don’t. It’s basically all the up to date software of Fedora, without the annoyance of Fedora.

Next, you need to enable packages from EPEL for Munin:

echo 'includepkgs=munin munin-common munin-node perl-* rrdtool rrdtool-perl' >> /etc/yum.repos.d/epel.repo

Update: 4/3/2010: I’ve adjusted the includepkgs line to reflect what’s needed in recent updates – mainly, munin-common and moving the perl package includes to perl-* – there’s a lot more perl packages required. That’s not a bad thing – it’s generally better to grab perl packages in RPM form than through cpan, after all – much easier to keep up to date.

Now, install Munin:

yum -y install munin munin-node httpd

We’re installing munin-node as well as munin, so you can immediately see graphing awesomeness by monitoring your Munin server itself. Apache (or some webserver) is necessary to actually view your Munin data. Anyhoo, once installed, edit a few things. Mostly:

/etc/munin/munin.conf

Find the localhost entry and replace it with your monitoring server’s hostname. EG:

[lemuria.taldar.com]
    address 127.0.0.1
    use_node_name yes

Add in an alias to your apache config so you can, you know, see Munin:

echo 'Alias /munin /var/www/html/munin' >> /etc/httpd/conf/httpd.conf
/etc/init.d/httpd graceful

You may want to put that in a specific virtual host, if you have them – or even set up .htaccess protection. That’s outside the scope of this post, however.

Finally, start munin-node and ensure it starts on booth:

/etc/init.d/munin-node start
chkconfig munin-node on

Open your browser, navigate to your Munin installation (http://yourserver/munin if following the listed procedure), and… Chances are, you’ll have to wait a few minutes for Munin to actually collect data. But in a short time, you should have more pretty graphs than you ever dreamed possible.

Client:

Setting up a client is just as easy – you can skip the ‘munin’ part and just install ‘munin-node’. Once done, check:

/etc/munin/munin-node.conf

…for settings. For the most part, all you really need to adjust is:

allow ^127\.0\.0\.1$

Replace 127.0.0.1 with the IP of your munin server. Save, quit, start the node:

/etc/init.d/munin-node start
chkconfig munin-node on

Back on the server, open up:

/etc/munin/munin.conf

…And add an entry for your new client. Example:

[fatov.taldar.com]
    address 192.168.42.69
    use_node_name yes

No need to restart anything on the server, since Munin’s monitoring is cron-based.

If you’ve got iptables running, you probably need to add a rule to allow the client to accept traffic from the server. Open up:

/etc/sysconfig/iptables

..And add:

-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --source IPOFYOURMUNINSERVER --dport 4949 -j ACCEPT

Then restart iptables:

/etc/init.d/iptables restart

Note the –source line; this should restrict tcp/4949 to your Munin server. Since nothing else should be talking to your client on 4949, you might as well go the distance in keeping things secure.

Enabling More Stuff

You’ll notice that different servers might have different graphs – you might see MySQL show up on a database server, while it doesn’t show up on a webserver. When you install munin-node, the installer does some checking and enables plugins that seem to make sense. What if you want more stuff? Simply check:

/usr/share/munin/plugins/

…And see what’s available. To enable something – we’ll use ‘netstat’ as example, you just make a symlink in the Munin config plugins directory. Example:

ln -s /usr/share/munin/plugins/netstat /etc/munin/plugins/netstat

Restart the node after this:

/etc/init.d/munin-node start

MySQL

You might have a MySQL plugin enabled by default, but – by default, it won’t pull anything of use. To remedy this, hop on your node, and open up:

/etc/munin/plugin-conf.d/munin-node

Add or modify the following:

[mysql*]
env.mysqlopts -u munin -pSEKRITPASSWORDSHHH

Replace ‘SEKRITPASSWORD…’ with some sort of password. Note the MySQL user (-u munin) you’ll be creating is fairly unprivileged. Speaking of which, let’s create that:

mysql
create user munin@'localhost' identified by 'SEKRITPASSWORDSHHH';
flush privileges;
quit;


…Again, replace the password. After all this – you guessed it, restart the node:

/etc/init.d/munin-node start

Shortly thereafter you should start getting MySQL data on the server.

Thankfully, the cables leading to the drive backplane are just barely capable of reaching the middle slot. Helps to pull them and reroute them – you can get them to fit snugly but not ridiculously tight.

Blah.