It’s a good feeling knowing that I can still bend ldap, kerberos, and radius to my will. :P Squid gave me some trouble; since it logs to a strange place, it took me longer than it should to find a configuration problem. By and large the important pieces are working again, and all running shiny new 3.2 kernels (the first time I’ve used a 3+ kernel). I’ll consider this my late contribution to the world backup day, since really I should have moved off of the hard drive in my domU ages ago (since it always seems to return errors whenever I try to backup xen images). I really should go ahead and update my domU to a new fanless box that doesn’t take 10 minutes to boot up. I’ve been eying some mini-itx fanless boxes, which would allow me to migrate the important things over to new hardware before the old hardware fails… But all in all if I only have to spend one weekend every 2+ years doing (non-filer) admin work on the network, I think I have to consider that super win.

I should have been taking better notes, but here are some things I took away from the weekend:

• squid3: Setting “error_directory /usr/share/squid-langpack/en” since it was looking in /usr/share/squid3/errors/templates/ for a reason I still haven’t figure out
• installing testing debian: requires stupid ethernet firmware. REally?1? I understand the whole free software thing, but this was a little too pure. Reminds me of having to put ethernet drivers/modules on floppy during an install. Made me think, “man, I really shouldn’t have to do this”. Ditto for the netboot initrd not having the hard disk controller drivers I needed forcing me to connect up a cdrom. I gave up on getting serial working, as it’s never worked out as well as I’d liked and always seems to bite me when I need to debug (e.g. no vga console output when I need it most).
• dist-upgrade: I’d like to think I could have upgraded the domU all the way from like 4.0 or whatever it was to current testing, but I just didn’t have the heart. I figured it’d take me less time to just install from scratch than to fight with xen, udev, libc, grub, grub2, device naming, etc, etc. I used to think that you always could dist-upgrade a box and it would usually work out ok. Now I’m not so sure, since it seems like there have been a huge amount of changes, which make automagic dist-upgrading very painful.
• backups: I really should do a better job. Maybe one day it’ll bite me hard enough I’ll do better — that or it’ll teach me that the data wasn’t that important in the first place. :)
• ldap: There still seems like the current version of ldap has ssl/tls issues since it’s compiled against gnutls (http://bugs.debian.org/645810). I’m too lazy to fight with this one at this point, so I just disabled tls on both the client and serve side. I’d like to have tls working, but it’s not the end of the world. I did have to fight with the upgrade process to get it to complete. Though I think the data should be cleaner now, since there was some cruft in there that I was able to remove. It did take quite a bit of finagling, tho…
• xen: Installing the latest version of xen directly from a fresh install seemed pretty painless (though I don’t recall it being super painful before, either). After a while I figured out the steps required to upgrade the dom0′s, so eventually it went smoothly (but did take some time to upgrade many++ packages to be able to install the correct kernel).
• kerberos: Since I had to restore my auth xen image from backups, the kerberos database was the only thing that I realized halfway into all of this had data that I really needed from the current copy (current passwords, machine credentials). A chroot + kerberos dump/restore fixed it up nicely. Dodged a bullet on that one.
• freeradius: I finally got this working this morning. I thought it was fixed last night, but this morning my phone wasn’t connected to wifi. After a while I realized that the config was borked. Since my config was pretty old, it looks like there might be more standard ways to configure ldap, but after some time trying what appeared to be their way, I just reverted back to my old configs which did work with the current version of freeradius. I wish it was a little less hacky (dummy certs and not the “recommended” way to setup ldap), but hey whatever works. :)
• homogeneity: While flipping back and forth between vms, I realized that several common configs (ldap, apt.conf, sources.list) varied some. I think my plan was to get them all synced via puppet, but since that never got off the ground I really should sync them up now…
• ipv6: It continues to work out of the box, but I realized that sometimes I was doing apt-get downloads through my ipv6 tunnel, which means that it’s traversing the US — which explains why sometimes the downloads were slower than expected. One of these days I’ll migrate to a closer endpoint. :)
• linux 3.2: I was a little apprehensive that everything (joe random userland tool) would work with a 3.2 kernel. Haven’t yet found anything broken by it.

My actual notes for upgrading dom0 vms:

# update console to listen on hvc0 instead of tty0 (before
# starting the vm)
mount /path/to/image /mnt/loop0
vim /mnt/loop0/etc/inittab

# start the vm
xm create -c image.cfg

# add hvc0 to the list of allowable local consoles
# (so root can login locally);
# h/t to http://docs.quantact.com/xen-fs-changes
# (even though I didn’t wind up updating fstab or mknod’ing anything)
echo “hvc0″ >> /etc/securetty

# update sources.list to use testing
vim /etc/apt/sources.list

# sync to testing
apt-get update

# these seem broken w/ the new kernel, so upgrade them first so
# they don’t break other apt installs
apt-get install findutils debconf

# ditto my older version of cpio caused trouble during the
# kernel install
apt-get install cpio

# install the kernel so when things depmod it won’t spew a warning
apt-get install linux-image-3.2.0-2-686-pae