Xen, Solaris and ZFS

Posted by Thomas Wed, 27 Sep 2006 01:21:47 +0000

So, I’ve been wanting to play with ZFS for a really long time. Finally tonight I got my ducks in a row. I should have written some notes on the process, but I now have a functional OpenSolaris server running as a domU in a Debian dom0 in a little Xen cluster on my 2U.

Following (basically) the instructions on the web, I got the xen packages installed on top of my pre-existing Debian Etch install. I bought a WRT54G over the weekend, so that I could free up the 2U for more ambition endeavors. I uninstalled Shorewall so that it wouldn’t get in the way, and disabled dhcp3-server.

I installed

xen-tools xen-utils-3.0.2-1

I don’t know if I need both of the xen-utils packages… I was using Lilo (yay lilo!), but apparently Xen needs some functionality only available with Grub. So, I tried to install that, but it continually locked up during grub-install (that fix deserves its own post…). I got that fixed, and it booted fine. After configuring it for serial access, I was finally good to go. The next hurdle involved figuring out that not one of those xen packages added the necessary bridge interface to /etc/network/interfaces.

auto xenbr0
iface xenbr0 inet static
bridge_ports eth0
bridge_maxwait 0

Everybody seems to use a different name for that interface. There was at least xen-br0, but the Debian scripts that I installed are looking for xenbr0. You also have to manually bring up the bridge. Note that this will clobber the settings of eth0. So, if you were ssh’ed into the box, the ssh session will die, and, if the bridge came up properly, you can now ssh into So, if eth0′s ip was dynamically assigned, it will now be static and That took me a bit to figure out. So, once that’s done, you should be able to edit /etc/xen/xend-config.sxp, and finally do a xen-create-image and xm create that won’t error out.

So, I created a Debian image to verify that everything was working, and then I downloaded the ready to use Solaris domU image (twice because apparently I didn’t download it all the first time). I got it running, added some pseudo disks to the xen config, and tinkered with ZFS. I added me as a user and added a home directory, got the permissions all setup and even was able to mount it remotely via NFS. It’s all far from automated, but it is somewhere. I realized in all of this that I probably could export my fibre channel drives via xen to the Solaris domU. I could then put ZFS on top of them and re-export them to the entire network. It’s convoluted, I know, but it’s the best solution until Linux gets some form of ZFS. The overhead of xen isn’t supposed to be that bad, but it still seems to make for an overly complex, yet working solution. I also had thought that there wouldn’t be any reason not to be able to run apache, ftp, etc. directly on top of OpenSolaris, but dealing with OpenSolaris the little that I have, I would pull my hair out. So, just having nfs running, running all of the other value added services from nfs mounts, seems like a possible plan. You get clean snapshotting from ZFS for everybody. I guess the next hurdle is moving everything over to LDAP and Kerberos. Looking forward to it… :)

Don’t worry if you didn’t follow any of that. I guarantee you are not alone.

Posted in Technology | 2 Comments


  1. bigbrother0074 said on September 28, 2006 @ 5:38 pm:

    no, i got it, it’s ok. i got everything all the way up to the point you said: “So, I’ve been wanting to play with” and after that you lost me.

  2. KB said on September 28, 2006 @ 10:10 pm:

    I second that!

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.