Solaris on Xen -- a Linux domU without installing Linux

Update: Bernd Schemmer discovered that newer Xen demo CDs appear to have a 32 bit PAE kernel as the default. This won’t work with a Solaris dom0, as we don’t support PAE mode with Xen. Bernd kept a copy of the older kernel that works.

If you’re interested in Solaris on Xen then by now you should be downloading the recently release snapshot of our work from the OpenSolaris Xen community site. We’ve made quite a bit of progress since the last snapshot, and it’s our intention that they will be more frequent from now on.

If you do have your Solaris domain 0 (dom0) up and running, perhaps you’re wondering about running something other than Solaris as a guest (domU). Linux is of common interest, especially if there’s something that BrandZ doesn’t quite do for you (for example, you need to use a particular Linux kernel module). Creating disk images for non-native domU guests can be quite a painful process, usually involving installing the OS on a physical machine and then building the guest environment from that. In the Solaris case we’re providing vbdcfg to help people get started, but that doesn’t help if you have a Solaris dom0 and want a Linux domU.

Here’s a quick way to get something running as a test: grab the Xen demo CD and extract the Linux kernel and filesystem image. It’s not particularly difficult to do, but if you only have Solaris machines, it’s more involved than you’d expect.

First, you’ll need to get a copy of the Xen demo CD itself. Go to the XenSource downloads page and choose either HTTP or bittorrent. For HTTP you have to supply some details, which is why there’s no direct download link here. After the download you’ll have an ISO image called xen-3.0-demo-livecd-3.0.iso.

We’ll assume that you have some free diskspace in a directory called /space, and that everything is going to go there. Using these instructions will require around 4 gigabytes of space. If you choose to run multiple linux guest machines this way, each one will add another 1.5 gigabytes or so.

Once you have the CD image, there’s still some work to do. The CD itself uses an extended form of the the ISO filesystem called zisofs. This allows the contents of the CD to be compressed, saving space, but still provide ‘random access’ for easy use. As you might guess, this requires an implementation of the ISO filesystem code that supports zisofs, which at the moment Solaris doesn’t have. Linux does have an implementation, so you could always extract the bits you need there if you have a Linux machine up and running.

For Solaris, we can use the user-level zisofs-tools to read the compressed file from the image. Here’s roughly how that goes:

1. Get a copy of the latest zisofs-tools:

   $ wget ftp://ftp.kernel.org/pub/linux/utils/fs/zisofs/zisofs-tools-1.0.7.tar.gz
   

2. Extract and build:

   $ cd /space
   $ gzcat zisofs-tools-1.0.7.tar.gz | tar xf -
   $ cd zisofs-tools-1.0.7
   $ CFLAGS="-DNAME_MAX=1024" ./configure
   $ gmake
   

   Now you should have a mkzftree utility. This is used to create new zisofs images and, more relevant, extract the contents of an existing image. Put mkzftree somewhere in your PATH, so that you can find it later.

3. Mount the Xen demo ISO image:

   # lofiadm -a /space/xen-3.0-demo-livecd-3.0.iso
   /dev/lofi/1
   # mount -r -F hsfs /dev/lofi/1 /mnt
   # ls /mnt
   boot           boot.catalog   centos-fs.img  REVISION       rootfs.img
   #
   

4. Extract the contents of the CD, uncompressing as we go:

   # cd /space
   # mkzftree -uv /mnt xen-demo
   /mnt/boot -> xen-demo/boot
   /mnt/boot/config-2.6.12.6-xen -> xen-demo/boot/config-2.6.12.6-xen
   /mnt/boot/grub -> xen-demo/boot/grub
   /mnt/boot/grub/default -> xen-demo/boot/grub/default
   /mnt/boot/grub/device.map -> xen-demo/boot/grub/device.map
   /mnt/boot/grub/e2fs_stage1_5 -> xen-demo/boot/grub/e2fs_stage1_5
   /mnt/boot/grub/fat_stage1_5 -> xen-demo/boot/grub/fat_stage1_5
   …
   /mnt/centos-fs.img -> xen-demo/centos-fs.img
   /mnt/REVISION -> xen-demo/REVISION
   /mnt/rootfs.img -> xen-demo/rootfs.img
   #
   

   This process can take a while - the resulting .img files are something around 1.5 gigabytes.

5. Unmount, etc.:

   # umount /mnt
   # lofiadm -d /dev/lofi/1
   

Now that we have the contents of the demo CD extracted, we can go ahead and create a virtual machine based around the contents. Given that the guest domain is going to want to modify the root filesystem, we take a copy:

$ cd /space
$ mkdir xen-centos1
$ cp xen-demo/boot/vmlinuz-2.6-xen xen-centos1
$ cp xen-demo/centos-fs.img xen-centos1/rootfs.img
$ chmod 666 xen-centos1/rootfs.img

When all of the copying is completed it’s time to create the configuration file for our new domain. Start with something simple and work forward - it’s easier to debug any problems that way. In this case, we’ll assume that the Xen tools can choose an appropriate MAC address for the domain, that DHCP works on our local network and that the system we’re using for domain 0 has already been configured with the relevant settings for the software ethernet bridge (see the how to guide to get this far!).

The configuration file is straightforward:

$ cd /space
$ cat xen-centos1/xen-centos1.py
kernel = “/space/xen-centos1/vmlinuz-2.6-xen”
memory = 256
name = “xen-centos1”
disk = [‘file:/space/xen-centos1/rootfs.img,sda1,w’]
vif = [ ’’ ]
root = “/dev/sda1 ro”
$

Now we can try to boot our guest domain:

# cd /space
# xm create -c xen-centos1/xen-centos1.py
Using config file “xen-centos1/xen-centos1.py”.
Started domain xen-centos1
Linux version 2.6.12.6-xen (apeace@paploo) (gcc version 3.4.4 20050721
(Red Hat 3.4.4-2)) #1 SMP Sun Dec 4 18:37:30 GMT 2005
BIOS-provided physical RAM map:
Xen: 0000000000000000 - 0000000010000000 (usable)
0MB HIGHMEM available.
264MB LOWMEM available.
ACPI in unprivileged domain disabled
…
Xen virtual console successfully installed as tty1
Event-channel device installed.
netfront: Initialising virtual ethernet driver.
…
INIT: version 2.85 booting
Welcome to  CentOS release 4.1 (Final)
Press ‘I’ to enter interactive startup.
Starting udev:  [  OK  ]
…
Starting system message bus: [  OK  ]
Starting HAL daemon: [  OK  ]
Starting guestgdm:  [  OK  ]


Xen 3.0 Live CD running CentOS release 4.1 (Final)
Kernel 2.6.12.6-xen on an i686


You can log in as ‘root’ using password ‘xensource’.


npt84 login:

As the instructions say, you can now log into the virtual machine as root using the password xensource. Poke around a bit and check out what IP address was assigned:

[root@npt84 ~]# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr AA:00:00:56:A4:45

inet addr:192.168.84.195  Bcast:192.168.84.255  Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:415 errors:0 dropped:0 overruns:0 frame:0
TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:27388 (26.7 KiB)  TX bytes:726 (726.0 b)


[root@npt84 ~]#

[root@npt84 ~]# ethtool -K eth0 tx off
[root@npt84 ~]#

If you now install a VNC client on your desktop machine and you can connect to the virtual machine over the network!

Using root/xensource works here as well, and results in a desktop session with the xfce environment:

If you’d like to have multiple guest Linux domains running, simply create multiple xen-centos directories, one for each guest. Make sure that each guest has its’ own copy of the rootfs.img filesystem (they can share a single kernel image) and that the .py configuration file for each guest points to the right filesystem.

If you have any questions or comments on this approach, perhaps send them to the xen-discuss list?