[Xen-devel] HOWTO make PV drivers for 2.6.32.11 domU HVM

[Alex Bligh <alex@xxxxxxxxxxx>]

I have had some success compiling xen 4.0 PV drivers for 2.6.32.11, and
seeing as this appears to be non-trivial, I thought it worth documenting.
All the information is out there, but I thought I'd publish a HOWTO for
other people; perhaps I am very stupid, but it took me a while to work
out, partly as the unmodified_drivers documentation is misleading.
My target was Ubuntu, but as I start off with kernel.org sources, this
is largely irrelevant.

Question: this may be a very naive question, but would it not make
life simpler for people if those distributing the xenify parch set
(the ones I grabbed from google) also distributed an "unmodified_drivers"
directory with the path mangling done by mkbuildtree already in place.
Then it would be simply a matter of going into unmodified_drivers
and calling a script that sets XEN= and XL= and calls make. This would
avoid the need to bring down the entire xen source, and would mean
only combining two potions, rather than three. As far as I can tell,
it doesn't actually /need/ the xen source, though I may be wrong. And
pv drivers built this way work with older versions (I've tried 3.3)
as well as Xen 4.0.

HOWTO make PV drivers for 2.6.32.11 domU HVM

Step 1
======

Build and install an HVM kernel built from xenified kernel sources, but
*without* any of the Xen options turned on (this got me for a while). You
want a normal guest configuration which knows as little about Xen as
possible.

$ mkdir ~/test
$ cd ~/test
$ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.11.tar.bz2
$ tar xvjf linux-2.6.32.11.tar.bz2

Now xenify it (patch script courtesy of Boris Derzhavets)

$ cat >patcher
#!/bin/bash
for P in `ls ../xen-patches-2.6.32-1/6*.patch1 | sort `
do
patch -p1 -s -i $P
if [ $? = 0 ]; then
echo $P applied
else
echo "Error processing "$P
exit 1
fi
done
^D
$ chmod ogua+x patcher

Get the patches from Gentoo:
$ wget 
http://gentoo-xen-kernel.googlecode.com/files/xen-patches-2.6.32-1.tar.bz2
$ mkdir xen-patches-2.6.32-1
$ cd xen-patches-2.6.32-1
$ tar -xvjf ../xen-patches-2.6.32-1.tar.bz2
$ cd ..

Apply the patches
$ cd linux-2.6.32.11
$ ../patcher

Make an appropriate .config - NOTE: DO NOT TURN ON ANY XEN OPTIONS
USE A STANDARD x86_64 config.
$ cp /boot/config-`uname -r` .config
$ make menuconfig
[make any changes and exit]
To speed up build:
$ make localmodconfig

Build kernel, either traditionally (make), or via
$  fakeroot make-kpkg --initrd --append-to-version=-xen-hvm-domu \
   kernel-image kernel-headers


NB: for me this refused to build an initrd, which I had to do manually (i.e.
make modules ; make modules_install
mkinitramfs -o initrd.img-2.6.32.11-xen-hvm-domu 2.6.32.11-xen-hvm-domu
) - this step should be unnecessary.

Step 2
======

Make pv drivers

This assumes you are inside the linux directory (i.e. at the end of step
1), which has a built kernel in it (it will need a generated .config, plus
the kernel symbols, possibly plus other stuff).

First make a file with this patch in
$ cat > pvdrivers.patch
diff -r 88d9619e21c3 unmodified_drivers/linux-2.6/blkfront/Kbuild
--- a/unmodified_drivers/linux-2.6/blkfront/Kbuild	Wed Apr 21 08:36:58 2010 
+0100
+++ b/unmodified_drivers/linux-2.6/blkfront/Kbuild	Sat Apr 24 22:03:59 2010 
+0100
@@ -2,4 +2,4 @@

obj-m += xen-vbd.o

-xen-vbd-objs := blkfront.o vbd.o
+xen-vbd-objs := blkfront.o vbd.o vcd.o
^D

First get the xen source
$ mkdir pvdrivers
$ cd pvdrivers
$ hg clone http://xenbits.xen.org/xen-4.0-testing.hg
$ cd xen-4.0-testing.hg

Now fix a broken makefile (otherwise insmod on xen-vbd.ko will give you
unresolved symbols for vcd_register, vcd_unregister)
$ perl -pi -e 's/ vbd.o$/ vbd.o vcd.o/' 
unmodified_drivers/linux-2.6/blkfront/Kbuild
Now set paths up for build, and build it, totally ignoring the instructions
which I believe are wrong. Note that both XL and XEN point into kernel
source you have built, not into the xen source tree.

$ export XL=`cd ../..;pwd`
$ export XEN=$XL/include/xen
$ cd unmodified_drivers/linux-2.6/
$ ./mkbuildtree
$ make -C $XL modules M=$PWD
$ find . -name '*.ko'

That gives you the locations of your pvdrivers as kernel modules.

What I did was install the kernel (without the pvdrivers first)
and then insmod the pvdrivers in this order:
$ sudo insmod xen-platform-pci.ko
$ sudo insmod xen-balloon.ko
$ sudo insmod xen-vnif.ko
$ sudo insmod xen-vbd.ko
$ sudo insmod xen-scsi.ko


--
Alex Bligh

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel