Re: [Xen-devel] libxl: problem with devices in PV

[Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx>]

On Wed, 2011-07-20 at 14:10 +0100, Roger Pau Monnà wrote:
> 2011/7/20 Ian Campbell <Ian.Campbell@xxxxxxxxxx>:
> > On Wed, 2011-07-20 at 12:37 +0100, Stefano Stabellini wrote:
> >> On Wed, 20 Jul 2011, Roger Pau Monnà wrote:
> >> > Hello,
> >> >
> >> > I'm trying to run PV machines using the new libxenlight toolstack on
> >> > NetBSD. So far, I've been able to connect to the domu using the
> >> > console (I was unable to do so before). I'm attaching a little patch
> >> > that removes setting the consback to IOEMU when detecting a qdisk
> >> > (that was preventing the domu from even booting). With the
> >> > introduction of libxenlight, Xen doesn't use vbd for disk backend
> >> > anymore, it uses qdisk, which I assume Qemu automatically attaches to
> >> > running guests. It works fine with HVM guests, but it seems to fail
> >> > with PV guests. The config file I'm using is:
> >> >
> >>
> >> Xen uses qdisk only when blktap is not available.
> >> I suggest you install blktap if you can because it is significantly
> >> faster than qdisk at the moment.
> >
> > This is a NetBSD host so I don't think blktap is an option.
> 
> NetBSD has the vnd driver, which provides a disk interface to a file
> (it's basically a loop device):
> 
> http://netbsd.gw.com/cgi-bin/man-cgi?vnd+4+NetBSD-current
> 
> It is used with xend and xenbackendd to mount raw disks.

If I understand correctly (which is remotish possibility):

xenbackendd watches the backend/vbd directory and creates disks based on
it. A "vbd" directory roughly corresponds to a "phy:" type device
configuration in libxl.

xend, at least on Linux, internally handles "file:" by setting up a loop
device and translating the config into a "phy:"==vbd refering to
the /dev/loopN device. I suspect that on NetBSD xend used to just create
the vbd referring to the file directly and let xenbackendd take care of
any necessary loop back stuff.

libxl on the other hand does not do this loop device magic but instead
palms "file:" off onto either qdisk or blktap and furthermore libxl
requires that a "phy:" disk configuration refers to a block device and
not a file.

The upshot is that the backend selection logic in libxl is apparently
not correct for NetBSD which _can_ handle files passed as "phy:" due to
xenbackendd doing the right thing via the vnd driver.

Fortunately Ian Jackson recently cleaned up the backend selection logic
in libxl and it should be much cleaner and easier to express these sorts
of system specific things, presumably by patching disk_try_backend to
allow files as well as block devices for LIBXL_DISK_BACKEND_PHY on
NetBSD.

> >> Otherwise if you use a physical partition and specify phy: in the config
> >> file you should get the kernel based blkback that is the fastest option
> >> available.
> >
> > phy: would be worth a go since it should tie into NetBSD's equivalent of
> > blkback, whereas file: presumably goes to qdisk in the absence of
> > blktap.
> 
> Haven't tried phy:/ with unstable, but I supose it works, since it
> worked in previous versions.

Since you are passing a file name a block device it is possible that
libxl today might reject it when used with phy:/...

[...]
> What I don't get is why qdisk work with HVM machines but not with PV
> machines, shouldn't it be the same?

I suspect that in the HVM case you aren't really getting a qdisk (a PV
backend) but actually an emulated disk device. Or rather you are
probably getting both but the guest only tries to use the emulated one
and hence the PV backend never tries to initialise and so never falls
over due to lack of gntdev.

Can you confirm if your HVM guest uses emulated or PV frontends?

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel