Re: [Xen-devel] qemu and xl semantics

[Christoph Egger <Christoph.Egger@xxxxxxx>]

On Wednesday 22 December 2010 18:10:55 Ian Campbell wrote:
> On Wed, 2010-12-22 at 16:47 +0000, Christoph Egger wrote:
> > On Wednesday 22 December 2010 17:08:38 Ian Campbell wrote:
> > > On Wed, 2010-12-22 at 15:42 +0000, Christoph Egger wrote:
> > > > On Monday 20 December 2010 11:23:59 Ian Campbell wrote:
> > > > > On Fri, 2010-12-17 at 17:21 +0000, Christoph Egger wrote:
> > > > > > On Friday 17 December 2010 11:32:41 Ian Campbell wrote:
> > > > > > > On Fri, 2010-12-17 at 09:49 +0000, Christoph Egger wrote:
> > > > > > > > On Friday 17 December 2010 10:15:14 Ian Campbell wrote:
> > > > > > > > > On Fri, 2010-12-17 at 09:00 +0000, Christoph Egger wrote:
> > > > >
> > > > > The only recent qemu change which I am aware of in this area is the
> > > > > backport of the blkback functionality from upstream Qemu. However
> > > > > this should only be enabled in cases where blkback or blktap are
> > > > > not available and furthermore is tied to libxl/xl so shouldn't have
> > > > > done anything on xend (although shouldn't != doesn't).
> > > >
> > > > Is this supposed to interact with the Dom0 PV disk backend driver ?
> > >
> > > The qemu disk backend is intended to be used when there is no dom0 PV
> > > disk backend driver in the kernel, if there is a driver in the kernel
> > > then it is intended that the kernel driver should be used.
> > >
> > > The background to this is that we have just gotten the basic dom0
> > > support into the upstream Linux kernel and are about to start on
> > > upstreaming the backend drivers. However we observed that there may not
> > > be any need to upstream a block backend since a userspace
> > > implementation may well suffice. It's not a done decision but the early
> > > signs are good, especially compared with blktap which hits userspace
> > > anyway.
> > >
> > > Even if we do eventually decide to upstream the Linux blkback using the
> > > disk backend in qemu tides us over for the time being.
> >
> > NetBSD has a kernel backend driver since Xen 1.x days ... upstream.
> > The last major change happened to it when it got updated to fit for Xen 2
> > and still fits well for Xen 3/4, too.
>
> That's cool, I didn't realise the block protocol had been so consistent
> over the releases.

Yes. The only one pitfall of the block protocol is the limitation to 32kb 
blocksize. NetBSD uses 64kb blocksize. The NetBSD domU has to split it into
two requests and the backend driver has to merge it again.


> > > > > > xbdback backend/vbd/1/832: can't VOP_OPEN device 0xe13: 16
> > > > > > 16 means EBUSY.
> > > > >
> > > > > But it works other than this message?
> > > >
> > > > Well, that means that there are now two processes which want to open
> > > > the vbd simultaneously. The first one wins. And on guest shutdown the
> > > > VOP_CLOSE fails.
> > >
> > > Right. This suggests that the qemu disk backend is being erroneously
> > > activated on NetBSD when what you actually want is to use the xbdback
> > > process. I think you likely need to patch libxl to do the right thing
> > > for NetBSD, currently the decision is based on the presence or absence
> > > of blktap, I guess it needs extending to detect NetBSD's backend too.
> >
> > xbdback is actually the kernel backend driver. The conflicting processes
> > are qemu-dm and the script that assigns the disk through the loopback
> > device.
> >
> > It seems to work in either way depending on which process wins...
>
> Hrm, you shouldn't be getting both in the first place...

Right.

>
> > > Alternatively perhaps NetBSD also wants to transition to using the qemu
> > > based block backend, I suppose there is benefit to be had from sharing
> > > this code between Linux and NetBSD dom0.
> >
> > Yes, I think so as long as it doesn't start requiring a kernel driver
> > other than xbdback.
>
> If you choose this route then there should be no kernel driver at all,
> that's the point.

Ok. Can the qemu based block backend handle 64kb blocksize?

> > > > I did some more debugging and figured out xenbackendd runs the
> > > > vif-bridge script so network with a PV driver works.
> > > >
> > > > The scripts not running is the one associated with 'vbd' and
> > > > qemu-ifup. AFAIK  qemu-dm always run qemu-ifup which is no longer the
> > > > case.
> > > >
> > > > qemu-ifup adds tap interface to the bridge.
> > >
> > > This changed (on Linux) quite a while ago so I didn't think of it.
> > > qemu-ifup is not used there any more and instead the vif-bridge script
> > > is called for both PV and TAP devices. See xen-unstable.hg
> > > 21944:0232bc7c9544, I guess either some equivalent change is needed to
> > > tools/hotplug/NetBSD or the libxl bit needs to be conditional on
> > > NetBSD.
> >
> > hmm... so when I start the guest with 'xm' then qemu runs qemu-ifup.
> > when I start the guest with 'xl' then qemu does not.
> >
> > So when I adjust the script then I fix one and break the other.
> > So to have it working on both the best way is to have the libxl bit
> > conditional on NetBSD.
>
> Or add the "script=no" stuff to the qemu command line in xend too so
> that it behaves like libxl. I think this would be preferred (assuming it
> works).

I will give that a try and submit a patch if that works.

Christoph


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel