|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] qemu and xl semantics
On Monday 20 December 2010 11:23:59 Ian Campbell wrote:
> On Fri, 2010-12-17 at 17:21 +0000, Christoph Egger wrote:
> > On Friday 17 December 2010 11:32:41 Ian Campbell wrote:
> > > On Fri, 2010-12-17 at 09:49 +0000, Christoph Egger wrote:
> > > > On Friday 17 December 2010 10:15:14 Ian Campbell wrote:
> > > > > On Fri, 2010-12-17 at 09:00 +0000, Christoph Egger wrote:
> > > > > > Hi!
> > > > > >
> > > > > > When I start a guest with xm the disk startup script assigns a
> > > > > > loopback device for qemu to open it.
> > > > > >
> > > > > > Now it seems that qemu opens the disk image directly. Then when
> > > > > > the loopback device wants to open the disk image then that fails
> > > > > > with EBUSY.
> > > > >
> > > > > By "Now..." you mean "With xl..." ?
> > > >
> > > > no, I mean with xm.
> > >
> > > Hrm, I didn't think xm had changed in this area for ages, but I don't
> > > really keep an eye on xm/xend.
> >
> > It is not xm/xend that changed. It is qemu that changed and
> > the startup scripts need an adaption to the new behaviour.
> > Hence I am asking for the new semantics to know how to do that.
>
> Which qemu changeset? (you obviously had this information to hand, why
> not pro-actively help me to help you? Don't make me guess what you know
> or have discovered)
It is git head or git tip - whatever latest c/s is called.
>
> The only recent qemu change which I am aware of in this area is the
> backport of the blkback functionality from upstream Qemu. However this
> should only be enabled in cases where blkback or blktap are not
> available and furthermore is tied to libxl/xl so shouldn't have done
> anything on xend (although shouldn't != doesn't).
Is this supposed to interact with the Dom0 PV disk backend driver ?
> Perhaps the problem is that this stuff is erroneously getting activated
> on NetBSD?
>
> > > > > I think this is all very specific to the precise disk type you have
> > > > > in your config, i.e. tap: vs file: vs phy: etc. Which are you
> > > > > using?
> > > >
> > > > I use 'file'.
> > >
> > > If you are using xm and not xl this is not relevant but:
> > >
> > > AIUI libxl treats file: as tap:aio:, in other words it will fire up
> > > blktap to provide the device backend. xend probably used a loop device
> > > and treated it more like phy: i.e. invoked blkback.
> >
> > Yes, I noticed that and I wonder how the algorithm works behind that.
>
> See libxl_device.c:libxl__device_disk_backend_type_of_phystype and
> libxl.c:libxl_device_disk_add.
>
> A file: type device is really just a "degenerate" thing in the style of
> qcow or vhd so using tapdisk to export it makes some sort of sense and
> when it came to implementing libxl it was the simpler option compared
> with integrating with the loopback subsystem.
>
> > > Since NetBSD doesn't have blktap perhaps libxl needs to be updated to
> > > do the NetBSD equivalent? (I thought I'd seen a patch from you which
> > > did this?)
> >
> > The disk access works on NetBSD - don't ask me how - otherwise I wouldn't
> > be able to boot a guest at all. I have to run the network script manually
> > in the dom0 to have network access from and to the guest.
> >
> > > Recently libxl was also changed to use the qemu disk backend in cases
> > > where blktap is not available -- perhaps that had an undesired knockon
> > > effect on NetBSD?
> >
> > Yes. Recently this message appears from the NetBSD disk backend driver:
> >
> > xbdback backend/vbd/1/832: can't VOP_OPEN device 0xe13: 16
> > 16 means EBUSY.
>
> But it works other than this message?
Well, that means that there are now two processes which want to open the vbd
simultaneously. The first one wins. And on guest shutdown the VOP_CLOSE fails.
> I don't have much insight into how the NetBSD side of things works, I'm
> afraid I think you will need to dig in and debug it.
>
> > > Further to this there was a patch floating around (from Stefano) which
> > > ensured that a qemu was launched when it was necessary for PV guests to
> > > provide the disk backend. I'm not sure this got merged but ion the
> > > meantime the workaround is to add a vfb which also causes qemu to get
> > > launched for a PV guest.
> >
> > What is the longterm solution?
>
> The patch from Stefano.
>
> Even longer term we may improve the libxl interface to make it more
> explicit to the calling toolstack when it needs to start a qemu.
>
> > > > > > The network startup script adds the tap device to the bridge
> > > > > > or assigns an ip address.
> > > > > > With xl neither the disk nor the network script runs.
> > > > > > So when I start the guest with xl then I have
> > > > > > the tap device assigned to the guest but the
> > > > > > tap device is not configured in the dom0.
> > > > > >
> > > > > > How does the 'xl' way work in respect to the network script
> > > > > > used with 'xm' ?
> > > > >
> > > > > On Linux these are run from the hotplug event, via the udev rules.
> > > > > I presume you are talking about on NetBSD though?
> > > >
> > > > Yes.
> > > >
> > > > > Under Linux I think it was always the same under xm too although
> > > > > there have been some tweaks recently, e.g. the vif script is now
> > > > > always /etc/xen/scripts/vif-setup which handles the indirection to
> > > > > the script in the domain config or the default. Previously xend the
> > > > > hotplug rules called the configured script directly. This change
> > > > > was
> > > > > 21549:8bcaec29574e and was common to xm and xl I think.
> > > >
> > > > On NetBSD a xenbackendd starts along with xenstored.
> > > > xenbackendd watches on /local/domain/0/backend
> > > > and invokes the corresponding scripts when something changes
> > > > beneath that.
> > > >
> > > > 'xend' is doing that in DevController.py. Since 'xl' is not
> > > > interacting with 'xend' the scripts don't get launched at all.
> > >
> > > xl also creates stuff under /local/domain/0/backend (as it must) so why
> > > is xenbackendd not firing up and running the scripts?
> >
> > xenbackendd is firing up for things
> > like '/local/domain/0/backend/console/1/0/script'.
> >
> > xenbackendd listens for changes on vec[XS_WATCH_PATH]/state.
> > Then it reads vec[XS_WATCH_PATH]/hotplug-status.
> >
> > And then it looks at '/local/domain/0/backend/vbd'
> > and '/local/domain/0/backend/vif' and starts the corresponding
> > scripts.
I did some more debugging and figured out xenbackendd runs the vif-bridge
script so network with a PV driver works.
The scripts not running is the one associated with 'vbd' and qemu-ifup.
AFAIK qemu-dm always run qemu-ifup which is no longer the case.
qemu-ifup adds tap interface to the bridge.
> > > Perhaps xenbackendd is expecting some additional key which xend adds
> > > but libxl doesn't?
> >
> > Maybe. Is something missing from above?
>
> I've no idea -- what does xenbackendd expect? What do you get in
> xenstore? Howe do the two match up?
This is the output of 'xenstore-ls':
tool = ""
xenstored = ""
local = ""
domain = ""
0 = ""
backend = ""
qdisk = ""
2 = ""
768 = ""
frontend = "/local/domain/2/device/vbd/768"
params = "aio:/hvm-guest/netbsd_64b.img"
frontend-id = "2"
online = "1"
removable = "1"
bootable = "1"
state = "1"
dev = "hda"
type = "tap"
mode = "w"
832 = ""
frontend = "/local/domain/2/device/vbd/832"
params = "aio:/hvm-guest/netbsd_disk2.img"
frontend-id = "2"
online = "1"
removable = "1"
bootable = "1"
state = "1"
dev = "hdb"
type = "tap"
mode = "w"
vif = ""
2 = ""
0 = ""
frontend = "/local/domain/2/device/vif/0"
frontend-id = "2"
online = "1"
state = "2"
script = "/usr/local.22573.netbsd/etc/xen/scripts/vif-bridge"
mac = "00:16:3e:4d:e0:51"
bridge = "bridge0"
handle = "0"
feature-rx-copy = "1"
feature-rx-flip = "1"
hotplug-status = "connected"
console = ""
2 = ""
0 = ""
frontend = "/local/domain/2/console"
frontend-id = "2"
online = "1"
state = "1"
domain = "HVM64-NetBSD"
protocol = "vt100"
device-model = ""
2 = ""
disable_pf = "0"
state = "running"
2 = ""
vm = "/vm/50df51d0-d90d-e011-b0d4-00e081806fbe"
name = "HVM64-NetBSD"
device = ""
suspend = ""
event-channel = ""
vbd = ""
768 = ""
backend = "/local/domain/0/backend/qdisk/2/768"
backend-id = "0"
state = "1"
virtual-device = "768"
device-type = "disk"
832 = ""
backend = "/local/domain/0/backend/qdisk/2/832"
backend-id = "0"
state = "1"
virtual-device = "832"
device-type = "disk"
vif = ""
0 = ""
backend = "/local/domain/0/backend/vif/2/0"
backend-id = "0"
state = "1"
handle = "0"
mac = "00:16:3e:4d:e0:51"
data = ""
cpu = ""
0 = ""
availability = "online"
1 = ""
availability = "online"
2 = ""
availability = "online"
3 = ""
availability = "online"
memory = ""
static-max = "3145728"
target = "3137536"
videoram = "8192"
error = ""
drivers = ""
control = ""
platform-feature-multiprocessor-suspend = "1"
attr = ""
messages = ""
domid = "2"
store = ""
port = "5"
ring-ref = "1044476"
console = ""
backend = "/local/domain/0/backend/console/2/0"
backend-id = "0"
limit = "1048576"
type = "xenconsoled"
output = "pty"
port = "6"
ring-ref = "1044479"
tty = "/dev/pts/1"
vnc-port = "5900"
image = ""
device-model-pid = "558"
serial = ""
0 = ""
tty = "/dev/pts/2"
vm = ""
50df51d0-d90d-e011-b0d4-00e081806fbe = ""
uuid = "50df51d0-d90d-e011-b0d4-00e081806fbe"
name = "HVM64-NetBSD"
pool_name = "Pool-0"
rtc = ""
timeoffset = ""
image = ""
ostype = "hvm"
start_time = "1293028997.78"
vncpasswd = "\000"
Christoph
--
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |