Re: [Xen-devel] Upstream QEMU based stubdom and rump kernel

[Martin Lucina <martin@xxxxxxxxxx>]

(Adding some of the Mirage folks to Cc:)

wei.liu2@xxxxxxxxxx said:
> Hi all
> 
> I'm now working on upstream QEMU stubdom, and rump kernel seems to be a
> good fit for this purpose.
> 
> A bit background information. A stubdom is a service domain.  With QEMU
> stubdom we are able to run QEMU device emulation code in a separate
> domain so that bugs in QEMU don't affect Dom0 (the controlling domain).
> Xen currently has a QEMU stubdom, but it's based on our fork of ancient
> QEMU (plus some other libraries and mini-os). Eventually we would like
> to use upstream QEMU in stubdom.
> 
> I've now successfully built QEMU upstream with rump kernel. However to
> make it fully functional as a stubdom, there are some missing pieces to
> be added in.
> 
> 1. The ability to access QMP socket (a unix socket) from Dom0. That
>    will be used to issue command to QEMU.
> 2. The ability to access files in Dom0. That will be used to write to /
>    read from QEMU state file.

As I understand from Stefano's and Anthony's replies in this thread, both
of the above can be implemented using an AF_UNIX or AF_INET socket on the
QEMU end. Such an implementation would not require anything special done in
QEMU, just telling it which socket to use using existing mechanisms.

So, let's step back a bit: What we need is a trusted communication channel
from a Rump Kernel domU to dom0, using existing socket or socket-like[*]
APIs at both the domU and dom0 ends.

This fits in with a couple of things I hope to make time to work on in the
next couple of months:

 1. Introspection of Rump Kernel domUs for ops purposes, i.e. get some
    basic "ps", "top", "vmstat"-like information about what the domU is
    doing from the dom0.

 2. Connecting up multiple Rump Kernel domUs and/or Mirage domUs. The
    general idea here is that you can have e.g. a Mirage domU running a
    HTTP+TLS frontend, communicating with a Rump Kernel domU running PHP +
    FastCGI.

    The Mirage folks are already doing something similar in their 
    Jitsu work, using a protocol called Conduit which runs over vchan.

Now, both of the above require exactly the same underlying mechanism.

Point 2. will further require implementing support in the Rump Kernel,
either for a shim which would proxy AF_UNIX / AF_INET transparently using
vchan, or possibly later implementing a separate socket family (AF_VCHAN /
AF_HYPER?). Once that is done you should be able to just drop it in to
QEMU on Rump.

[*] Aside: What I mean by socket-like is that the implementation does not
need to be in the dom0 kernel, it can just be a user-space library. For
example, see the nanomsg or ZeroMQ APIs, which I have worked on extensively
in the past.

> 3. The building process requires mini-os headers. That will be used
>    to build libxc (the controlling library).

As Antti already suggested, if you can use POSIX interfaces rather than
mini-os ones in QEMU, then that would be a better approach.

> One of my lessons learned from the existing stubdom stuffs is that I
> should work with upstream and produce maintainable code. So before I do
> anything for real I'd better consult the community. My gut feeling is
> that the first two requirements are not really Xen specific. Let me know
> what you guys plan and think.

Thanks for getting in touch. I think this is an important discussion!

Martin

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel