Re: [Xen-devel] qemu-upstream stubdom - status?

[George Dunlap <George.Dunlap@xxxxxxxxxxxxx>]

On Mon, Apr 7, 2014 at 9:50 PM, Eric Shelton <eshelton@xxxxxxxxx> wrote:
> Where does this feature stand currently?  It pops up under a couple of names
> and proposed implementations (e.g., "qemu-xen stubdomains", "qemu-upstream
> stubdom, Linux", and "qemu-upstream stubdom, BSD libc"), so it seems likely
> I missed something.
>
> If I understand this email from Sept 2013
> (http://lists.xen.org/archives/html/xen-devel/2013-09/msg02881.html),
> qemu-xen stubdomains might have made it into 4.4 if the schedule had been
> slipped a little, suggesting some active effort.  On the other hand, the
> same email designated the efforts attached to Anthony and Ian as "prognosis:
> ?".  Anthony released a patchset/branch about a year ago, but that got
> sidelined in favor of getting Xen on ARM.  However, I did not identify any
> more recent patches.
>
> It remains appealing to make use of the more recent developments in
> qemu-upstream, but with the isolation provided by a stubdomain.  The many
> improvements made in teasing driver domains out of dom0 strengthen this
> appeal (for example, using OpenSolaris as a ZFS-based storage domain).
>
> What are the most recent patches or repository for this feature?

I think stubdomains for qemu-xen should be a blocker for 4.5.

The only thing that's missing for stubdoms, AFAIK, is an integrated
way to deploy the stubdom VM in which qemu-xen can run.
qemu-traditional uses minios with a version of newlib (IIRC) ported to
it.  The problem at the moment is that qemu-xen requires much more
from its libc than newlib provides.  There are several potential
solutions that have been investigated:

1. Use Linux with glibc.
2. Port a more fully-functional libc (e.g., a BSD libc) to minios
3. Use the "rump kernel" functionality
4. Use OSv.

#1 involves mainly trying to cut down the Linux kernel and the guest
FS as small as possible.  Anthony spent some time on #1 in the 4.3 and
4.4 development cycles, and I think got the image down to 32MB.
That's still fairly large, however; and I think he got stuck trying to
integrate an image builder into the Xen build system.

#2 involves dealing with the mismatch between the OS functionality
provided by minios and the functionality required by the libc:
threads, async IO, &c.  This entails either disabling / working around
the lack of functionality, or implementing the requisite
functionality.  Ian Jackson spent some time in the 4.3 development
cycle working on this, but ended up having to switch to something
else.

#3 on the surface looks promising: the "rump kernels" approach allows
you to lift bits of the BSD kernel into another context, like
user-space; or in our case, a PV "Cloud OS".  The advantage of this is
that it's using all the existing tested code that works together, but
just replacing certain bits of it: e.g., the lowest layers with PV,
and the systemcall boundary with a function call.  This has already
been used to make single-address-space PV BSD guests boot, IIRC; it
would just take someone familiar with both to sit down integrate it.
At the moment I don't think anyone is actively looking at this.

#4 would involve taking the existing OSv system and adding missing
functionality, if any.  I don't think anyone is looking at this
either.

If you're looking for a solution to use right now, I'm sure Anthony
can point you to wherever he left off.  If you're looking to maybe get
involved and contribute, I think the consensus was that #3 looked like
the most promising option -- I'm sure IanJ can give you some pointers
of where to look.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel