[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] Re: [RFC, PATCH 3/24] i386 Vmi interface definition

Chris Wright wrote:
* Zachary Amsden (zach@xxxxxxxxxx) wrote:
Master definition of VMI interface, including calls, constants, and
interface version.

+/* VROM call table definitions */
+#define VROM_CALL_LEN             32
+typedef struct VROMCallEntry {
+   char f[VROM_CALL_LEN];
+} VROMCallEntry;

And the call entry is meant to be handled in whatever mechanism hypervisor
prefers for its entry points (ABI constraints notwithstanding) as in,
arbitrary software interrupt, or call gate, etc?  I guess for transparent
it has to, since those would be local calls.   Quite similar to the
hypercall entry point that Xen places on the hypercall_page, so easily

See below.

The document is slightly more descriptive.  The above reserved slots
are shown as:

        char            reserved[32];
        char            elfHeader[64];

But that's only 3 (0-2).  I think I'm missing some small bit of magic.

+typedef struct VROMCallTable {
+   VROMCallEntry    vromCall[128];           // @ 0x80: ROM calls 4-127
+} VROMCallTable;

That comment eludes me.  Are 0-3 special somehow (IOW, I thought it was
just 0-2 as per above), and is it suggesting int 0x80?

Yeah, most of this is rather crufty - it is in transition. We had a full blown ROM image with 32-byte aligned stub points at one point for all of the VMI calls. In fact, it still is in that form, and it was required to overlap exactly with a native ROM that was built into Linux. See patch 6, VMI magic fixes for the details.

The machinery is already in place to do this, and it is a very nice thing that Xen has decided to adopt a similar approach (to the ROM) of publishing hypervisor code. I think they even use 32-byte alignment as well. The power of an indirection at this layer is just too attractive, and once you decide on a single binary image, I think it is inevitable that everyone will converge on the same idea. The same concept surfaces over and over - vsyscall being another example. You're basically dynamically linking in code at runtime, which is a pretty common thing to do, and gives you a very powerful redirection interface.

But we discovered that we couldn't achieve native performance, even using directly linked calls into the ROM. Calling out for sti / cli / popf / pushf on fast paths is just too expensive. We had to inline the native code to match native performance. And this leaves an opportunity for flexibility and performance in the hypervisor implementation.

You don't want direct calls to 32-byte stubs; in fact, as Joshua found, you can't get optimal performance in a hypervisor that way. What you need is the ability to preferentially inline certain hot calls into the VMI layer by using NOP padding. The non-hot calls can call out to the hypercall page.

Publishing code is the first step - inlining is the second, and it gets you back the hit you took when indirecting your fast paths.


Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.