[Xen-devel] Re: [PATCH 00/13] Nested Virtualization: Overview

[Christoph Egger <Christoph.Egger@xxxxxxx>]

On Thursday 09 September 2010 03:45:55 Dong, Eddie wrote:
> Well, so you decide to cycle here, though it obviously not only hurt Intel
> but also AMD's progress.
>
> Let me ask several questions first:
> 1: Does this heavy weight wrapper a must for nested virtualization?

It's not a question of a must. It's a question of design.

> 2: What is the gain and lose?

in what respect ?

> 3: How single layer virtualization does? Are they abstracting with heavy
> weight wrapper or light weight and why?

Maybe you should explain what you mean by 'heavy weight' and 'light weight'
wrapper.

By software concept, a wrapper is a function that implements one interface by
calling an other interface. The purpose is to provide/emulate yet another API.
Example: In NetBSD, the Linux syscalls are implemented by calling the
native sycalls. This allows to run Linux binaries on NetBSD.

In that sense, the answer to your question is "Neither nor".
It just puts common logic into one place that has been found over time.

> Particularly do they abstract the 1st level VM exit/entry event or do they
> invent 3rd name space for VMCS/VMCB field?

Yes, there is the guest_cr array for use in generic code, for example.


> The only reason I can see so far is that you called out the "abstract" in
> nested virtualization side, that is for some reason we have to appreciate
> but not enough for us to swallow the side effect of its impact to future
> nested VMX development.

You are worried about that the interface is fixed, right?

Note that this interface is an internal one. In opposition to an external
interface (xen/include/public/*) you do not need to care about backward
compatibility.

Common to an internal and an external interface are:
- care about not breaking what has worked before
- think first before doing a change what for, if it is really needed
   and that it makes really sense to avoid make it go wild

One example for a wild gone interface are the hypercalls
to get/set C- and P-States in Xen. When I want to start implementing
the feature in a Dom0 from scratch, I do not know which ones I should use.

Tim said in
http://lists.xensource.com/archives/html/xen-devel/2010-08/msg01044.html
"I've no strong feelings about the details of the interface between the
common and the arch-specific code, but it seems like with a bit of
flexibility on both sides it could be made suitable for everyone."

Your point about standardization from below comes in here, too.


> Taking your "car" abstract as example,
>
> > All cars are different but they all have three pedals and a steering
> > wheel.
>
> Well, automatic cars have 2 pedals only (but I know most German cars are
> mechanical car for green fuel reason which does have 3 pedals.
> Steering wheel is current implementation, but there are researches inventing
> automatic controlled cars that is fully controlled by computer, and I am
> not sure if trolley car is a car? If it is, it can work without steering
> wheel though it may have today.

Having 2 or 3 pedals or a steering wheel or not, here by concept
the question is who *uses* the interface.

Switching the gears and the capability to turn left/right is still there, 
because the implementation of the same interface still exists.

The difference is that who can see and use which interface.
Who switches the gears and who turns left/right
- is it the human or a computer ?


You also mentioned research: You know that doing research means to develop
prototypes ?  Prototypes may be but must not be production stable.
The purpose for writing prototypes is figure out how things work and
what problems may come up.

In software engineering, prototypes are something that get thrown away to
implement the production version from scratch where you know the
problems ahead.

Below you say, nested virtualization is at the very beginning. That's right
for the code, the idea is old (Goldberg's paper).

If you want to call the patch series a prototype then that means that
all of our code needs to be thrown away and we both must start from
scratch. Do you really want that?

>
> Above comments doesn;t mean I against your abstract of "car with 3 pedals
> and a steering wheeling" even it may be not accurate enough. But it does
> reflect the trade off between innovation and industry standardization. W/o
> standaradization, there are many variants coming in to compete in different
> area, but once a standardization comes, the innovation is deprioritized,
> rather other competition comes such as price. Hardly we can see if we need
> innovation more (no standardization) or cost saving more. But at very
> beginning of a new product/feature, I think innovation is more important to
> allow different variants to compete.

I disagree in that standardization and innovation are exclusive on each other.
A community consisting full of software engineers show again and again
that there are synergy effects:

Example: In NetBSD, the access to  a bus are standardized through an interface 
(http://netbsd.gw.com/cgi-bin/man-cgi?bus_space++NetBSD-current).
The drivers have a frontend and a backend. The frontend attaches
the driver onto the pci, isa, whatever bus. The backend contains all the
code using the interface, there's no code specific to pci, isa, etc.

Now when you come up with a new and innovative bus for the PC,
then you can add support for it by implementing the interface.
Instead of writing/porting all drivers to the new bus you go the other
way: You use the existing drivers to test if your bus implementation
works. And if you find a bug in the driver then the fix is not only good
for the new innovative bus.

> Taking software or nested virtualization as example, defining those
> wrappers or abstract to force each vendor to follow does deprioritize
> innovation, but they may be easier for followers to understand easily if
> they doesn't understand SVM/VMX itself very well.

I do not say you are wrong here. The sad thing is that it doesn't
work in reality.

You said in
http://lists.xensource.com/archives/html/xen-devel/2010-08/msg01007.html

"Drawing a picture which doesn't have a real usage with massive common code 
change is a kind of too much load for us now. Xen hvm_function table is a good 
example. Intel enabled VMX at very beginning of Xen HVM support, and SVM comes 
later on with a lot of code reuse from VMX side. Then the community and both 
side work together to make an API wrapper to reuse common code better and 
accomodate both architecture. I think we have to go similar path to make it 
work first."

I looked through the changesets where xen/arch/x86/hvm/hvm.c
was involved (hg log -v xen/arch/x86/hvm/hvm.c). I did not find a
changeset from Intel where code from SVM and VMX has moved into
hvm.c. So "both side work together" does not apply here.
Factoring code out from VMX was a very hard time for AMD developers
and that time is still not over.

Tim said in
http://lists.xensource.com/archives/html/xen-devel/2010-04/msg00812.html

"I would like to see some reasonable discussion of exactly how much of
the nested virtualization logic can be shared.  With HVM it has turned
out to be quite a lot, but it's taken years of reshuffling to pull code
out into common HVM routines (and we're not there yet)."

I do not want to repeat that again with nested virtualization.

Later when the nested virtualization becomes major where
the 'standardization' matters according to you, I doubt that
Intel developers will take account for it for two reasons:

a) Almost every patch from Intel that touches a file under
xen/include/public/* breaks backward-compatibility.
b) The fact of rare if not missing Intel patches where code from SVM and VMX
moves into hvm.c w/o breaking anything.

That all speaks a lot for pulling as much logic as possible
into common code.

> However, as I said, nested virtualization is still at its very beginning and
> we need innovation more to find what is the best solution especially from
> performance point of view.

Well, that is the research aspect with prototypes I mentioned above.

> Otherwise re-abstract does require double effort for VMX developers 
> to consider SVM support as well

At least in thinking, but not necessarily in coding. The result is higher code
quality that makes this effort worth. Keir will more often apply patches
w/o asking you to fix/re-work things first.

> (and even more than double effort because normally Intel VMX developers
> doesn't know SVM architecture very well

Not knowing something does not mean being stupid. There are manuals
to learn from.

> and we don't have test machine for SVM). 

You surely can afford some AMD machines given the much higher revenue
Intel has over AMD.

Christoph


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel