[Xen-devel] Re: [PATCH 00/13] Nested Virtualization: Overview

[Christoph Egger <Christoph.Egger@xxxxxxx>]

On Wednesday 08 September 2010 10:58:52 Dong, Eddie wrote:
> >>> When the VCPU switches between host and guest mode then you need to
> >>> save the l1 guest state, restore the l2 guest state and vice versa.
> >>
> >> No. Switching from L1 to L2 guest, it is a simple restore of L2
> >> guest state (VMCLEAR/VMPTRLD). VMX doesn't save L1 state,
> >
> > So the function hooks prepare4vmentry is pretty cheap and the
> > 'hostsave' function hook is empty.
>
> I don;t know how you conclude here, extending SVM knowledge to VMX again?
> Prepare4vmentry has to load several VMCS fields, which is expansive in VMX.
>
> From SVM code, it loads 11 VMCB fields, which may be 11 VMCS field access
> in VMX. nsvm_vmcb_prepare4vmexit does reverse thing which touches at least
> 6 VMCS field again.

I better start with explaining the basic thing for software design: Interface
and Implementation.

Think on cars. All cars are different but they all have three pedals and a 
steering wheel. The three pedals and the steering wheel have the same
function in all cars. To drive a car you need to know what they do and how you
use them. But you do not have to know how they work.

Speaking in software design, this is called an interface.

How the pedals and the steering wheel work is up to the car manufacturers how
they make them work to function as intended. They build different motors
with different numbers of cylinders, one needs benzine an other needs diesel.
By concept the car manufacturer can even build in hamster wheels as a motor
(Getting the hamster running faster/slower is up to the car manufacturer
when using the pedals).

Speaking in software design, this is called an implementation.


Now translated to nested virtualization, the interface 'prepare4vmentry'
does prepare the l2 guest state ready to run. The way you think you have to
implement it is a possible way to make it work but the assumption you have
to follow the SVM implementation is wrong.

If I were you I would take the function 'virtual_vmentry' from
http://lists.xensource.com/archives/html/xen-devel/2010-04/msg01166.html
and implement 'prepare4vmrun' this way:

static void virtual_vmentry(struct vcpu *v, struct cpu_user_regs *regs)
{
    struct nestedhvm *hvm = &vcpu_nestedhvm(v);
    struct vmx_nest_struct *nest = hvm->nh_arch;
    struct vmcs_struct *svmcs = hvm->nh_vm;
    struct vmcs_struct *host_vmcs = v->arch.hvm_vmx.vmcs;

#ifdef __x86_64__
    unsigned long lm_l1, lm_l2;
#endif

    vmx_vmcs_switch_current(v, host_vmcs, svmcs);

    /* generic code switches vcpu into guest mode */

    nest->vmresume_pending = 0;

    /* generic code indicates that we are in vmentry emulation by
     * nh_hostflags.fields.vmentry flag.
     */

#ifdef __x86_64__
    /*
     * EFER handling:
     * hvm_set_efer won't work if CR0.PG = 1, so we change the value
     * directly to make hvm_long_mode_enabled(v) work in L2.
     * An additional update_paging_modes is also needed is
     * there is 32/64 switch. v->arch.hvm_vcpu.guest_efer doesn't
     * need to be saved, since its value on vmexit is determined by
     * L1 exit_controls
     */
    lm_l1 = !!hvm_long_mode_enabled(v);
    lm_l2 = !!(__get_vvmcs(nest->vvmcs, VM_ENTRY_CONTROLS) &
                           VM_ENTRY_IA32E_MODE);

    if ( lm_l2 )
        v->arch.hvm_vcpu.guest_efer |= EFER_LMA | EFER_LME;
    else
        v->arch.hvm_vcpu.guest_efer &= ~(EFER_LMA | EFER_LME);
#endif

    load_l2_control(nest);
    load_vvmcs_guest_state(nest);

#ifdef __x86_64__
    if ( lm_l1 != lm_l2 )
    {
        paging_update_paging_modes(v);
    }
#endif

    regs->rip = __get_vvmcs(nest->vvmcs, GUEST_RIP);
    regs->rsp = __get_vvmcs(nest->vvmcs, GUEST_RSP);
    regs->rflags = __get_vvmcs(nest->vvmcs, GUEST_RFLAGS);

    /* TODO: EPT_POINTER */
}


Analogous to this I would take 'virtual_vmexit' and implement it
for 'prepare4vmexit'.

Will this work for you ?
Do you still see utterly much VMCS field accesses when you go this way?


> >> Anyway, to me given that nested SVM & VMX is still on the very
> >> beginning of development, I can only say yes to those wrappers that
> >> have clear understanding to both side.
> >
> > Good to know. See my offer below.
>
> I am not sure if you want to efficiently start with lighweight wrapper
> first (those wrapper with consense and individual solution for those not),
> or you want to keep spinning on the heavy wrapper.  For me, if you go with
> lighweight wrapper like single layer VMX/SVM virtualization does, I can ack
> soon. But for the heavy weight wrapper, it is not necessary to me and will
> block future development of nested VMX, so I can't ack. Even current code
> is a lot of hard to understand given that I need to understand the
> semantics of those new (vendor-neutral) fields, however the semantics of
> VMX fields are clear to all VMX developers. Sorry for that, although I know
> Tim is more considerative to the effort you have called it out. I
> appreciate your effort as well, but that reason is not strong enough yet to
> impact nested VMX development.
>
> >> I would rather leave those uncertain wrappers to future, once the
> >> basic shape of nested virtualization is good and stable enough, i.e.
> >> lightweight wrapper.  We have plenty of performance work ahead such
> >> as virtual VTd support, enhanced PV driver for nested etc. Excessive
> >> wrapper is simple a burden to nested VMX developers for those future
> >> features.
> >>
> >> Qing will post his patch today or tomorrow for your reference if you
> >> want.
> >
> > Thanks. May I take code from there and add into my patch series ?
>
> Sure.

Thanks.

> But you won;t assume this is the logic we won't change. 

Of course not.

Christoph


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel