VMCS perf, Was: [PATCH v2] x86/mem_sharing: support forks with active vPMU state

[Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>]

On 21/07/2022 13:35, Tamas K Lengyel wrote:
> On Thu, Jul 21, 2022 at 8:03 AM Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx> 
> wrote:
>> On 20/07/2022 19:47, Tamas K Lengyel wrote:
>>> +#endif
>>>  };
>>>
>>>  static const struct arch_vpmu_ops *__init common_init(void)
>>> diff --git a/xen/arch/x86/cpu/vpmu_intel.c b/xen/arch/x86/cpu/vpmu_intel.c
>>> index 8612f46973..01d4296485 100644
>>> --- a/xen/arch/x86/cpu/vpmu_intel.c
>>> +++ b/xen/arch/x86/cpu/vpmu_intel.c
>>> @@ -282,10 +282,17 @@ static inline void __core2_vpmu_save(struct vcpu *v)
>>>      for ( i = 0; i < fixed_pmc_cnt; i++ )
>>>          rdmsrl(MSR_CORE_PERF_FIXED_CTR0 + i, fixed_counters[i]);
>>>      for ( i = 0; i < arch_pmc_cnt; i++ )
>>> +    {
>>>          rdmsrl(MSR_IA32_PERFCTR0 + i, xen_pmu_cntr_pair[i].counter);
>>> +        rdmsrl(MSR_P6_EVNTSEL(i), xen_pmu_cntr_pair[i].control);
>>> +    }
>>>
>>>      if ( !is_hvm_vcpu(v) )
>>>          rdmsrl(MSR_CORE_PERF_GLOBAL_STATUS, core2_vpmu_cxt->global_status);
>>> +    /* Save MSR to private context to make it fork-friendly */
>>> +    else if ( mem_sharing_enabled(v->domain) )
>>> +        vmx_read_guest_msr(v, MSR_CORE_PERF_GLOBAL_CTRL,
>>> +                           &core2_vpmu_cxt->global_ctrl);
>> /sigh.  So we're also not using the VMCS perf controls either.
>>
>> That wants fixing too, but isn't a task for now.
> It does get saved and swapped on vmexit but we don't want to do this
> vmx_read/vmx_write in the mem_sharing codebase. It's much cleaner if
> this is saved into the vpmu context structure and reloaded from there,
> so we can just do a memcpy in mem_sharing without having to know the
> details.

This is specifically why I introduced the {pv,hvm}_{get,set}_reg()
interfaces.

Lots of callers want to operate on a specific register, without wanting
to know if it's live in an MSR, or in the VMCB, VMCS, MSR load/save
list, or in a random structure in memory.

This is a perfect example that wants converting.  One patch to move
MSR_CORE_PERF_GLOBAL_CTRL into the get/set reg infrastructure (no
practical change), and then a second patch to make the VT-x
implementation conditional between the MSR load/save lists and the VMCS
host/guest controls depending on hardware support.

~Andrew