Re: [Xen-devel] [PATCH 0/8] Introduce AMD SVM AVIC

[Brian Woods <brian.woods@xxxxxxx>]

On Tue, Apr 03, 2018 at 06:01:16PM -0500, Janakarajan Natarajan wrote:
> OVERVIEW
> ========
> This patchset is the first of a two-part patch series to introduce
> the AMD Advanced Virtual Interrupt Controller (AVIC) support.
> 
> The AVIC hardware virtualizes local APIC registers of each vCPU via
> the virtual APIC (vAPIC) backing page. This allows the guest to access
> certain APIC registers without the need for emulation of hardware
> behaviour in the hypervisor. More information about AVIC can be found in
> 
> * AMD64 Architecture Programmers Manual Volume 2 - System Programming
>   https://support.amd.com/TechDocs/24593.pdf
> 
> For SVM AVIC, we extend the existing SVM driver to:
> * Check CPUID to detect AVIC support in the processor.
> * Program new fields in VMCB to enable AVIC.
> * Introduce new AVIC data structures and add code to manage them.
> * Handle two new AVIC #VMEXITs.
> * Add new interrupt injection code using vAPIC backing page
>   instead of the existing V_IRQ, V_INTR_PRIO, V_INTR_VECTOR and
>   V_IGN_TPR fields.
> 
> This patchset does not enable AVIC by default since it does not
> yet support nested VMs. Until then, users can enable SVM AVIC by
> specifying Xen parameter svm=avic.
> 
> Later, in Part 2, we will introduce the IOMMU AVIC support, which
> provides speed-up for the PCI device passthrough use case by allowing
> the IOMMU hardware to inject interrupts directly into the guest via
> the vAPIC backing page.
> 
> OVERALL PERFORMANCE
> ===================
> AVIC is available on AMD Family 15h models 6Xh (Carrizo) processors
> and newer. An AMD Family 17h Epyc processor is used to collect the
> performance data shown below.
> 
> Generally, SVM AVIC alone (w/o IOMMU AVIC) should provide overall
> speed-up for HVM guest since it does not require #VMEXIT into the
> hypervisor to emulate certain guest accesses to local APIC registers.
> 
> It should also improve performance when the hypervisor wants to
> inject interrupts into a running vcpu. It can do this by setting the 
> corresponding IRR bit in the vAPIC backing page and triggering the
> AVIC_DOORBELL.
> 
> For sending IPI interrupts between running vcpus in a Linux guest,
> Xen defaults to using event channels. However, in case of
> non-paravirtualized guests, AVIC can also provide performance
> improvements for sending IPIs.
> 
> BENCHMARK: HACKBENCH
> ====================
> For measuring IPI performance used for scheduling workload, some
> performance numbers are collected using hackbench.
> 
>     # hackbench -p -l 100000
>     Running in process mode with 10 groups using 40 file descriptors
>     each (== 400 tasks)
>     Each sender will pass 100000 messages of 100 bytes
>                         |      3 vcpus (sec)        |
>     --------------------------------------------------
>     No AVIC w/  xen_nopv  |           517             |
>     AVIC w/ xen_nopv      |           173             |
>     No AVIC w/o xen_nopv  |           141             |
>     AVIC w/o xen_nopv     |           135             |
> 
> Each benchmark test was averaged over 10 runs.
> 
> CURRENT UNTESTED USE_CASES
> ==========================
> * Nested VM
> 
> Any feedback and comments are very much appreciated.
> 
> Suravee Suthikulpanit (8):
>   x86/SVM: Modify VMCB fields to add AVIC support
>   x86/HVM/SVM: Add AVIC initialization code
>   x86/SVM: Add AVIC vmexit handlers
>   x86/SVM: Add vcpu scheduling support for AVIC
>   x86/SVM: Add interrupt management code via AVIC
>   x86/HVM: Hook up miscellaneous AVIC functions
>   x86/SVM: Introduce svm command line option
>   x86/SVM: Add AMD AVIC key handler
> 
>  docs/misc/xen-command-line.markdown |  16 +
>  xen/arch/x86/hvm/svm/Makefile       |   1 +
>  xen/arch/x86/hvm/svm/avic.c         | 626 
> ++++++++++++++++++++++++++++++++++++
>  xen/arch/x86/hvm/svm/intr.c         |   4 +
>  xen/arch/x86/hvm/svm/svm.c          |  77 ++++-
>  xen/arch/x86/hvm/svm/vmcb.c         |   3 +
>  xen/arch/x86/hvm/vlapic.c           |  20 +-
>  xen/arch/x86/hvm/vmx/vmx.c          |   8 +-
>  xen/include/asm-x86/hvm/hvm.h       |   4 +-
>  xen/include/asm-x86/hvm/svm/avic.h  |  43 +++
>  xen/include/asm-x86/hvm/svm/svm.h   |   2 +
>  xen/include/asm-x86/hvm/svm/vmcb.h  |  52 ++-
>  xen/include/asm-x86/hvm/vlapic.h    |   4 +
>  xen/include/asm-x86/msr-index.h     |   1 +
>  14 files changed, 831 insertions(+), 30 deletions(-)
>  create mode 100644 xen/arch/x86/hvm/svm/avic.c
>  create mode 100644 xen/include/asm-x86/hvm/svm/avic.h
> 
> -- 
> 2.11.0
> 

Not a Xen expert by any means but I've looked over the patch set and
nothing jumps out as wrong.

Reviewed-by: Brian Woods <brian.woods@xxxxxxx>

-- 
Brian Woods

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel