[Xen-devel] PCI passthrough performance loss with Skylake-SP

["Andreas Kinzler" <hfp@xxxxxxxxx>]

I am currently testing PCI passthrough on the Skylake-SP platform using a  
Supermicro X11SPi-TF mainboard. Using PCI passthrough (an LSI SAS HBA)  
causes severe performance loss on the Skylake-SP platform while Xeon E3 v5  
is not affected at all. The test uses standard iperf3 as a client - the  
passed PCI device is not used in the test - so that just the presence of  
the passed device will cause the iperf3 performance to drop from 6.5  
gbit/sec (no passthrough) to 4.5 gbit/sec.
Any explanation/fixes for that?

Below the first part of xl dmesg for both systems.

Regards Andreas

Xeon E3-1240v5:
        (XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
        (XEN) Intel VT-d Snoop Control enabled.
        (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
        (XEN) Intel VT-d Queued Invalidation enabled.
        (XEN) Intel VT-d Interrupt Remapping enabled.
        (XEN) Intel VT-d Posted Interrupt not enabled.
        (XEN) Intel VT-d Shared EPT tables enabled.
        (XEN) 0000:00:13.0: unknown type 0
        (XEN) I/O virtualisation enabled
        (XEN)  - Dom0 mode: Relaxed
        (XEN) Interrupt remapping enabled
        (XEN) Enabled directed EOI with ioapic_ack_old on!
        (XEN) ENABLING IO-APIC IRQs
        (XEN)  -> Using old ACK method
        (XEN) Allocated console ring of 16 KiB.
        (XEN) VMX: Supported advanced features:
        (XEN)  - APIC MMIO access virtualisation
        (XEN)  - APIC TPR shadow
        (XEN)  - Extended Page Tables (EPT)
        (XEN)  - Virtual-Processor Identifiers (VPID)
        (XEN)  - Virtual NMI
        (XEN)  - MSR direct-access bitmap
        (XEN)  - Unrestricted Guest
        (XEN)  - VMCS shadowing
        (XEN)  - VM Functions
        (XEN)  - Virtualisation Exceptions
        (XEN)  - Page Modification Logging
        (XEN) HVM: ASIDs enabled.
        (XEN) HVM: VMX enabled
        (XEN) HVM: Hardware Assisted Paging (HAP) detected
        (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB

Skylake-SP:
        (XEN) Intel VT-d iommu 2 supported page sizes: 4kB, 2MB, 1GB.
        (XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
        (XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
        (XEN) Intel VT-d iommu 3 supported page sizes: 4kB, 2MB, 1GB.
        (XEN) Intel VT-d Snoop Control enabled.
        (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
        (XEN) Intel VT-d Queued Invalidation enabled.
        (XEN) Intel VT-d Interrupt Remapping enabled.
        (XEN) Intel VT-d Posted Interrupt not enabled.
        (XEN) Intel VT-d Shared EPT tables enabled.
        (XEN) I/O virtualisation enabled
        (XEN)  - Dom0 mode: Relaxed
        (XEN) Interrupt remapping enabled
        (XEN) Enabled directed EOI with ioapic_ack_old on!
        (XEN) ENABLING IO-APIC IRQs
        (XEN)  -> Using old ACK method
        (XEN) Allocated console ring of 128 KiB.
        (XEN) VMX: Supported advanced features:
        (XEN)  - APIC MMIO access virtualisation
        (XEN)  - APIC TPR shadow
        (XEN)  - Extended Page Tables (EPT)
        (XEN)  - Virtual-Processor Identifiers (VPID)
        (XEN)  - Virtual NMI
        (XEN)  - MSR direct-access bitmap
        (XEN)  - Unrestricted Guest
        (XEN)  - APIC Register Virtualization
        (XEN)  - Virtual Interrupt Delivery
        (XEN)  - Posted Interrupt Processing
        (XEN)  - VMCS shadowing
        (XEN)  - VM Functions
        (XEN)  - Virtualisation Exceptions
        (XEN)  - Page Modification Logging
        (XEN)  - TSC Scaling

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel