Re: [PATCH v4 00/14] PCI devices passthrough on Arm

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

I committed patches #1, #4, #5 of this series


On Mon, 4 Oct 2021, Rahul Singh wrote:
> Hello All,
> 
> The purpose of this patch series is to add PCI passthrough support to Xen on
> Arm. PCI passthrough support on ARM is the collaboration work between EPAM and
> ARM. ARM submitted the partial RFC [1][2] last year to get early feedback. We
> tried to fix all the comments and added more features to this patch series.
> 
> Working POC with all the features can be found at [3]. Working POC is tested
> on x86 so that there will be no regression on x86. Design presentation can be
> found at [4]
> 
> PCI passthrough support is divided into different patches. This patch series
> includes following features: 
> 
> Preparatory work to implement the PCI passthrough support for the ARM:
> - Refactor MSI code.
> - Fixed compilation error when HAS_PCI enabled for ARM.
> 
> Discovering PCI Host Bridge in XEN:
> - PCI init to initialize the PCI driver.
> - PCI host bridge discovery in XEN and map the PCI ECAM configuration space to
>   the XEN memory.
> - PCI access functions.
> 
> Discovering PCI devices:
> - To support the PCI passthrough, XEN should be aware of the PCI
>   devices.
> - Hardware domain is in charge of doing the PCI enumeration and will discover
>   the PCI devices and then communicate to the XEN via a hypercall to add the
>   PCI devices in XEN.
> 
> Enable the existing x86 virtual PCI support for ARM:
> - Add VPCI trap handler for each of the PCI device added for config space
>   access.
> - Register the trap handler in XEN for each of the host bridge PCI ECAM config
>   space access.
> 
> Emulated PCI device tree node in libxl:
> - Create a virtual PCI device tree node in libxl to enable the guest OS to
>   discover the virtual PCI during guest boot.
> 
> This patch series does not inlcude the following features. Following features
> will be send for review in the next version of the patch series once initial
> patch series merged.
> 
> - VPCI support for DOMU guests (Non-identity mappings guest view of the BARs)
> - Virtual bus topology implementation
> - IOMMU related changes (generic, SMMUv2, SMMUv3)
> - MSI support for DOMU guests.
> - Virual ITS support for DOMU guests
> 
> [1] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg01184.html
> [2] 
> https://lists.xenproject.org/archives/html/xen-devel/2020-07/threads.html#01184
> [3] 
> https://gitlab.com/rahsingh/xen-integration/-/commits/pci-passthrough-upstream-all
> [4] 
> https://static.sched.com/hosted_files/xen2021/e4/PCI_Device_Passthrough_On_Arm.pdf
> 
> Oleksandr Andrushchenko (1):
>   xen/arm: Add support for Xilinx ZynqMP PCI host controller
> 
> Rahul Singh (13):
>   xen/pci: gate APEI support on ARM
>   xen/arm: xc_domain_ioport_permission(..) not supported on ARM.
>   xen/arm: Add PHYSDEVOP_pci_device_(*add/remove) support for ARM
>   xen/pci: Include asm/pci.h after pci_sbdf_t in xen/pci.h
>   xen/arm: Add support for PCI init to initialize the PCI driver.
>   xen/arm: Add cmdline boot option "pci-passthrough = <boolean>"
>   xen/arm: PCI host bridge discovery within XEN on ARM
>   xen/arm: Implement pci access functions
>   xen/domctl: Introduce XEN_DOMCTL_CDF_vpci flag
>   xen/arm: Enable the existing x86 virtual PCI support for ARM.
>   xen/arm: Transitional change to build HAS_VPCI on ARM.
>   arm/libxl: Emulated PCI device tree node in libxl
>   xen/arm: Add linux,pci-domain property for hwdom if not available.
> 
>  docs/misc/xen-command-line.pandoc   |   7 +
>  tools/include/libxl.h               |   6 +
>  tools/libs/ctrl/xc_domain.c         |   9 +
>  tools/libs/light/libxl_arm.c        | 105 ++++++++++
>  tools/libs/light/libxl_create.c     |   9 +
>  tools/libs/light/libxl_types.idl    |   1 +
>  tools/ocaml/libs/xc/xenctrl.ml      |   1 +
>  tools/ocaml/libs/xc/xenctrl.mli     |   1 +
>  tools/xl/xl_parse.c                 |   8 +
>  xen/arch/arm/Makefile               |   1 +
>  xen/arch/arm/domain.c               |   8 +-
>  xen/arch/arm/domain_build.c         |  19 ++
>  xen/arch/arm/pci/Makefile           |   5 +
>  xen/arch/arm/pci/ecam.c             |  61 ++++++
>  xen/arch/arm/pci/pci-access.c       | 140 ++++++++++++++
>  xen/arch/arm/pci/pci-host-common.c  | 287 ++++++++++++++++++++++++++++
>  xen/arch/arm/pci/pci-host-generic.c |  48 +++++
>  xen/arch/arm/pci/pci-host-zynqmp.c  |  65 +++++++
>  xen/arch/arm/pci/pci.c              |  63 ++++++
>  xen/arch/arm/physdev.c              |   5 +-
>  xen/arch/arm/vpci.c                 | 102 ++++++++++
>  xen/arch/arm/vpci.h                 |  36 ++++
>  xen/arch/x86/domain.c               |   6 +
>  xen/arch/x86/physdev.c              |  52 +----
>  xen/arch/x86/x86_64/physdev.c       |   2 +-
>  xen/common/domain.c                 |   2 +-
>  xen/drivers/passthrough/pci.c       |  20 +-
>  xen/drivers/pci/Makefile            |   1 +
>  xen/drivers/pci/physdev.c           |  86 +++++++++
>  xen/drivers/vpci/Makefile           |   3 +-
>  xen/drivers/vpci/header.c           |   2 +
>  xen/include/asm-arm/device.h        |   1 +
>  xen/include/asm-arm/domain.h        |   7 +-
>  xen/include/asm-arm/pci.h           |  85 ++++++++
>  xen/include/asm-x86/pci.h           |   8 +-
>  xen/include/public/arch-arm.h       |  21 +-
>  xen/include/public/domctl.h         |   4 +-
>  xen/include/xen/hypercall.h         |  11 ++
>  xen/include/xen/pci.h               |   5 +-
>  39 files changed, 1238 insertions(+), 65 deletions(-)
>  create mode 100644 xen/arch/arm/pci/ecam.c
>  create mode 100644 xen/arch/arm/pci/pci-access.c
>  create mode 100644 xen/arch/arm/pci/pci-host-common.c
>  create mode 100644 xen/arch/arm/pci/pci-host-generic.c
>  create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c
>  create mode 100644 xen/arch/arm/vpci.c
>  create mode 100644 xen/arch/arm/vpci.h
>  create mode 100644 xen/drivers/pci/physdev.c
> 
> -- 
> 2.25.1
>