[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v5 08/11] xen/arm: Enable the existing x86 virtual PCI support for ARM.


  • To: Rahul Singh <rahul.singh@xxxxxxx>
  • From: Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • Date: Wed, 13 Oct 2021 10:45:23 +0200
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vUMhzYqbOOMsmZJ3Q4tne0O0Ea6RvZswLzLM+SzuoBU=; b=hxFLXNXg612NRGSc7ufsQO8OStvETCmjJAuSgl70oNMchiNIrYErcHauLk6xoReNqG9PhaOqdDHMnRBOxBbryFxgxTUdmMbAn1ocD0tGS8XRgcxyIiekQoYzyk1t+vXTdZ43qo8IFVuw1DYG8j+tFHa2wf6YYLuxrONJ7hGi4QyfboUSPMLw9C3RekInAvj481mPZE/A5INC8WA8d7y79SUjLOmK4CtI8QvfXl7DBKgEQ2vF5R8RiNMgn3BC+uBdFfS+c1OS5J+9gJSihDqHyv1EAflFmg5vwzCXTY/CndxmcnF1uqCLgyysGVWpNv5EcqL76YKyIHgTy1Ad+zKFag==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hMxc7zXInWWlD0Vz0tTh8SYl8pQhqiE5girif0XsqUSR+33k2MM5xOCoVWCw83WYuTyPov5+dAvxPnfTYAlwJMuRPPpQqi0AusQqO2q1Z2uZsrt4NbsaFwgQO8G9tQH+SEpao3AACUECtFFfAW5KT8zJNzzWqfpdO6I05OBHBUnB/fJLWe3kXnycXMK6O56GSrNiz+7lyRROVeEwX1Xd0/IF5AHYLlL/BqwjDp1vhFdYLLJ7JfK7KXIJMJzzDlcTZ37p+SvRxL6S246tSFTMWrQMCtaK7sRnUIn6z+ywM9DZQk7j+zN9ID/nXTIbBwnpl1ZZUMWsZRSKu9JSqdi8Cg==
  • Authentication-results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
  • Cc: <xen-devel@xxxxxxxxxxxxxxxxxxxx>, <bertrand.marquis@xxxxxxx>, <Andre.Przywara@xxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, "Julien Grall" <julien@xxxxxxx>, Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, George Dunlap <george.dunlap@xxxxxxxxxx>, Ian Jackson <iwj@xxxxxxxxxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>, Wei Liu <wl@xxxxxxx>, Paul Durrant <paul@xxxxxxx>
  • Delivery-date: Wed, 13 Oct 2021 08:46:08 +0000
  • Ironport-data: A9a23:adMU/a0vXw9UKXYeqfbD5d13kn2cJEfYwER7XKvMYLTBsI5bpzwEy mIbXjrQOveONzDwctl+bNuz9hgC7cXWmNY1TVFupC1hF35El5HIVI+TRqvS04J+DSFhoGZPt Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970Es7wrVh2+aEvPDia++zk YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhk4xwk IpBsaCKdCgEMoH+w8kwQ0kEOnQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIIEgmlv1pEm8fD2a OtaRRxyayT7bjJSKG8JS80FgNm6iSyqG9FfgA3M/vdmi4TJ9yRwz7rFINfTYsaNR8heggCfv G2u11r+BhYWJdmO0w2v+3inhvLMtS7jUYdUH7q9ntZoilCOwm0YCDUNSEC25/K+jyaWQ9tCL 2QE9yxoqrI9nGS0SvHtUhv+p2SL1jYeRt5RHusS+AyLjK3O7G6xFmUCCzJMdtEinMs3XiAxk E+EmcvzAj5iu6HTTmiSnp+OpDOvPW4OLGkNZQcNVw5D6N7myKk5gwjTVN9lHOiwh8fsBDDr6 zmQqW41gLB7pdUQy6yx8FTDgjStjpvEVAg44kPQRG3Nxip/aJSoIbSy+0nE5OxJBI+DSx+Ku 31spiSFxLlQV9fXznXLGbhTWuHyjxqYDNHCqVIzHsF++zCRwUOMUIx80mpTCVc5bu9RLFcFf 3TvkQ9W4ZZSOl6jYql2f5+9BqwW8ETwKTj2fquLNoQWM/CdYCfCpXs0PRfBgAgBhWB1yflnU ap3Z/pAGprz5U5P9zGxW/sGmYEiwiQz1As/rriqkkz5j9JyiJOTIIrp0WdijMhlsstoQy2Pq r6z0vdmLT0ED4UShQGNoOYuwagidyRTOHwPg5U/mhS/CgRnAno9LPTa3KksfYdo94wMyLyUp i/hABcClAGj7ZEiFelsQis8AF8IdcwuxU/XwAR2ZQr4s5TdSdfHAFgjm2sfIuB8qb0LIQ9cR PgZYcSQasmjuRyckwnxmaLV9dQ4HDzy3FrmF3P8PFAXIs4xLySUq4SMVla+q0Ez4t+f6JJWT 0uIjViAH/Luhm1KUa7rVR5Y5wrp5yZCw7guAxKgzxs6UByEzbWG4hfZ15cfC8oNNQ/C1n2d0 QOXCg0fvu7Dv8k+99yhuExOh9zB/zJWEhUIEm/Fw6yxMCWGrGOvzZUZCLSDfCzHVXOy86KnP L0Hw/b5OfwBvVBLr4sjTOo7kfNgv4Pi9+1A0wBpPHTXdFD3WLluFWaLgJtUvapXy74H5QbvA hCT+sNXMKmiMd/+FAJDPxIsa+mOjKlGmjTb4fkvDl/94St7oOiOXUlIZkHegy1BNrpldogix L556sIR7gW+jDssM8qH0X8IpzjdcCRYXvx+5J8AAYLthg468X14YMTRWn3s/ZWCS9RQKU12c DWas7XP2uZHzU3YfnttSXWUhbhBhY4DsQxhxUMZIwjbgcLMg/I60UED8TkzSQgJnBxL3/gqZ zpuPkxxY66P4y1plI5IWGX1Q1NNAxiQ+0rQzVoVlTKGExn0BzKVdGBta/yQ+E058n5HemkJ9 b6V/2/pTDL2cZyjxSA1Q0Nk96TuQNEZGtcuQyx78xBpx6UHXAc=
  • Ironport-hdrordr: A9a23:1bfc+qEuxJ803n8spLqFe5HXdLJyesId70hD6qkvc3Jom52j+P xGws526faVslYssHFJo6HnBEClewKgyXcT2/hsAV7CZnidhILMFuBfBOTZsljd8kHFh4pgPO JbAtdD4b7LfChHZKTBkXGF+r8bqbHtms3Y5pa9vgRQpENRGtpdBm9Ce3em+yZNNXB77PQCZf 2hDp0tnUvfRZ1bVLX2OlA1G8z44/HbnpPvZhALQzYh9Qm1lDutrJr3CQKR0BsyWy5Ghe5Kyx mKryXJooGY992rwB7V0GHeq7xQhdva09NGQOiBkNIcJDnAghuhIK5hR7qBljYop/zH0idkrP D85zMbe+hj4XLYeW+45TPrxgnbyT4rr0TvzFeJ6EGT6vDRdXYfMY5slIhZehzW5w4Lp9dnyp 9G2Gqfqt5+EQ7AtD6V3amKazha0m6P5VYym+8aiHJSFaEEbqVKkIAZ9ERJVL8dASPB7pw9Gu UGNrCd2B9vSyLeU5nlhBgq/DT1NU5DWStuA3Jy+/B96gIm3kyQlCAjtYoidnRpzuN0d3AL3Z WcDkzE/Is+OfP+VpgNcdvpd/HHQ1AlcSi8Ql56Hm6XYJ3vG0i94aIfs49Fqd1DRvQzve0Pcd L6IQtliVI=
  • Ironport-sdr: ja8r+H3BacIvJs3drW2WRb75Q5DeosQ/HLLiH84WbzOFi954djHRD7aRjGTEKrGvYvkt2mhZMs E46ARktmcSrlkH+K+3RXgz0x0Oldt84PBWhtVsNgHAmG9/RJ/nfWhWX9deqqEHk62e/FOcfsCM fxViE9jmWi3168Ucq9n9LA1/WwyIk+snS2r+owPQHYW17h/vYEGJD6ROnq3cXOice4xiZVuT7X RqznJVxPN7cnvodHV1PmYS3IGQvIosu1nVKPgVIPrbiz7roDlSSpWq6U46fzkrURWYcGy+IfIs xTuYlWw5vO3Ne3Llov09eClN
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On Wed, Oct 06, 2021 at 06:40:34PM +0100, Rahul Singh wrote:
> The existing VPCI support available for X86 is adapted for Arm.
> When the device is added to XEN via the hyper call
> “PHYSDEVOP_pci_device_add”, VPCI handler for the config space
> access is added to the Xen to emulate the PCI devices config space.
> 
> A MMIO trap handler for the PCI ECAM space is registered in XEN
> so that when guest is trying to access the PCI config space,XEN
> will trap the access and emulate read/write using the VPCI and
> not the real PCI hardware.
> 
> For Dom0less systems scan_pci_devices() would be used to discover the
> PCI device in XEN and VPCI handler will be added during XEN boots.
> 
> Signed-off-by: Rahul Singh <rahul.singh@xxxxxxx>
> Reviewed-by: Stefano Stabellini <sstabellini@xxxxxxxxxx>
> ---
> Change in v5:
> - Add pci_cleanup_msi(pdev) in cleanup part.
> - Added Reviewed-by: Stefano Stabellini <sstabellini@xxxxxxxxxx>
> Change in v4:
> - Move addition of XEN_DOMCTL_CDF_vpci flag to separate patch
> Change in v3:
> - Use is_pci_passthrough_enabled() in place of pci_passthrough_enabled 
> variable
> - Reject XEN_DOMCTL_CDF_vpci for x86 in arch_sanitise_domain_config()
> - Remove IS_ENABLED(CONFIG_HAS_VPCI) from has_vpci()
> Change in v2:
> - Add new XEN_DOMCTL_CDF_vpci flag
> - modify has_vpci() to include XEN_DOMCTL_CDF_vpci
> - enable vpci support when pci-passthough option is enabled.
> ---
> ---
>  xen/arch/arm/Makefile         |   1 +
>  xen/arch/arm/domain.c         |   4 ++
>  xen/arch/arm/domain_build.c   |   3 +
>  xen/arch/arm/vpci.c           | 102 ++++++++++++++++++++++++++++++++++
>  xen/arch/arm/vpci.h           |  36 ++++++++++++
>  xen/drivers/passthrough/pci.c |  18 ++++++
>  xen/include/asm-arm/domain.h  |   7 ++-
>  xen/include/asm-x86/pci.h     |   2 -
>  xen/include/public/arch-arm.h |   7 +++
>  xen/include/xen/pci.h         |   2 +
>  10 files changed, 179 insertions(+), 3 deletions(-)
>  create mode 100644 xen/arch/arm/vpci.c
>  create mode 100644 xen/arch/arm/vpci.h
> 
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 44d7cc81fa..fb9c976ea2 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -7,6 +7,7 @@ ifneq ($(CONFIG_NO_PLAT),y)
>  obj-y += platforms/
>  endif
>  obj-$(CONFIG_TEE) += tee/
> +obj-$(CONFIG_HAS_VPCI) += vpci.o
>  
>  obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o
>  obj-y += bootfdt.init.o
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 36138c1b2e..fbb52f78f1 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -39,6 +39,7 @@
>  #include <asm/vgic.h>
>  #include <asm/vtimer.h>
>  
> +#include "vpci.h"
>  #include "vuart.h"
>  
>  DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
> @@ -767,6 +768,9 @@ int arch_domain_create(struct domain *d,
>      if ( is_hardware_domain(d) && (rc = domain_vuart_init(d)) )
>          goto fail;
>  
> +    if ( (rc = domain_vpci_init(d)) != 0 )
> +        goto fail;
> +
>      return 0;
>  
>  fail:
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index c5afbe2e05..f4c89bde8c 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -3053,6 +3053,9 @@ void __init create_dom0(void)
>      if ( iommu_enabled )
>          dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>  
> +    if ( is_pci_passthrough_enabled() )
> +        dom0_cfg.flags |= XEN_DOMCTL_CDF_vpci;
> +
>      dom0 = domain_create(0, &dom0_cfg, true);
>      if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
>          panic("Error creating domain 0\n");
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> new file mode 100644
> index 0000000000..76c12b9281
> --- /dev/null
> +++ b/xen/arch/arm/vpci.c
> @@ -0,0 +1,102 @@
> +/*
> + * xen/arch/arm/vpci.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +#include <xen/sched.h>
> +
> +#include <asm/mmio.h>
> +
> +#define REGISTER_OFFSET(addr)  ( (addr) & 0x00000fff)
> +
> +/* Do some sanity checks. */
> +static bool vpci_mmio_access_allowed(unsigned int reg, unsigned int len)
> +{
> +    /* Check access size. */
> +    if ( len > 8 )
> +        return false;
> +
> +    /* Check that access is size aligned. */
> +    if ( (reg & (len - 1)) )
> +        return false;
> +
> +    return true;
> +}
> +
> +static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
> +                          register_t *r, void *p)
> +{
> +    unsigned int reg;
> +    pci_sbdf_t sbdf;
> +    unsigned long data = ~0UL;
> +    unsigned int size = 1U << info->dabt.size;
> +
> +    sbdf.sbdf = MMCFG_BDF(info->gpa);
> +    reg = REGISTER_OFFSET(info->gpa);
> +
> +    if ( !vpci_mmio_access_allowed(reg, size) )
> +        return 0;
> +
> +    data = vpci_read(sbdf, reg, min(4u, size));
> +    if ( size == 8 )
> +        data |= (uint64_t)vpci_read(sbdf, reg + 4, 4) << 32;
> +
> +    *r = data;
> +
> +    return 1;
> +}
> +
> +static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
> +                           register_t r, void *p)
> +{
> +    unsigned int reg;
> +    pci_sbdf_t sbdf;
> +    unsigned long data = r;
> +    unsigned int size = 1U << info->dabt.size;
> +
> +    sbdf.sbdf = MMCFG_BDF(info->gpa);
> +    reg = REGISTER_OFFSET(info->gpa);
> +
> +    if ( !vpci_mmio_access_allowed(reg, size) )
> +        return 0;
> +
> +    vpci_write(sbdf, reg, min(4u, size), data);
> +    if ( size == 8 )
> +        vpci_write(sbdf, reg + 4, 4, data >> 32);

I think those two helpers (and vpci_mmio_access_allowed) are very
similar to the existing x86 ones (see vpci_mmcfg_{read,write}), up to
the point where I would consider moving the shared code to vpci.c as
vpci_ecam_{read,write} and call them from the arch specific trap
handlers.

Thanks, Roger.



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.