Re: [PATCH v5 00/14] PCI devices passthrough on Arm, part 3

[Oleksandr Andrushchenko <Oleksandr_Andrushchenko@xxxxxxxx>]

Dear rest maintainers!

Could you please review this series which seems to get stuck?

Thank you in advance,
Oleksandr

On 25.11.21 13:02, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@xxxxxxxx>
>
> Hi, all!
>
> 1. This patch series is focusing on vPCI and adds support for non-identity
> PCI BAR mappings which is required while passing through a PCI device to
> a guest. The highlights are:
>
> - Add relevant vpci register handlers when assigning PCI device to a domain
>    and remove those when de-assigning. This allows having different
>    handlers for different domains, e.g. hwdom and other guests.
>
> - Emulate guest BAR register values based on physical BAR values.
>    This allows creating a guest view of the registers and emulates
>    size and properties probe as it is done during PCI device enumeration by
>    the guest.
>
> - Instead of handling a single range set, that contains all the memory
>    regions of all the BARs and ROM, have them per BAR.
>
> - Take into account guest's BAR view and program its p2m accordingly:
>    gfn is guest's view of the BAR and mfn is the physical BAR value as set
>    up by the host bridge in the hardware domain.
>    This way hardware doamin sees physical BAR values and guest sees
>    emulated ones.
>
> 2. The series also adds support for virtual PCI bus topology for guests:
>   - We emulate a single host bridge for the guest, so segment is always 0.
>   - The implementation is limited to 32 devices which are allowed on
>     a single PCI bus.
>   - The virtual bus number is set to 0, so virtual devices are seen
>     as embedded endpoints behind the root complex.
>
> 3. The series has complete re-work of the locking scheme used/absent before 
> with
> the help of the work started by Roger [1]:
> [PATCH v5 03/13] vpci: move lock outside of struct vpci
>
> This way the lock can be used to check whether vpci is present, and
> removal can be performed while holding the lock, in order to make
> sure there are no accesses to the contents of the vpci struct.
> Previously removal could race with vpci_read for example, since the
> lock was dropped prior to freeing pdev->vpci.
> This also solves synchronization issues between all vPCI code entities
> which could run in parallel.
>
> 4. There is an outstanding TODO left unimplemented by this series:
> for unprivileged guests vpci_{read|write} need to be re-worked
> to not passthrough accesses to the registers not explicitly handled
> by the corresponding vPCI handlers: without fixing that passthrough
> to guests is completely unsafe as Xen allows them full access to
> the registers.
>
> Xen needs to be sure that every register a guest accesses is not
> going to cause the system to malfunction, so Xen needs to keep a
> list of the registers it is safe for a guest to access.
>
> For example, we should only expose the PCI capabilities that we know
> are safe for a guest to use, i.e.: MSI and MSI-X initially.
> The rest of the capabilities should be blocked from guest access,
> unless we audit them and declare safe for a guest to access.
>
> As a reference we might want to look at the approach currently used
> by QEMU in order to do PCI passthrough. A very limited set of PCI
> capabilities known to be safe for untrusted access are exposed to the
> guest and registers need to be explicitly handled or else access is
> rejected. Xen needs a fairly similar model in vPCI or else none of
> this will be safe for unprivileged access.
>
> 5. The series was also tested on:
>   - x86 PVH Dom0 and doesn't break it.
>   - x86 HVM with PCI passthrough to DomU and doesn't break it.
>
> Thank you,
> Oleksandr
>
> [1] 
> https://lore.kernel.org/xen-devel/20180717094830.54806-2-roger.pau@xxxxxxxxxx/
>
> Oleksandr Andrushchenko (13):
>    rangeset: add RANGESETF_no_print flag
>    vpci: fix function attributes for vpci_process_pending
>    vpci: cancel pending map/unmap on vpci removal
>    vpci: add hooks for PCI device assign/de-assign
>    vpci/header: implement guest BAR register handlers
>    vpci/header: handle p2m range sets per BAR
>    vpci/header: program p2m with guest BAR view
>    vpci/header: emulate PCI_COMMAND register for guests
>    vpci/header: reset the command register when adding devices
>    vpci: add initial support for virtual PCI bus topology
>    xen/arm: translate virtual PCI bus topology for guests
>    xen/arm: account IO handlers for emulated PCI MSI-X
>    vpci: add TODO for the registers not explicitly handled
>
> Roger Pau Monne (1):
>    vpci: move lock outside of struct vpci
>
>   tools/tests/vpci/emul.h       |   5 +-
>   tools/tests/vpci/main.c       |   4 +-
>   xen/arch/arm/vpci.c           |  33 +++-
>   xen/arch/x86/hvm/vmsi.c       |   8 +-
>   xen/common/rangeset.c         |   5 +-
>   xen/drivers/Kconfig           |   4 +
>   xen/drivers/passthrough/pci.c |  11 ++
>   xen/drivers/vpci/header.c     | 352 +++++++++++++++++++++++++++-------
>   xen/drivers/vpci/msi.c        |  11 +-
>   xen/drivers/vpci/msix.c       |   8 +-
>   xen/drivers/vpci/vpci.c       | 252 +++++++++++++++++++++---
>   xen/include/xen/pci.h         |   6 +
>   xen/include/xen/rangeset.h    |   7 +-
>   xen/include/xen/sched.h       |   8 +
>   xen/include/xen/vpci.h        |  47 ++++-
>   15 files changed, 644 insertions(+), 117 deletions(-)
>