[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: PCI devices passthrough on Arm design proposal

On 18/07/2020 10:55, Bertrand Marquis wrote:

On 17 Jul 2020, at 18:05, Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:

On Fri, Jul 17, 2020 at 03:47:25PM +0000, Bertrand Marquis wrote:
On 17 Jul 2020, at 17:26, Julien Grall <julien@xxxxxxx> wrote:
On 17/07/2020 15:47, Bertrand Marquis wrote:
* Dom0Less implementation will require to have the capacity inside Xen to 
discover the PCI devices (without depending on Dom0 to declare them to Xen).

# Enable the existing x86 virtual PCI support for ARM:

The existing VPCI support available for X86 is adapted for Arm. When the device is added to 
XEN via the hyper call “PHYSDEVOP_pci_device_add”, VPCI handler for the config 
space access is added to the PCI device to emulate the PCI devices.

A MMIO trap handler for the PCI ECAM space is registered in XEN so that when 
guest is trying to access the PCI config space, XEN will trap the access and 
emulate read/write using the VPCI and not the real PCI hardware.

* No handler is register for the MSI configuration.
* Only legacy interrupt is supported and tested as of now, MSI is not 
implemented and tested.
IIRC, legacy interrupt may be shared between two PCI devices. How do you plan 
to handle this on Arm?
We plan to fix this by adding proper support for MSI in the long term.
For the use case where MSI is not supported or not wanted we might have to find 
a way to forward the hardware interrupt to several guests to emulate some kind 
of shared interrupt.

Sharing interrupts are a bit pain because you couldn't take advantage of the 
direct EOI in HW and have to be careful if one guest doesn't EOI in timely 

This is something I would rather avoid unless there is a real use case for it.

I would expect that most recent hardware will support MSI and this
will not be needed.

Well, PCI Express mandates MSI support, so while this is just a spec,
I would expect most (if not all) devices to support MSI (or MSI-X), as
Arm platforms haven't implemented legacy PCI anyway.

Yes that’s our assumption to. But we have to start somewhere so MSI is
planned but in a future step. I would think that supporting non MSI if not
impossible will be a lot more complex due to the interrupt sharing.
I do think that not supporting non MSI should be ok on Arm.

When MSI is not used, the only solution would be to enforce that
devices assigned to different guest are using different interrupts
which would limit the number of domains being able to use PCI
devices on a bus to 4 (if the enumeration can be modified correctly
to assign the interrupts properly).

If we all agree that this is an acceptable limitation then we would
not need the “interrupt sharing”.

I might be easier to start by just supporting devices that have MSI
(or MSI-X) and then move to legacy interrupts if required?

MSI support requires also some support in the interrupt controller part
on arm. So there is some work to achieve that.

You should have most of the pieces you require already implemented
since that's what x86 uses, and hence could reuse almost all of it?

Inside PCI probably but the GIC part will require some work.

We already have an ITS implementation in Xen. This is required in order to use PCI devices in DOM0 on thunder-x (there is no legacy interrupts supported).

It wasn't yet exposed to the guest because we didn't fully investigate the security aspect of the implementation. However, for a tech preview this should be sufficient.

Julien Grall



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.