[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: PCI devices passthrough on Arm design proposal

On Fri, Jul 17, 2020 at 03:47:25PM +0000, Bertrand Marquis wrote:
> > On 17 Jul 2020, at 17:26, Julien Grall <julien@xxxxxxx> wrote:
> > On 17/07/2020 15:47, Bertrand Marquis wrote:
> >>>>> * Dom0Less implementation will require to have the capacity inside Xen 
> >>>>> to discover the PCI devices (without depending on Dom0 to declare them 
> >>>>> to Xen).
> >>>>> 
> >>>>> # Enable the existing x86 virtual PCI support for ARM:
> >>>>> 
> >>>>> The existing VPCI support available for X86 is adapted for Arm. When 
> >>>>> the device is added to XEN via the hyper call 
> >>>>> “PHYSDEVOP_pci_device_add”, VPCI handler for the config space access is 
> >>>>> added to the PCI device to emulate the PCI devices.
> >>>>> 
> >>>>> A MMIO trap handler for the PCI ECAM space is registered in XEN so that 
> >>>>> when guest is trying to access the PCI config space, XEN will trap the 
> >>>>> access and emulate read/write using the VPCI and not the real PCI 
> >>>>> hardware.
> >>>>> 
> >>>>> Limitation:
> >>>>> * No handler is register for the MSI configuration.
> >>>>> * Only legacy interrupt is supported and tested as of now, MSI is not 
> >>>>> implemented and tested.
> >>>> IIRC, legacy interrupt may be shared between two PCI devices. How do you 
> >>>> plan to handle this on Arm?
> >> We plan to fix this by adding proper support for MSI in the long term.
> >> For the use case where MSI is not supported or not wanted we might have to 
> >> find a way to forward the hardware interrupt to several guests to emulate 
> >> some kind of shared interrupt.
> > 
> > Sharing interrupts are a bit pain because you couldn't take advantage of 
> > the direct EOI in HW and have to be careful if one guest doesn't EOI in 
> > timely maneer.
> > 
> > This is something I would rather avoid unless there is a real use case for 
> > it.
> I would expect that most recent hardware will support MSI and this
> will not be needed.

Well, PCI Express mandates MSI support, so while this is just a spec,
I would expect most (if not all) devices to support MSI (or MSI-X), as
Arm platforms haven't implemented legacy PCI anyway.

> When MSI is not used, the only solution would be to enforce that
> devices assigned to different guest are using different interrupts
> which would limit the number of domains being able to use PCI
> devices on a bus to 4 (if the enumeration can be modified correctly
> to assign the interrupts properly).
> If we all agree that this is an acceptable limitation then we would
> not need the “interrupt sharing”.

I might be easier to start by just supporting devices that have MSI
(or MSI-X) and then move to legacy interrupts if required?

You should have most of the pieces you require already implemented
since that's what x86 uses, and hence could reuse almost all of it?

IIRC Julien even said that Arm was likely to require much less traps
than x86 for accesses to MSI and MSI-X since you could allow untrusted
guests to write directly to the registers as there's another piece of
hardware that would already translate the interrupts?

I think it's fine to use this workaround while you don't have MSI
support in order to start testing and upstreaming stuff, but maybe
that shouldn't be committed?




Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.