[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: PCI devices passthrough on Arm design proposal

  • To: Julien Grall <julien@xxxxxxx>
  • From: Rahul Singh <Rahul.Singh@xxxxxxx>
  • Date: Mon, 20 Jul 2020 11:32:05 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vtK0mFLmn1WxwOxjjXutOXLAiAPq+X+utAToKjDl/eY=; b=Svww4EFHV/VVLmcBAp9dYuvqDgv9S6jXMYYkvQ7Tg7f8qneob5+KjlgwnMhIltbslk2b4WZJv83Eq/+KcwlhelIjFMT3GLECt67SCEolu4LwzNH3EfWwsMasGdJ8GFKzA64cuWgVcZeXGrTWM9+ytYAdAY/D4myIQzQui2AbCZRvUaly8AlWcd2IZQ+rC2K8hd+YznrJ8rVZ662XfF1mT5ov9vPLmPPS6GHglOkZ84MiZvWY+mG+UFePiNC8z/cSRygiT5LwZCUp2Vmw4DqpyZ8Z9Dk90zWsoLStLEqYNnCUYwkTUQEcHvU44uJYha1VgnJdLZCbwB5AjU3ZbCpwpA==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UbOb7/vj4e941f2Nmt4U5g2hDae9v596cJwuWvQUeRHaGLx5APFrIkAYRzMrsT15W/Ozw3todP+puKsSwyQ5P3Vo8uMpWUTnCZZCxdbFNuTCnQtEzgUeJzAH/vwAYy4jCqyEV1yJ0pa3brmTRTNo0vZQzTCiwivXZ8odzBL7PBmT14Q9jfXyo6sR06UL3m4vnFJa5Lrf+zzJzMGIXymTDtnt2qdUfVi6o/xkxUtx2WViDcEeOT7/1fpY6TyeLaJIp/eEssJf8AajWB6gAPRwszYAEm5+a9bJ6QMtGXLBeaW0Qh9qQXfN8HeS3ROim961rA9Fe6Rqnw3fqGikG0MXow==
  • Authentication-results-original: xen.org; dkim=none (message not signed) header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
  • Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>, Julien Grall <julien.grall.oss@xxxxxxxxx>, Bertrand Marquis <Bertrand.Marquis@xxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>, nd <nd@xxxxxxx>, Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • Delivery-date: Mon, 20 Jul 2020 11:32:23 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Nodisclaimer: true
  • Original-authentication-results: xen.org; dkim=none (message not signed) header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
  • Thread-topic: PCI devices passthrough on Arm design proposal

> On 18 Jul 2020, at 12:14 pm, Julien Grall <julien@xxxxxxx> wrote:
> On 18/07/2020 10:55, Bertrand Marquis wrote:
>>> On 17 Jul 2020, at 18:05, Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
>>> On Fri, Jul 17, 2020 at 03:47:25PM +0000, Bertrand Marquis wrote:
>>>>> On 17 Jul 2020, at 17:26, Julien Grall <julien@xxxxxxx> wrote:
>>>>> On 17/07/2020 15:47, Bertrand Marquis wrote:
>>>>>>>>> * Dom0Less implementation will require to have the capacity inside 
>>>>>>>>> Xen to discover the PCI devices (without depending on Dom0 to declare 
>>>>>>>>> them to Xen).
>>>>>>>>> # Enable the existing x86 virtual PCI support for ARM:
>>>>>>>>> The existing VPCI support available for X86 is adapted for Arm. When 
>>>>>>>>> the device is added to XEN via the hyper call 
>>>>>>>>> “PHYSDEVOP_pci_device_add”, VPCI handler for the config space access 
>>>>>>>>> is added to the PCI device to emulate the PCI devices.
>>>>>>>>> A MMIO trap handler for the PCI ECAM space is registered in XEN so 
>>>>>>>>> that when guest is trying to access the PCI config space, XEN will 
>>>>>>>>> trap the access and emulate read/write using the VPCI and not the 
>>>>>>>>> real PCI hardware.
>>>>>>>>> Limitation:
>>>>>>>>> * No handler is register for the MSI configuration.
>>>>>>>>> * Only legacy interrupt is supported and tested as of now, MSI is not 
>>>>>>>>> implemented and tested.
>>>>>>>> IIRC, legacy interrupt may be shared between two PCI devices. How do 
>>>>>>>> you plan to handle this on Arm?
>>>>>> We plan to fix this by adding proper support for MSI in the long term.
>>>>>> For the use case where MSI is not supported or not wanted we might have 
>>>>>> to find a way to forward the hardware interrupt to several guests to 
>>>>>> emulate some kind of shared interrupt.
>>>>> Sharing interrupts are a bit pain because you couldn't take advantage of 
>>>>> the direct EOI in HW and have to be careful if one guest doesn't EOI in 
>>>>> timely maneer.
>>>>> This is something I would rather avoid unless there is a real use case 
>>>>> for it.
>>>> I would expect that most recent hardware will support MSI and this
>>>> will not be needed.
>>> Well, PCI Express mandates MSI support, so while this is just a spec,
>>> I would expect most (if not all) devices to support MSI (or MSI-X), as
>>> Arm platforms haven't implemented legacy PCI anyway.
>> Yes that’s our assumption to. But we have to start somewhere so MSI is
>> planned but in a future step. I would think that supporting non MSI if not
>> impossible will be a lot more complex due to the interrupt sharing.
>> I do think that not supporting non MSI should be ok on Arm.
>>>> When MSI is not used, the only solution would be to enforce that
>>>> devices assigned to different guest are using different interrupts
>>>> which would limit the number of domains being able to use PCI
>>>> devices on a bus to 4 (if the enumeration can be modified correctly
>>>> to assign the interrupts properly).
>>>> If we all agree that this is an acceptable limitation then we would
>>>> not need the “interrupt sharing”.
>>> I might be easier to start by just supporting devices that have MSI
>>> (or MSI-X) and then move to legacy interrupts if required?
>> MSI support requires also some support in the interrupt controller part
>> on arm. So there is some work to achieve that.
>>> You should have most of the pieces you require already implemented
>>> since that's what x86 uses, and hence could reuse almost all of it?
>> Inside PCI probably but the GIC part will require some work.
> We already have an ITS implementation in Xen. This is required in order to 
> use PCI devices in DOM0 on thunder-x (there is no legacy interrupts 
> supported).
> It wasn't yet exposed to the guest because we didn't fully investigate the 
> security aspect of the implementation. However, for a tech preview this 
> should be sufficient.

Ok We will have a look for the ITS implementation once we will start working on 
the MSI support. Thanks for the pointer.
> -- 
> Julien Grall



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.