[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v10 13/17] vpci: add initial support for virtual PCI bus topology


  • To: Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • From: Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>
  • Date: Tue, 28 Nov 2023 23:45:34 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PUF2HtVycfSDWw3/A4DL4AWuV9Ws3sJQj0NAAblpbJ0=; b=K1y4QvEEkRAgYxlo9VubVVxv8oMaC5CJcwLmsoy0KW4vzvQFOTXHUjECUKQW3+xtj3xOtGVanyT4KthKBmSAZ5yIy+ulEe1xQnpxrLKQCXQT13A5x719Rb2EDz18Vgm79Zq40rNqMIhz7hQdY9IpXfEil7q0i8lNizPitl+EiNyrLtTCgMaOFJa8Qtwaqw8qe3gkDPePy7qnjjFguUxaxblJ7ysx7IezyYdGF9Y4EkK/qdzIKXBrz2ZP26mhfVlUXcjmO4VPlftukHGvpEtsQw0D7uT7bulU7GB1sxhJmL3H+/MlRiO2eS+LZMrOhCmwkm2ilifwPYLM6nWyL7z66Q==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ExKageRZj9JVBJepveNFNXbxdUkBT+ijsjM0gF7YG8vwurdsflOXT35BbVbNoCka/OeSmS2lhk+NTz+et5DxnTHo8Ov0pjJ/frVK+SbnwkloAJQfY1JbXduMTNSY0KPe/ZgjaFzKQDW/JxWys/saxv7kD4MN0e3zU75e1XM548BrtE5OQ3acKNnK9GXnQqZKLFuKgAbsLkO54tjYmddSpVG8IbL9w03NW3eY4RP3MItKs84eaeQiMB4SXTozBY1/Iw6o3KU8DoQQbI1JEmaZ6/039yxxk45JBNx5LlNxFxdaTpAU/vOj15onFmgF0eu0Hml1LTzQTgO/U6ni5W7gBA==
  • Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>, Julien Grall <julien@xxxxxxx>, Stewart Hildebrand <stewart.hildebrand@xxxxxxx>, Oleksandr Andrushchenko <Oleksandr_Andrushchenko@xxxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, George Dunlap <george.dunlap@xxxxxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>, Wei Liu <wl@xxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Tue, 28 Nov 2023 23:46:05 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Thread-index: AQHZ/Vi+wP3QsyKZ0EartcLXqtnBRLB9UyCAgAB7YgCAAAiUAIAAEHKAgADZXQCAAEyPgIAAGSwAgAAczgCAAAOwAIAALzmAgASxF4CAAZ+8gIAAs0sAgACdvQCAALuWAIAI2veA
  • Thread-topic: [PATCH v10 13/17] vpci: add initial support for virtual PCI bus topology

Hi Roger,

Roger Pau Monné <roger.pau@xxxxxxxxxx> writes:

> On Wed, Nov 22, 2023 at 01:18:32PM -0800, Stefano Stabellini wrote:
>> On Wed, 22 Nov 2023, Roger Pau Monné wrote:
>> > On Tue, Nov 21, 2023 at 05:12:15PM -0800, Stefano Stabellini wrote:
>> > > Let me expand on this. Like I wrote above, I think it is important that
>> > > Xen vPCI is the only in-use PCI Root Complex emulator. If it makes the
>> > > QEMU implementation easier, it is OK if QEMU emulates an unneeded and
>> > > unused PCI Root Complex. From Xen point of view, it doesn't exist.
>> > > 
>> > > In terms if ioreq registration, QEMU calls
>> > > xendevicemodel_map_pcidev_to_ioreq_server for each PCI BDF it wants to
>> > > emulate. That way, Xen vPCI knows exactly what PCI config space
>> > > reads/writes to forward to QEMU.
>> > > 
>> > > Let's say that:
>> > > - 00:02.0 is PCI passthrough device
>> > > - 00:03.0 is a PCI emulated device
>> > > 
>> > > QEMU would register 00:03.0 and vPCI would know to forward anything
>> > > related to 00:03.0 to QEMU, but not 00:02.0.
>> > 
>> > I think there's some work here so that we have a proper hierarchy
>> > inside of Xen.  Right now both ioreq and vpci expect to decode the
>> > accesses to the PCI config space, and setup (MM)IO handlers to trap
>> > ECAM, see vpci_ecam_{read,write}().
>> > 
>> > I think we want to move to a model where vPCI doesn't setup MMIO traps
>> > itself, and instead relies on ioreq to do the decoding and forwarding
>> > of accesses.  We need some work in order to represent an internal
>> > ioreq handler, but that shouldn't be too complicated.  IOW: vpci
>> > should register devices it's handling with ioreq, much like QEMU does.
>> 
>> I think this could be a good idea.
>> 
>> This would be the very first IOREQ handler implemented in Xen itself,
>> rather than outside of Xen. Some code refactoring might be required,
>> which worries me given that vPCI is at v10 and has been pending for
>> years. I think it could make sense as a follow-up series, not v11.
>
> That's perfectly fine for me, most of the series here just deal with
> the logic to intercept guest access to the config space and is
> completely agnostic as to how the accesses are intercepted.
>
>> I think this idea would be beneficial if, in the example above, vPCI
>> doesn't really need to know about device 00:03.0. vPCI registers via
>> IOREQ the PCI Root Complex and device 00:02.0 only, QEMU registers
>> 00:03.0, and everything works. vPCI is not involved at all in PCI config
>> space reads and writes for 00:03.0. If this is the case, then moving
>> vPCI to IOREQ could be good.
>
> Given your description above, with the root complex implemented in
> vPCI, we would need to mandate vPCI together with ioreqs even if no
> passthrough devices are using vPCI itself (just for the emulation of
> the root complex).  Which is fine, just wanted to mention the
> dependency.
>
>> On the other hand if vPCI actually needs to know that 00:03.0 exists,
>> perhaps because it changes something in the PCI Root Complex emulation
>> or vPCI needs to take some action when PCI config space registers of
>> 00:03.0 are written to, then I think this model doesn't work well. If
>> this is the case, then I think it would be best to keep vPCI as MMIO
>> handler and let vPCI forward to IOREQ when appropriate.
>
> At first approximation I don't think we would have such interactions,
> otherwise the whole premise of ioreq being able to register individual
> PCI devices would be broken.
>
> XenSever already has scenarios with two different user-space emulators
> (ie: two different ioreq servers) handling accesses to different
> devices in the same PCI bus, and there's no interaction with the root
> complex required.
>

Out of curiosity: how legacy PCI interrupts are handled in this case? In
my understanding, it is Root Complex's responsibility to propagate
correct IRQ levels to an interrupt controller?

[...]

-- 
WBR, Volodymyr

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.