[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v10 13/17] vpci: add initial support for virtual PCI bus topology



Hi Volodymyr,

On 17/11/2023 14:09, Volodymyr Babchuk wrote:

Hi Stefano,

Stefano Stabellini <sstabellini@xxxxxxxxxx> writes:

On Fri, 17 Nov 2023, Volodymyr Babchuk wrote:
I still think, no matter the BDF allocation scheme, that we should try
to avoid as much as possible to have two different PCI Root Complex
emulators. Ideally we would have only one PCI Root Complex emulated by
Xen. Having 2 PCI Root Complexes both of them emulated by Xen would be
tolerable but not ideal.

But what is exactly wrong with this setup?

[...]

The worst case I would like to avoid is to have
two PCI Root Complexes, one emulated by Xen and one emulated by QEMU.

This is how our setup works right now.

If we have:
- a single PCI Root Complex emulated in Xen
- Xen is safety certified
- individual Virtio devices emulated by QEMU with grants for memory

We can go very far in terms of being able to use Virtio in safety
use-cases. We might even be able to use Virtio (frontends) in a SafeOS.

On the other hand if we put an additional Root Complex in QEMU:
- we pay a price in terms of complexity of the codebase
- we pay a price in terms of resource utilization
- we have one additional problem in terms of using this setup with a
   SafeOS (one more device emulated by a non-safe component)

Having 2 PCI Root Complexes both emulated in Xen is a middle ground
solution because:
- we still pay a price in terms of resource utilization
- the code complexity goes up a bit but hopefully not by much
- there is no impact on safety compared to the ideal scenario

This is why I wrote that it is tolerable.

Ah, I see now. Yes, I am agree with this. Also I want to add some more
points:

- There is ongoing work on implementing virtio backends as a separate
   applications, written in Rust. Linaro are doing this part. Right now
   they are implementing only virtio-mmio, but if they want to provide
   virtio-pci as well, they will need a mechanism to plug only
   virtio-pci, without Root Complex. This is argument for using single Root
   Complex emulated in Xen.

- As far as I know (actually, Oleksandr told this to me), QEMU has no
   mechanism for exposing virtio-pci backends without exposing PCI root
   complex as well. Architecturally, there should be a PCI bus to which
   virtio-pci devices are connected. Or we need to make some changes to
   QEMU internals to be able to create virtio-pci backends that are not
   connected to any bus. Also, added benefit that PCI Root Complex
   emulator in QEMU handles legacy PCI interrupts for us. This is
   argument for separate Root Complex for QEMU.

As right now we have only virtio-pci backends provided by QEMU and this
setup is already working, I propose to stick to this
solution. Especially, taking into account that it does not require any
changes to hypervisor code.

I am not against two hostbridge as a temporary solution as long as this is not a one way door decision. I am not concerned about the hypervisor itself, I am more concerned about the interface exposed by the toolstack and QEMU.

To clarify, I don't particular want to have to maintain the two hostbridges solution once we can use a single hostbridge. So we need to be able to get rid of it without impacting the interface too much.

Cheers,

--
Julien Grall



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.