[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Virtio in Xen on Arm (based on IOREQ concept)

Hi Alex,

Thank you for your feedback!

On 21/07/2020 15:15, Alex Bennée wrote:
Julien Grall <julien@xxxxxxx> writes:

(+ Andree for the vGIC).

Hi Stefano,

On 20/07/2020 21:38, Stefano Stabellini wrote:
On Fri, 17 Jul 2020, Oleksandr wrote:
*A few word about solution:*
As it was mentioned at [1], in order to implement virtio-mmio Xen on Arm
Any plans for virtio-pci? Arm seems to be moving to the PCI bus, and
it would be very interesting from a x86 PoV, as I don't think
virtio-mmio is something that you can easily use on x86 (or even use
at all).

Being honest I didn't consider virtio-pci so far. Julien's PoC (we are based
on) provides support for the virtio-mmio transport

which is enough to start working around VirtIO and is not as complex as
virtio-pci. But it doesn't mean there is no way for virtio-pci in Xen.

I think, this could be added in next steps. But the nearest target is
virtio-mmio approach (of course if the community agrees on that).

Aside from complexity and easy-of-development, are there any other
architectural reasons for using virtio-mmio?


For instance, what's your take on notifications with virtio-mmio? How
are they modelled today?

The backend will notify the frontend using an SPI. The other way around
(frontend -> backend) is based on an MMIO write.

We have an interface to allow the backend to control whether the
interrupt level (i.e. low, high). However, the "old" vGIC doesn't handle
properly level interrupts. So we would end up to treat level interrupts
as edge.

Technically, the problem is already existing with HW interrupts, but the
HW should fire it again if the interrupt line is still asserted. Another
issue is the interrupt may fire even if the interrupt line was
deasserted (IIRC this caused some interesting problem with the Arch timer).

I am a bit concerned that the issue will be more proeminent for virtual
interrupts. I know that we have some gross hack in the vpl011 to handle
a level interrupts. So maybe it is time to switch to the new vGIC?

Are they good enough or do we need MSIs?

I am not sure whether virtio-mmio supports MSIs. However for virtio-pci,
MSIs is going to be useful to improve performance. This may mean to
expose an ITS, so we would need to add support for guest.

virtio-mmio doesn't support MSI's at the moment although there have been
proposals to update the spec to allow them. At the moment the cost of
reading the ISR value and then writing an ack in vm_interrupt:

        /* Read and acknowledge interrupts */
        status = readl(vm_dev->base + VIRTIO_MMIO_INTERRUPT_STATUS);
        writel(status, vm_dev->base + VIRTIO_MMIO_INTERRUPT_ACK);

Hmmmm, the current way to handle MMIO is the following:
    * pause the vCPU
    * Forward the access to the backend domain
    * Schedule the backend domain
    * Wait for the access to be handled
    * unpause the vCPU

So the sequence is going to be fairly expensive on Xen.

puts an extra vmexit cost to trap an emulate each exit. Getting an MSI
via an exitless access to the GIC would be better I think.
I'm not quite
sure what the path to IRQs from Xen is.

vmexit on Xen on Arm is pretty cheap compare to KVM as we don't save a lot of things. In this situation, they handling an extra trap for the interrupt is likely to be meaningless compare to the sequence above.

I am assuming the sequence is also going to be used by the MSIs, right?

It feels to me that it would be worth spending time to investigate the cost of that sequence. It might be possible to optimize the ACK and avoid to wait for the backend to handle the access.


Julien Grall



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.