Re: Virtio in Xen on Arm (based on IOREQ concept)

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Tue, 21 Jul 2020, Alex Bennée wrote:
> Julien Grall <julien@xxxxxxx> writes:
> 
> > Hi Stefano,
> >
> > On 20/07/2020 21:37, Stefano Stabellini wrote:
> >> On Mon, 20 Jul 2020, Roger Pau Monné wrote:
> >>> On Mon, Jul 20, 2020 at 10:40:40AM +0100, Julien Grall wrote:
> >>>>
> >>>>
> >>>> On 20/07/2020 10:17, Roger Pau Monné wrote:
> >>>>> On Fri, Jul 17, 2020 at 09:34:14PM +0300, Oleksandr wrote:
> >>>>>> On 17.07.20 18:00, Roger Pau Monné wrote:
> >>>>>>> On Fri, Jul 17, 2020 at 05:11:02PM +0300, Oleksandr Tyshchenko wrote:
> >>>>>>> Do you have any plans to try to upstream a modification to the VirtIO
> >>>>>>> spec so that grants (ie: abstract references to memory addresses) can
> >>>>>>> be used on the VirtIO ring?
> >>>>>>
> >>>>>> But VirtIO spec hasn't been modified as well as VirtIO infrastructure 
> >>>>>> in the
> >>>>>> guest. Nothing to upsteam)
> >>>>>
> >>>>> OK, so there's no intention to add grants (or a similar interface) to
> >>>>> the spec?
> >>>>>
> >>>>> I understand that you want to support unmodified VirtIO frontends, but
> >>>>> I also think that long term frontends could negotiate with backends on
> >>>>> the usage of grants in the shared ring, like any other VirtIO feature
> >>>>> negotiated between the frontend and the backend.
> >>>>>
> >>>>> This of course needs to be on the spec first before we can start
> >>>>> implementing it, and hence my question whether a modification to the
> >>>>> spec in order to add grants has been considered.
> >>>> The problem is not really the specification but the adoption in the
> >>>> ecosystem. A protocol based on grant-tables would mostly only be used by 
> >>>> Xen
> >>>> therefore:
> >>>>     - It may be difficult to convince a proprietary OS vendor to invest
> >>>> resource on implementing the protocol
> >>>>     - It would be more difficult to move in/out of Xen ecosystem.
> >>>>
> >>>> Both, may slow the adoption of Xen in some areas.
> >>>
> >>> Right, just to be clear my suggestion wasn't to force the usage of
> >>> grants, but whether adding something along this lines was in the
> >>> roadmap, see below.
> >>>
> >>>> If one is interested in security, then it would be better to work with 
> >>>> the
> >>>> other interested parties. I think it would be possible to use a virtual
> >>>> IOMMU for this purpose.
> >>>
> >>> Yes, I've also heard rumors about using the (I assume VirtIO) IOMMU in
> >>> order to protect what backends can map. This seems like a fine idea,
> >>> and would allow us to gain the lost security without having to do the
> >>> whole work ourselves.
> >>>
> >>> Do you know if there's anything published about this? I'm curious
> >>> about how and where in the system the VirtIO IOMMU is/should be
> >>> implemented.
> >> 
> >> Not yet (as far as I know), but we have just started some discussons on
> >> this topic within Linaro.
> >> 
> >> 
> >> You should also be aware that there is another proposal based on
> >> pre-shared-memory and memcpys to solve the virtio security issue:
> >> 
> >> https://marc.info/?l=linux-kernel&m=158807398403549
> >> 
> >> It would be certainly slower than the "virtio IOMMU" solution but it
> >> would take far less time to develop and could work as a short-term
> >> stop-gap.
> >
> > I don't think I agree with this blank statement. In the case of "virtio 
> > IOMMU", you would need to potentially map/unmap pages every request 
> > which would result to a lot of back and forth to the hypervisor.

Yes, that's true.


> Can a virtio-iommu just set bounds when a device is initialised as to
> where memory will be in the kernel address space?

First let me premise to avoid possible miscommunication that what Julien
and I are calling "virtio IOMMU" is not an existing virtio-iommu driver
of some sort, but an idea for a cross-domain virtual IOMMU for the sake
of the frontends to explicitly permit memory to be accessed by the
backends. Hopefully it was clear already but better be sure :-)


If you are asking whether it would be possible to use the virtual IOMMU
just to setup memory at startup time, then it certainly could, but
effectively we would end up with one of the following scenarios:

1) one pre-shared bounce buffer
Effectively the same as https://marc.info/?l=linux-kernel&m=158807398403549
still requires memcpys
could still be nicer than Qualcomm's proposal because easier to
configure?

2) all domU memory allowed access to the backend
Not actually any more secure than placing the backends in dom0


Otherwise we need the dynamic maps/unmaps.

For completeness, if we could write the whole software stack from
scratch, it would also be possible to architect a protocol (like
virtio-net) and the software stack above it to always allocate memory
from a given buffer (the pre-shared buffer), hence greatly reducing the
amount of required memcpys, maybe even down to zero. In reality, most
interfaces in Linux and POSIX userspace expect the application to be the
one providing the buffer, hence they would require memcpys in the kernel
to move data between the user-provided buffers and the pre-shared buffers.