Re: Virtio in Xen on Arm (based on IOREQ concept)

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Tue, Jul 21, 2020 at 02:32:38PM +0100, Julien Grall wrote:
> Hi Roger,
> 
> On 21/07/2020 14:25, Roger Pau Monné wrote:
> > On Tue, Jul 21, 2020 at 01:31:48PM +0100, Julien Grall wrote:
> > > Hi Stefano,
> > > 
> > > On 20/07/2020 21:37, Stefano Stabellini wrote:
> > > > On Mon, 20 Jul 2020, Roger Pau Monné wrote:
> > > > > On Mon, Jul 20, 2020 at 10:40:40AM +0100, Julien Grall wrote:
> > > > > > 
> > > > > > 
> > > > > > On 20/07/2020 10:17, Roger Pau Monné wrote:
> > > > > > > On Fri, Jul 17, 2020 at 09:34:14PM +0300, Oleksandr wrote:
> > > > > > > > On 17.07.20 18:00, Roger Pau Monné wrote:
> > > > > > > > > On Fri, Jul 17, 2020 at 05:11:02PM +0300, Oleksandr 
> > > > > > > > > Tyshchenko wrote:
> > > > > > > > > Do you have any plans to try to upstream a modification to 
> > > > > > > > > the VirtIO
> > > > > > > > > spec so that grants (ie: abstract references to memory 
> > > > > > > > > addresses) can
> > > > > > > > > be used on the VirtIO ring?
> > > > > > > > 
> > > > > > > > But VirtIO spec hasn't been modified as well as VirtIO 
> > > > > > > > infrastructure in the
> > > > > > > > guest. Nothing to upsteam)
> > > > > > > 
> > > > > > > OK, so there's no intention to add grants (or a similar 
> > > > > > > interface) to
> > > > > > > the spec?
> > > > > > > 
> > > > > > > I understand that you want to support unmodified VirtIO 
> > > > > > > frontends, but
> > > > > > > I also think that long term frontends could negotiate with 
> > > > > > > backends on
> > > > > > > the usage of grants in the shared ring, like any other VirtIO 
> > > > > > > feature
> > > > > > > negotiated between the frontend and the backend.
> > > > > > > 
> > > > > > > This of course needs to be on the spec first before we can start
> > > > > > > implementing it, and hence my question whether a modification to 
> > > > > > > the
> > > > > > > spec in order to add grants has been considered.
> > > > > > The problem is not really the specification but the adoption in the
> > > > > > ecosystem. A protocol based on grant-tables would mostly only be 
> > > > > > used by Xen
> > > > > > therefore:
> > > > > >      - It may be difficult to convince a proprietary OS vendor to 
> > > > > > invest
> > > > > > resource on implementing the protocol
> > > > > >      - It would be more difficult to move in/out of Xen ecosystem.
> > > > > > 
> > > > > > Both, may slow the adoption of Xen in some areas.
> > > > > 
> > > > > Right, just to be clear my suggestion wasn't to force the usage of
> > > > > grants, but whether adding something along this lines was in the
> > > > > roadmap, see below.
> > > > > 
> > > > > > If one is interested in security, then it would be better to work 
> > > > > > with the
> > > > > > other interested parties. I think it would be possible to use a 
> > > > > > virtual
> > > > > > IOMMU for this purpose.
> > > > > 
> > > > > Yes, I've also heard rumors about using the (I assume VirtIO) IOMMU in
> > > > > order to protect what backends can map. This seems like a fine idea,
> > > > > and would allow us to gain the lost security without having to do the
> > > > > whole work ourselves.
> > > > > 
> > > > > Do you know if there's anything published about this? I'm curious
> > > > > about how and where in the system the VirtIO IOMMU is/should be
> > > > > implemented.
> > > > 
> > > > Not yet (as far as I know), but we have just started some discussons on
> > > > this topic within Linaro.
> > > > 
> > > > 
> > > > You should also be aware that there is another proposal based on
> > > > pre-shared-memory and memcpys to solve the virtio security issue:
> > > > 
> > > > https://marc.info/?l=linux-kernel&m=158807398403549
> > > > 
> > > > It would be certainly slower than the "virtio IOMMU" solution but it
> > > > would take far less time to develop and could work as a short-term
> > > > stop-gap.
> > > 
> > > I don't think I agree with this blank statement. In the case of "virtio
> > > IOMMU", you would need to potentially map/unmap pages every request which
> > > would result to a lot of back and forth to the hypervisor.
> > > 
> > > So it may turn out that pre-shared-memory may be faster on some setup.
> > 
> > AFAICT you could achieve the same with an IOMMU: pre-share (ie: add to
> > the device IOMMU page tables) a bunch of pages and keep bouncing data
> > to/from them in order to interact with the device, that way you could
> > avoid the map and unmaps (and is effectively how persistent grants
> > work in the blkif protocol).
> 
> Yes it is possible to do the same with the virtio IOMMU. I was more arguing
> on the statement that pre-shared-memory is going to be slower than the IOMMU
> case.
> 
> > 
> > The thread referenced by Stefano seems to point out this shared memory
> > model is targeted for very limited hypervisors that don't have the
> > capacity to trap, decode and emulate accesses to memory?
> 
> Technically we are in the same case for Xen on Arm as we don't have the
> IOREQ support yet. But I think IOREQ is worthwhile as it would enable
> existing unmodified Linux with virtio driver to boot on Xen.

Yes, I fully agree.

Roger.