Re: [Xen-devel] PV drivers and zero copying

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Mon, 31 Jul 2017, Oleksandr Andrushchenko wrote:
> 3 Sharing with page exchange (XENMEM_exchange)
> ==============================================
> 
> This API was pointed to me by Stefano Stabellini as one of the possible ways
> to
> achieve zero copying and share physically contiguous buffers. It is used by
> x86
> SWIOTLB code (xen_create_contiguous_region, [5]), but as per my understanding
> this API cannot be used on ARM as of now [6].  Conclusion: not an option for
> ARM
> at the moment

Let me elaborate on this. The purpose of XENMEM_exchange is to exchange
a number of memory pages with an equal number of contiguous memory
pages, possibly even under 4G. The original purpose of the hypercall was
to get DMA-able memory.

So far, it has only been used by Dom0 on x86. Dom0 on ARM doesn't need
it because it is mapped 1:1 by default and device assignment is not
allowed without an IOMMU. However it should work on ARM too, as the
implementation is all common code in Xen. Also, looking at the
implementation (xen/common/memory.c:memory_exchange) it would seem that
it can be called from a DomU too (but I have never tried).

Thus, if you have a platform without IOMMU and you disabled the IOMMU
checks in Xen to assign a device to a DomU anyway, then you could use
this hypercall from DomU to get memory under 4G to be used for DMA with
this device.

As far as I can tell XENMEM_exchange could help in the design of
zero-copy PV protocols only to address this specific use case:

- you have a frontend in DomU and a backend in Dom0
- pages shared by DomU get mapped in Dom0 and potentially used for DMA
- the device has under 4G DMA restrictions

Normally Dom0 maps a DomU page, then at the time of using the mapped
page for DMA it checks whether it is suitable for DMA (under 4G if the
device requires so). If it is not, Dom0 uses a bounce buffer borrowed
from the swiotlb. Obviously this introduces one or two memcpys.

Instead, if DomU calls XENMEM_exchange to get memory under 4G, and
shares one of the pages with Dom0 via PV frontends, then Dom0 wouldn't
have to use a bounce buffer to do DMA to this page.

Does it make sense?

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel