Re: [Xen-devel] PV drivers and zero copying

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 31/07/17 10:03, Paul Durrant wrote:
>> -----Original Message-----
> [snip]
>> Comparison for display use-case
>> ===============================
>>
>> 1 Number of grant references used
>> 1-1 grant references: nr_pages
>> 1-2 GNTTABOP_transfer: nr_pages
>> 1-3 XENMEM_exchange: not an option
>>
>> 2 Effect of DomU crash on Dom0 (its mapped pages)
>> 2-1 grant references: pages can be unmapped by Dom0, Dom0 is fully
>> recovered
>> 2-2 GNTTABOP_transfer: pages will be returned to the Hypervisor, lost
>> for Dom0
>> 2-3 XENMEM_exchange: not an option
>>
>> 3 Security issues from sharing Dom0 pages to DomU
>> 1-1 grant references: none
>> 1-2 GNTTABOP_transfer: none
>> 1-3 XENMEM_exchange: not an option
>>
>> At the moment approach 1 with granted references seems to be a winner for
>> sharing buffers both ways, e.g. Dom0 -> DomU and DomU -> Dom0.
>>
>> Conclusion
>> ==========
>>
>> I would like to get some feedback from the community on which approach
>> is more
>> suitable for sharing large buffers and to have a clear vision on cons
>> and pros
>> of each one: please feel free to add other metrics I missed and correct
>> the ones
>> I commented on.  I would appreciate help on comparing approaches 2 and 3
>> as I
>> have little knowledge of these APIs (2 seems to be addressed by
>> Christopher, and
>> 3 seems to be relevant to what Konrad/Stefano do WRT SWIOTLB).
>>
> Hi,
>
>   I once implemented a scheme where network frontends used memory granted 
> from backends and this hit quite a few problems:
>
> - If domU is allowed to grant map memory from dom0, there is not currently a 
> way to forcibly take it back (so I don't think you're quite correct in 2-1 
> above... but I may have missed something). Hence the domU can hold dom0's 
> memory to ransom. (In the network case this was avoided by using grant table 
> v2 'copy-only' grants).
> - If you end up having to grant buffers which do not originate in dom0 (i.e. 
> they were grant mapped from another domU) then this creates similar problems 
> with one domU holding another domU's memory to ransom, even when using 
> copy-only grants. I don’t think this would be an issue in your use-case.
> - Currently the default grant table size is 32 pages and it may not take that 
> many guests using a protocol where dom0 grants memory to domU to exhaust 
> dom0's grant table (depending on how many grants-per-domU the protocol 
> allows). If you're intending to grant large buffers then you may need quite a 
> few grants (since they are per-4k-chunk) to do this, so you might run into 
> this limit.

To follow up on what Paul said, google for XenServer Receive Side Copy. 
The issue is that it looks very attractive (from an offloading things
out of dom0 perspective), and does work at small scale, but the failure
cases are far more tricky than we imagined, and you run dom0 out of
grants very quickly, which puts a hard upper bound on scalability.  It
is high on the list of "worst mistakes we put into production".

It is not safe at all for dom0 to grant frames to domU without dom0
having a mechanism to revoke the grant.  (There was work looking into
this in the past, but it suffered from a lack of free time.)

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel