Re: [Xen-devel] Poor network performance between DomU with multiqueue support

[Zoltan Kiss <zoltan.kiss@xxxxxxxxxx>]

On 05/12/14 12:42, Wei Liu wrote:
> On Fri, Dec 05, 2014 at 01:17:16AM +0000, Zhangleiqiang (Trump) wrote:
> [...]
> > > I think that's expected, because guest RX data path still uses grant_copy while
> > > guest TX uses grant_map to do zero-copy transmit.
> > As far as I know, there are three main grant-related operations used in split 
> > device model: grant mapping, grant transfer and grant copy.
> > Grant transfer has not used now, and grant mapping and grant transfer both involve 
> > "TLB" refresh work for hypervisor, am I right?  Or only grant transfer has this 
> > overhead?
> Transfer is not used so I can't tell. Grant unmap causes TLB flush.
>
> I saw in an email the other day XenServer folks has some planned
> improvement to avoid TLB flush in Xen to upstream in 4.6 window. I can't
> speak for sure it will get upstreamed as I don't work on that.
>
> > Does grant copy surely has more overhead than grant mapping?
> At the very least the zero-copy TX path is faster than previous copying
> path.
>
> But speaking of the micro operation I'm not sure.
>
> There was once persistent map prototype netback / netfront that
> establishes a memory pool between FE and BE then use memcpy to copy
> data. Unfortunately that prototype was not done right so the result was
> not good.
>
> > >From the code, I see that in TX, netback will do gnttab_batch_copy as well as 
> > gnttab_map_refs:
> >
> > <code> //netback.c:xenvif_tx_action
> >         xenvif_tx_build_gops(queue, budget, &nr_cops, &nr_mops);
> >
> >         if (nr_cops == 0)
> >                 return 0;
> >
> >         gnttab_batch_copy(queue->tx_copy_ops, nr_cops);
> >         if (nr_mops != 0) {
> >                 ret = gnttab_map_refs(queue->tx_map_ops,
> >                                       NULL,
> >                                       queue->pages_to_map,
> >                                       nr_mops);
> >                 BUG_ON(ret);
> >         }
> > </code>
> The copy is for the packet header. Mapping is for packet data.
>
> We need to copy header from guest so that it doesn't change under
> netback's feet.
It is also important because if the above mentioned "TLB flush 
avoidance" patch goes in to Xen, it will be important to grant copy the 
header rather than grant map plus memcpy. The latter is the old way, it 
touches the page so you can't avoid TLB flush.
> Wei.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel