Re: [Xen-devel] [PATCH RFC V2] xen/netback: Count ring slots properly when larger MTU sizes are used

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Tue, 2012-12-18 at 19:43 +0000, Matt Wilson wrote:
> On Tue, Dec 18, 2012 at 10:02:48AM +0000, Ian Campbell wrote:
> > On Mon, 2012-12-17 at 20:09 +0000, Matt Wilson wrote:
> > > On Mon, Dec 17, 2012 at 11:26:38AM +0000, Ian Campbell wrote:
> [...]
> > > > Do you mean the ring or the actual buffers?
> > > 
> > > Sorry, the actual buffers.
> > > 
> > > > The current code tries to coalesce multiple small frags/heads because it
> > > > is usually trivial but doesn't try too hard with multiple larger frags,
> > > > since they take up most of a page by themselves anyway. I suppose this
> > > > does waste a bit of buffer space and therefore could take more ring
> > > > slots, but it's not clear to me how much this matters in practice (it
> > > > might be tricky to measure this with any realistic workload).
> > > 
> > > In the case where we're consistently handling large heads (like when
> > > using a MTU value of 9000 for streaming traffic), we're wasting 1/3 of
> > > the available buffers.
> > 
> > Sorry if I missed this earlier in the thread, but how do we end up
> > wasting so much?
> 
> I see SKBs with:
>   skb_headlen(skb) == 8157
>   offset_in_page(skb->data) == 64
> 
> when handling long streaming ingress flows from ixgbe with MTU (on the
> NIC and both sides of the VIF) set to 9000. When all the SKBs making
> up the flow have the above property, xen-netback uses 3 pages instead
> of two. The first buffer gets 4032 bytes copied into it. The next
> buffer gets 4096 bytes copied into it. The final buffer gets 29 bytes
> copied into it. See this post in the archives for a more detailed
> walk through netbk_gop_frag_copy():
>   http://lists.xen.org/archives/html/xen-devel/2012-12/msg00274.html

Thanks. This certainly seems wrong for the head bit.

> What's the down side to making start_new_rx_buffer() always try to
> fill each buffer?

As we discussed earlier in the thread it doubles the number of copy ops
per frag under some circumstances, my gut is that this isn't going to
hurt but that's just my gut.

It seems obviously right that the linear part of the SKB should always
fill entire buffers though. Perhaps the answer is to differentiate
between the skb->data and the frags?

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel