Re: [Xen-devel] (Possible) Inaccurate accounting in Xen's bandwidth control

[Keir Fraser <keir.xen@xxxxxxxxx>]

On 12/03/2011 07:59, "Luwei Cheng" <chengluwei@xxxxxxxxx> wrote:

> Hi,
> 
> These days I use netperf to evaluate Xen's network bandwidth control.
> I found that somehow the bandwidth is always inaccurate (slightly less than
> the promised bandwidth) 
> 
> After reading the source code of Linux-2.6.3** (domain 0), I am a bit curious
> about the following algorithm.
> file: /linux-main-dir/drivers/xen/netback/netback.c
> =============================================================
> static void tx_add_credit(netif_t *netif)
> {
>         unsigned long max_burst, max_credit;
> 
>         /*
>          * Allow a burst big enough to transmit a jumbo packet of up to 128kB.
>          * Otherwise the interface can seize up due to insufficient credit.
>          */
>         max_burst = RING_GET_REQUEST(&netif->tx, netif->tx.req_cons)->size;
>         max_burst = min(max_burst, 131072UL);
>         max_burst = max(max_burst, netif->credit_bytes);

Making the above line be max(max_burst, 2*netif->credit_bytes) would allow
the interface to 'carry over' the previous period's credit into the
following period. Hence allow the interface to get closer to its permitted
bandwidth limit in the long term average, at the cost of allowing it to
burst at a somewhat higher bandwidth over sub-second timescales.

Basically it's a tradeoff between giving the interface freedom over when it
consumes its credits, versus keeping traffic flow smooth and unbursty.

 -- Keir

>         /* Take care that adding a new chunk of credit doesn't wrap to zero.
> */
>         max_credit = netif->remaining_credit + netif->credit_bytes;
>         if (max_credit < netif->remaining_credit)
>                 max_credit = ULONG_MAX; /* wrapped: clamp to ULONG_MAX */
> 
>         netif->remaining_credit = min(max_credit, max_burst);
> }
> =============================================================
> 
> 
> Setting: rate=512Kb/s@30ms, Then,
> -----------------------
> "netif->credit_bytes" will be 1,920
> "netif->credit_usecs" will be 30,000
> =============================================================
> Suppose that at some moment:
> -----------------------
> "request size" is 1514
> "netif->remaining_credit" is 406
> -----------------------
> Since there's not enough credits for transmitting, the netif
> will be delayed for 30,000 usecs to refill credits (using timer).
> =============================================================
> Question:
> After the timer wakes up, should the netif get (406+1920) credits, or (1920)
> credits?  
> =============================================================
> In my mind, I think it should be fair to let the netif get (406+1920) credits.
> However, the above algorithm will eventually give only (1920) credits.
> 
> Just wonder whether there's some undocumented consideration to control in this
> way
> (only give 1920 credits)?
> 
> Kindly please correct me if my analysis is incorrect.
> 
> Thanks for your attention.
> 
> Best Regards,
> Luwei Cheng
> ----
> Department of Computer Science
> The University of Hong Kong
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel