Re: [Xen-devel] [libvirt] [PATCH 2/2] libxl: support vif outgoing bandwidth QoS

[Jim Fehlig <jfehlig@xxxxxxxx>]

On 01/07/2016 07:48 AM, Michal Privoznik wrote:
> On 29.12.2015 02:09, Jim Fehlig wrote:
>> The libxl_device_nic structure supports specifying an outgoing rate
>> limit based on a time interval and bytes allowed per interval. In xl
>> config a rate limit is specified as "<RATE>/s@<INTERVAL>". INTERVAL
>> is optional and defaults to 50ms.
>>
>> libvirt expresses outgoing limits by average (required), peak, burst,
>> and floor attributes in units of KB/s. This patch supports the outgoing
>> bandwidth limit by converting the average KB/s to bytes per interval
>> based on the same default interval (50ms) used by xl.
>>
>> Signed-off-by: Jim Fehlig <jfehlig@xxxxxxxx>
>> ---
>>  src/libxl/libxl_conf.c | 39 +++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 39 insertions(+)
>>
>> diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c
>> index 23c74e7..6320421 100644
>> --- a/src/libxl/libxl_conf.c
>> +++ b/src/libxl/libxl_conf.c
>> @@ -1093,6 +1093,7 @@ libxlMakeNic(virDomainDefPtr def,
>>  {
>>      bool ioemu_nic = def->os.type == VIR_DOMAIN_OSTYPE_HVM;
>>      virDomainNetType actual_type = virDomainNetGetActualType(l_nic);
>> +    virNetDevBandwidthPtr actual_bw;
>>  
>>      /* TODO: Where is mtu stored?
>>       *
>> @@ -1206,6 +1207,44 @@ libxlMakeNic(virDomainDefPtr def,
>>  #endif
>>      }
>>  
>> +    /*
>> +     * Set bandwidth.
>> +     * From $xen-sources/docs/misc/xl-network-configuration.markdown:
>> +     *
>> +     *
>> +     * Specifies the rate at which the outgoing traffic will be limited to.
>> +     * The default if this keyword is not specified is unlimited.
>> +     *
>> +     * The rate may be specified as "<RATE>/s" or optionally 
>> "<RATE>/s@<INTERVAL>".
>> +     *
>> +     * `RATE` is in bytes and can accept suffixes:
>> +     *     GB, MB, KB, B for bytes.
>> +     *     Gb, Mb, Kb, b for bits.
>> +     * `INTERVAL` is in microseconds and can accept suffixes: ms, us, s.
>> +     *     It determines the frequency at which the vif transmission credit
>> +     *     is replenished. The default is 50ms.
>> +
>> +     * Vif rate limiting is credit-based. It means that for "1MB/s@20ms",
>> +     * the available credit will be equivalent of the traffic you would have
>> +     * done at "1MB/s" during 20ms. This will results in a credit of 20,000
>> +     * bytes replenished every 20,000 us.
>> +     *
>> +     *
>> +     * libvirt doesn't support the notion of rate limiting over an interval.
>> +     * Similar to xl's behavior when interval is not specified, set a 
>> default
>> +     * interval of 50ms and calculate the number of bytes per interval based
>> +     * on the specified average bandwidth.
>> +     */
>> +    actual_bw = virDomainNetGetActualBandwidth(l_nic);
>> +    if (actual_bw && actual_bw->out && actual_bw->out->average) {
>> +        uint64_t bytes_per_sec = actual_bw->out->average * 1024;
>> +        uint64_t bytes_per_interval =
>> +            (((uint64_t) bytes_per_sec * 50000UL) / 1000000UL);
>> +
>> +        x_nic->rate_bytes_per_interval = bytes_per_interval;
>> +        x_nic->rate_interval_usecs =  50000UL;
>> +    }
>> +
> Interesting. I'd expect:
>
> x_nic->rate_bytes_per_interval = bytes_per_sec;
> x_nic->rate_interval_usecs = 1000*1000;

For the most part I mimicked the Xen code and wanted to stick with the default
interval of 50ms, which has been the default for a long time. It is even
mentioned in some old RHEL5 docs

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Virtualization/sect-Virtualization-Tips_and_tricks-Limit_network_bandwidth_for_a_Xen_guest.html

BTW, here is the Xen code that inspired this logic

http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libxl/libxlu_vif.c;h=0665e624dc178a6ca8058e04a7baacaf1475bd37;hb=HEAD#l131

rate_bytes_per_interval is set to (bytes/s * interval us)/1000000us

I guess we are saying the same thing, you're just setting interval to 1s (thus
rate_bytes_per_interval == bytes_per_sec) instead of the historical 50ms :-).

>
> I mean, if I understood the xl way of rate limiting correctly, one says
> how much bytes can be sent for how long. so for 1MB/s I'd expect to send
> 1024*1024 bytes each second.
>
> Or am I missing something?

Does the above explanation make sense? I might be missing something :-).  CC'd a
few Xen tools maintainer just in case.

Regards,
Jim


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel