Re: [Xen-devel] [PATCH v9 1/6] x86: detect and initialize Cache QoS Monitoring feature

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 18/03/14 10:46, Xu, Dongxiao wrote:
>> -----Original Message-----
>> From: Jan Beulich [mailto:JBeulich@xxxxxxxx]
>> Sent: Tuesday, March 18, 2014 6:28 PM
>> To: Xu, Dongxiao
>> Cc: andrew.cooper3@xxxxxxxxxx; Ian.Campbell@xxxxxxxxxx;
>> Ian.Jackson@xxxxxxxxxxxxx; stefano.stabellini@xxxxxxxxxxxxx;
>> xen-devel@xxxxxxxxxxxxx; konrad.wilk@xxxxxxxxxx; dgdegra@xxxxxxxxxxxxx;
>> keir@xxxxxxx
>> Subject: RE: [PATCH v9 1/6] x86: detect and initialize Cache QoS Monitoring
>> feature
>>
>>>>> On 18.03.14 at 11:15, "Xu, Dongxiao" <dongxiao.xu@xxxxxxxxx> wrote:
>>>>  -----Original Message-----
>>>> From: Jan Beulich [mailto:JBeulich@xxxxxxxx]
>>>> Sent: Tuesday, March 18, 2014 6:09 PM
>>>> To: Xu, Dongxiao
>>>> Cc: andrew.cooper3@xxxxxxxxxx; Ian.Campbell@xxxxxxxxxx;
>>>> Ian.Jackson@xxxxxxxxxxxxx; stefano.stabellini@xxxxxxxxxxxxx;
>>>> xen-devel@xxxxxxxxxxxxx; konrad.wilk@xxxxxxxxxx; dgdegra@xxxxxxxxxxxxx;
>>>> keir@xxxxxxx
>>>> Subject: RE: [PATCH v9 1/6] x86: detect and initialize Cache QoS Monitoring
>>>> feature
>>>>
>>>>>>> On 18.03.14 at 11:02, "Xu, Dongxiao" <dongxiao.xu@xxxxxxxxx> wrote:
>>>>> Previously due to the large amount of data, we use the method to let Xen
>>>>> share the CQM pages to Dom0 in read only way to avoid data copy.
>>>>> However since the data amount is reduced a lot with above approach, do
>> you
>>>>> think whether we still need to use this share way? Or like most of the
>>> other
>>>>> sysctl/domctl (e.g., xl list) command, which uses copy_to_guest() to pass
>>> the
>>>>> data?
>>>> Iirc we're talking about a square table with each dimension being the
>>>> socket count. Considering an 8-node 4-socket system, that would
>>>> still be 1024 entries, i.e. exceeding a page in size. Hence I would
>>>> think that the read-only sharing approach might still be better. Of
>>>> course, if the data amount was smaller (and by so much that even
>>>> on huge systems it's no more than a page), that would be different.
>>> Okay.
>>>
>>> By using dynamic memory allocation and data sharing mechanism, we may
>> need
>>> two hypercalls when Dom0 tool stack is querying CQM related info.
>>>  - 1st hypercall is to let Xen allocate the memory and put CQM data there.
>>>  - 2nd hypercall is to indicate Dom0 tool stack already digested the CQM 
>>> data
>>> and Xen can free the memory.
>> Why would that memory ever need de-allocating?
>>
>> Anyway, could you clarify again what amount of data we're
>> talking about, without me having to dig out the old patch series?
> Originally we statically allocate memory in initialization time, and the size 
> is "rmid_max * socket_max", which may be a very big value. As the propose in 
> today's first mail, we can use the dynamic memory allocation as "rmid_inuse * 
> socket_inuse" for CQM related pages when user is really issuing query 
> operations, this can save the allocated memory size, because at that point, 
> we know the exact socket number in the system and exact the in use RMID, and 
> no need to predict them as maximum values.
>
> Back to the above question why memory need de-allocating:
> Since the rmid_inuse and socket_inuse may be changing from time to time, the 
> allocating memory size will be different. Therefore we need to allocate them 
> when user issues the hypercall, and then free them after the data digest is 
> finished.
>
> Not sure whether I stated the problem clearly for you.
>
> Thanks,
> Dongxiao

There is a sensible upper bound for rmid_max, in the init function. 
There should be a set of pointers (one per socket), allocated on use,
which can contain rmid_max data.

Once allocated, they are large enough for any eventuality, and don't
need de/reallocating. This way, the amount of memory used is
sockets_inuse * rmid_max.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel