Re: [Xen-devel] [PATCH v9 1/6] x86: detect and initialize Cache QoS Monitoring feature

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 18.03.14 at 15:33, "Xu, Dongxiao" <dongxiao.xu@xxxxxxxxx> wrote:
>>  -----Original Message-----
>> From: Jan Beulich [mailto:JBeulich@xxxxxxxx]
>> Sent: Tuesday, March 18, 2014 6:58 PM
>> To: Xu, Dongxiao
>> Cc: andrew.cooper3@xxxxxxxxxx; Ian.Campbell@xxxxxxxxxx;
>> Ian.Jackson@xxxxxxxxxxxxx; stefano.stabellini@xxxxxxxxxxxxx;
>> xen-devel@xxxxxxxxxxxxx; konrad.wilk@xxxxxxxxxx; dgdegra@xxxxxxxxxxxxx;
>> keir@xxxxxxx 
>> Subject: RE: [PATCH v9 1/6] x86: detect and initialize Cache QoS Monitoring
>> feature
>> 
>> >>> On 18.03.14 at 11:46, "Xu, Dongxiao" <dongxiao.xu@xxxxxxxxx> wrote:
>> >> From: Jan Beulich [mailto:JBeulich@xxxxxxxx]
>> >> >>> On 18.03.14 at 11:15, "Xu, Dongxiao" <dongxiao.xu@xxxxxxxxx> wrote:
>> >> > By using dynamic memory allocation and data sharing mechanism, we may
>> need
>> >> > two hypercalls when Dom0 tool stack is querying CQM related info.
>> >> >  - 1st hypercall is to let Xen allocate the memory and put CQM data 
>> >> > there.
>> >> >  - 2nd hypercall is to indicate Dom0 tool stack already digested the CQM
>> data
>> >> > and Xen can free the memory.
>> >>
>> >> Why would that memory ever need de-allocating?
>> >>
>> >> Anyway, could you clarify again what amount of data we're
>> >> talking about, without me having to dig out the old patch series?
>> >
>> > Originally we statically allocate memory in initialization time, and the
>> > size is "rmid_max * socket_max", which may be a very big value. As the
>> > propose in today's first mail, we can use the dynamic memory allocation as
>> > "rmid_inuse * socket_inuse" for CQM related pages when user is really 
> issuing
>> > query operations, this can save the allocated memory size, because at that
>> > point, we know the exact socket number in the system and exact the in use
>> > RMID, and no need to predict them as maximum values.
>> 
>> That wasn't my question. The question (tailored to the above
>> description of yours) is - what are reasonable values of rmid_inuse
>> and socket_inuse on a huge system?
> 
> According to SDM figure 17-20, the possible maximum of rmid should be about 
> 2^10=1024. And for the rmid_inuse, it is somehow depending on the launched VM 
> number, if administrator is interested in the LLC usage.
> For socket number, I think 2/4 sockets are common in the market.

That's (a) not considering huge systems and (b) not even considering
multi-node ones.

>> > Back to the above question why memory need de-allocating:
>> > Since the rmid_inuse and socket_inuse may be changing from time to time, 
> the
>> > allocating memory size will be different. Therefore we need to allocate 
> them
>> > when user issues the hypercall, and then free them after the data digest is
>> > finished.
>> 
>> But once we have topology information in hand, deriving the
>> maximum possible socket number from MADT data ought to be
>> possible. Hence the allocation could be done with the maximum
>> size. An alternative (allowing the allocated size to be limited to
>> in-use resources) might be to do the de-allocation during CPU
>> hotplug rather than upon explicit Dom0 request.
> 
> System may be boot with no CPU plugged in the socket, so can we get the full 
> map in this case? I think it may be difficult...

Afaik the full topology has to be available, with hot-pluggable but
not present CPUs marked accordingly. We use this information to
e.g. set nr_cpu_ids.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel