Re: [Xen-devel] [RFC Design Doc] Intel L2 Cache Allocation Technology (L2 CAT) Feature enabling

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 13/05/16 09:55, Jan Beulich wrote:
>>>> On 13.05.16 at 09:43, <andrew.cooper3@xxxxxxxxxx> wrote:
>> On 13/05/2016 07:48, Jan Beulich wrote:
>>>>>> On 13.05.16 at 08:26, <he.chen@xxxxxxxxxxxxxxx> wrote:
>>>> On Thu, May 12, 2016 at 04:05:36AM -0600, Jan Beulich wrote:
>>>>>>>> On 12.05.16 at 11:40, <he.chen@xxxxxxxxxxxxxxx> wrote:
>>>>>> We plan to bring new PQoS feature called Intel L2 Cache Allocation
>>>>>> Technology (L2 CAT) to Xen.
>>>>>>
>>>>>> L2 CAT is supported on Atom codename Goldmont and beyond. “Big-core”
>>>>>> Xeon does not support L2 CAT in current generations.
>>>>> Looks mostly like a direct (and hence reasonable) extension of what
>>>>> we have for L3 right now. One immediate question I have is whether
>>>>> tying this to per-socket information is a good idea. As soon as Xeon-s
>>>>> would also gain such functionality, things would (aiui) need to become
>>>>> per-core (as L2 is per core there iirc).
>>>>>
>>>> L2 Cache capability keeps the same through all cores in a socket, so we
>>>> make it per-socket to balance code complexity and accessibility.
>>>>
>>>> I am not a expert in scheduler, do you mean in some cases, a domain
>>>> would apply different L2 cache access pattern when it is scheduled on
>>>> different cores even though the cores are in the same socket?
>>> No, I mean different domains may be running on different cores,
>>> and hence different policies may be needed to accommodate them
>>> all.
>> From the description, it sounds like L2 behaves almost exactly like L3. 
>> There is one set of capacity bitmaps which apply to all L2 caches in the
>> socket, and the specific capacity bitmap in effect is specified by
>> PSR_ASSOC CLOS, which is context switched with the vcpu.
> Well, I suppose the description is implying per-socket L2s. For per-
> core L2s I'd expect the MSRs to also become per-core.
>
> But anyway, L2 or L3 - I can't see how this context switching would
> DTRT when there are vCPU-s of different domains on the same
> socket (or core, if L2s and MSRs were per-core): The one getting
> scheduled later onto a socket (core) would simply overwrite what
> got written for the one which had been scheduled earlier.

PSR_ASSOC is a per-thread MSR which selects the CLOS to use.  CLOS is
currently managed per-domain in Xen, and context switched with vcpu.

Xen programs different capacity bitmaps into IA32_L2_QOS_MASK_0 ...
IA32_L2_QOS_MASK_n, and the CLOS selects which bitmap is enforced.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel