Re: [Xen-devel] [RFC 0/5] xen/arm: support big.little SoC

[George Dunlap <George.Dunlap@xxxxxxxxxxxxx>]

On Tue, Sep 20, 2016 at 1:01 AM, Stefano Stabellini
<sstabellini@xxxxxxxxxx> wrote:

>> Actually, with the cpupool solution, if you want a guest (or dom0) to
>> actually have both big and LITTLE vcpus, you necessarily have to
>> implement per-vcpu (rather than per-domain, as it is now) cpupool
>> membership. I said myself it's not impossible, but certainly it's some
>> work... with the scheduler solution you basically get that for free!
>>
>> So, basically, if we use cpupools for the basics of big.LITTLE support,
>> there's no way out of it (apart from going implementing scheduling
>> support afterwords, but that looks backwards to me, especially when
>> thinking at it with the code in mind).
>
> The question is: what is the best short-term solution we can ask Peng to
> implement that allows Xen to run on big.LITTLE systems today? Possibly
> getting us closer to the long term solution, or at least not farther
> from it?

So remember that there's the *interface* we're providing to the user
(specifying vcpus as little or BIG) and the guest OS (how does the
guest OS know whether a specific vcpu is little or BIG), and there's
the *implementation* of that.

For comparison, xl provides a "guest numa" interface; but the Xen
schedulers actually don't know anything about NUMA -- they only have
the concept of "soft affinity".  xl uses soft affinity to implement
the NUMA characteristics we want from the scheduler.

It seems to me that the best combination of functionality / simplicity
would be to provide a way to specify, in xl, which vcpus should be big
and LITTLE (something similar to what Dario mentions below); and then
to implement that initially only with cpu pinning inside of xl.

Then at some point we can extend that to tagging each vcpu with a
"pcpu class", and teaching schedulers about those classes, and making
sure each vcpu runs only within its own class.  This effectively
amounts to another layer of hard pinning, but one which is a bit more
robust (i.e., won't be confused if the user tries to set the hard
affinity of a vcpu).

That said, if the goal is to get *something* up as quick as humanly
possible, implementing "xl cpupool-bigLITTLE-split" (or a
'cpupool_setup=biglittle' Xen command-line option) would probably do
the job; but only by limiting domains to having only big or LITTLE
vcpus.

> Sure, but it needs to be very clear. We cannot ask people to spot
> architecture specific flags among the output of `xl info' to be able to
> appropriately start a guest. Even what I suggested isn't great as `xl
> cpupool-list' isn't a common command to run.

Well fundamentally, given that a vcpu has to start and stay on the
same class of processors, the user will have to make some action to
deviate from the default.  I don't fundamentally see any difference
between saying "In order to use the LITTLE cpus you have to specify
the LITTLE cpupool" and "In order to use the LITTLE cpus you have to
specify some vcpus as LITTLE vcpus".

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel