Re: [Xen-devel] [RFC 0/5] xen/arm: support big.little SoC

[Dario Faggioli <dario.faggioli@xxxxxxxxxx>]

On Mon, 2016-09-19 at 17:01 -0700, Stefano Stabellini wrote:
> On Tue, 20 Sep 2016, Dario Faggioli wrote:
> > And this would work even if/when there is only one cpupool, or in
> > general for domains that are in a pool that has both big and LITTLE
> > pcpus. Furthermore, big.LITTLE support and cpupools will be
> > orthogonal,
> > just like pinning and cpupools are orthogonal right now. I.e., once
> > we
> > will have what I described above, nothing prevents us from
> > implementing
> > per-vcpu cpupool membership, and either create the two (or more!)
> > big
> > and LITTLE pools, or from mixing things even more, for more complex
> > and
> > specific use cases. :-)
> 
> I think that everybody agrees that this is the best long term
> solution.
> 
Well, no, that wasn't obvious to me. If that's the case, it's already
something! :-)

> > 
> > Actually, with the cpupool solution, if you want a guest (or dom0)
> > to
> > actually have both big and LITTLE vcpus, you necessarily have to
> > implement per-vcpu (rather than per-domain, as it is now) cpupool
> > membership. I said myself it's not impossible, but certainly it's
> > some
> > work... with the scheduler solution you basically get that for
> > free!
> > 
> > So, basically, if we use cpupools for the basics of big.LITTLE
> > support,
> > there's no way out of it (apart from going implementing scheduling
> > support afterwords, but that looks backwards to me, especially when
> > thinking at it with the code in mind).
> 
> The question is: what is the best short-term solution we can ask Peng
> to
> implement that allows Xen to run on big.LITTLE systems today?
> Possibly
> getting us closer to the long term solution, or at least not farther
> from it?
> 
So, I still have to look closely at the patches in these series. But,
with Credit2 in mind, if one:

 - take advantage of the knowledge of what arch a pcpu belongs inside 
   the code that arrange the pcpus in runqueues, which means we'll end 
   up with big runqueues and LITTLE runqueues. I re-wrote that code, I
   can provide pointers and help, if necessary;
 - tweak the one or two instance of for_each_runqueue() [*] that there
   are in the code into a for_each_runqueue_of_same_class(), i.e.:

 if (is_big(this_cpu))
 {
   for_each_big_runqueue()
   {
      ..
   }
 }
 else
 {
   for_each_LITTLE_runqueue()
   {
     ..
   }
 } 

then big.LITTLE support in Credit2 would be done already, and all it
would be left is support for the syntax of new config switches in xl,
and a way of telling, from xl/libxl down to Xen, what arch a vcpu
belongs to, so that it can be associated with one runqueue of the
proper class.

Thinking to Credit1, we need to make sure thet, in load_balance() and
runq_steal(), a LITTLE cpu *only* ever try to steal work from another
LITTLE cpu, and __never__ from a big cpu (and vice versa). And also
that when a vcpu wakes up, and what it has in its v->processor is a
LITTLE pcpu, that only LITTLE processors are considered for being
tickled (I'm less certain of this last part, but it should be more or
less like this).

Then, of course the the same glue and vcpu classification code.

However, in Credit1, it's possible that a trick like that would affect
the accounting and credit algorithm, and hence provide unfair, or in
general, unexpected results. Credit2 should, OTOH, be a lot mere
resilient, wrt that.

> > > The whole process would be more explicit and obvious if we used
> > > cpupools. It would be easier for users to know what it is going
> > > on --
> > > they just need to issue an `xl cpupool-list' command and they
> > > would
> > > see
> > > two clearly named pools (something like big-pool and LITTLE-
> > > pool). 
> > > 
> > Well, I guess that, as part of big.LITTLE support, there will be a
> > way
> > to tell what pcpus are big and which are LITTLE anyway, probably
> > both
> > from `xl info' and from `xl cpupool-list -c' (and most likely in
> > other
> > ways too).
> 
> Sure, but it needs to be very clear. We cannot ask people to spot
> architecture specific flags among the output of `xl info' to be able
> to
> appropriately start a guest. 
>
As mentioned in previous mail, and as drafted when replying to Peng,
the only think that the user should know is how many big and how many
LITTLE vcpus she wants (and, potentially, which one would be each). :-)

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel