[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Xen-devel] [PATCH 8/9] xen: sched: allow for choosing credit2 runqueues configuration at boot
On 10/01/2015 09:23 AM, Dario Faggioli wrote:
On Thu, 2015-10-01 at 07:48 +0200, Juergen Gross wrote:
On 09/29/2015 06:56 PM, Dario Faggioli wrote:
In fact, credit2 uses CPU topology to decide how to arrange
its internal runqueues. Before this change, only 'one runqueue
per socket' was allowed. However, experiments have shown that,
for instance, having one runqueue per physical core improves
performance, especially in case hyperthreading is available.
In general, it makes sense to allow users to pick one runqueue
arrangement at boot time, so that:
- more experiments can be easily performed to even better
assess and improve performance;
- one can select the best configuration for his specific
use case and/or hardware.
This patch enables the above.
Note that, for correctly arranging runqueues to be per-core,
just checking cpu_to_core() on the host CPUs is not enough.
In fact, cores (and hyperthreads) on different sockets, can
have the same core (and thread) IDs! We, therefore, need to
check whether the full topology of two CPUs matches, for
them to be put in the same runqueue.
Note also that the default (although not functional) for
credit2, since now, has been per-socket runqueue. This patch
leaves things that way, to avoid mixing policy and technical
changes.
I think you should think about a way to make this parameter a per
cpupool one instead a system global one.
Believe it or not, I though about this already, and yes, it is in my
plans to make this per-cpupool. However...
As this will require some
extra work regarding the tools interface I'd be absolutely fine with
adding this at a later time, but you should have that in mind when
setting this up now.
...yes, that was phase II in my mind as well.
So (sorry, but just to make sure I understand), since you said you're
fine with it coming later, are you also fine with this patch, or do you
think some adjustments are necessary, right here, right now, because of
that future plan?
No, I'm fine.
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -82,10 +82,6 @@
@@ -194,6 +190,41 @@ static int __read_mostly
opt_overload_balance_tolerance = -3;
integer_param("credit2_balance_over",
opt_overload_balance_tolerance);
/*
+ * Runqueue organization.
+ *
+ * The various cpus are to be assigned each one to a runqueue, and
we
+ * want that to happen basing on topology. At the moment, it is
possible
+ * to choose to arrange runqueues to be:
+ *
+ * - per-core: meaning that there will be one runqueue per each
physical
+ * core of the host. This will happen if the
opt_runqueue
+ * parameter is set to 'core';
+ *
+ * - per-socket: meaning that there will be one runqueue per each
physical
+ * socket (AKA package, which often, but not always,
also
+ * matches a NUMA node) of the host; This will
happen if
+ * the opt_runqueue parameter is set to 'socket';
Wouldn't it be a nice idea to add "per-numa-node" as well?
I think it is.
This would make a difference for systems with:
- multiple sockets per numa-node
- multiple numa-nodes per socket
Yep.
It might even be a good idea to be able to have only one runqueue in
small cpupools (again, this will apply only in case you have a per
cpupool setting instead a global one).
And I agree on this too.
TBH, I had considered these too, and I was thinking to make them happen
in phase II as well. However, they're simple enough to be implemented
now (as in, in v2 of this series), so I think I'll do that.
Thanks.
Juergen
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel
|
