[Xen-devel] [PATCH 2/3] xen: Have schedulers revise initial placement

[George Dunlap <george.dunlap@xxxxxxxxxx>]

The generic domain creation logic in
xen/common/domctl.c:default_vcpu0_location() attempts to try to do
initial placement load-balancing by placing vcpu 0 on the least-busy
non-primary hyperthread available.  Unfortunately, the logic can end
up picking a pcpu that's not in the online mask.  When this is passed
to a scheduler such which assumes that the initial assignment is
valid, it causes a null pointer dereference looking up the runqueue.

Furthermore, this initial placement doesn't take into account hard or
soft affinity, or any scheduler-specific knowledge (such as historic
runqueue load, as in credit2).

To solve this, when inserting a vcpu, always call the per-scheduler
"pick" function to revise the initial placement.  This will
automatically take all knowledge the scheduler has into account.

Signed-off-by: George Dunlap <george.dunlap@xxxxxxxxxx>
---
CC: Dario Faggioli <dario.faggioli@xxxxxxxxxx>
CC: Anshul Makkar <anshul.makkar@xxxxxxxxxx>
CC: Meng Xu <mengxu@xxxxxxxxxxxxx>
CC: Jan Beulich <jbeulich@xxxxxxxx>
---
 xen/common/sched_credit.c  |  3 +++
 xen/common/sched_credit2.c | 12 ++++++++++--
 xen/common/sched_rt.c      |  3 +++
 3 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c
index ac22746..a3ef00a 100644
--- a/xen/common/sched_credit.c
+++ b/xen/common/sched_credit.c
@@ -998,6 +998,9 @@ csched_vcpu_insert(const struct scheduler *ops, struct vcpu 
*vc)
 
     BUG_ON( is_idle_vcpu(vc) );
 
+    /* This is safe because vc isn't yet being scheduled */
+    vc->processor = csched_cpu_pick(ops, vc);
+
     lock = vcpu_schedule_lock_irq(vc);
 
     if ( !__vcpu_on_runq(svc) && vcpu_runnable(vc) && !vc->is_running )
diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 3b9aa27..5a04985 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -1620,15 +1620,23 @@ csched2_vcpu_insert(const struct scheduler *ops, struct 
vcpu *vc)
 
     BUG_ON(is_idle_vcpu(vc));
 
+    /* Locks in cpu_pick expect irqs to be disabled */
+    local_irq_disable();
+    
+    /* This is safe because vc isn't yet being scheduled */
+    vc->processor = csched2_cpu_pick(ops, vc);
+
     /* Add vcpu to runqueue of initial processor */
-    lock = vcpu_schedule_lock_irq(vc);
+    lock = vcpu_schedule_lock(vc);
 
     runq_assign(ops, vc);
 
-    vcpu_schedule_unlock_irq(lock, vc);
+    vcpu_schedule_unlock(lock, vc);
 
     sdom->nr_vcpus++;
 
+    local_irq_enable();
+
     SCHED_STAT_CRANK(vcpu_insert);
 
     CSCHED2_VCPU_CHECK(vc);
diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index bd3a2a0..41c61a7 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -874,6 +874,9 @@ rt_vcpu_insert(const struct scheduler *ops, struct vcpu *vc)
 
     BUG_ON( is_idle_vcpu(vc) );
 
+    /* This is safe because vc isn't yet being scheduled */
+    vc->processor = rt_cpu_pick(ops, vc);
+
     lock = vcpu_schedule_lock_irq(vc);
 
     now = NOW();
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel