[Xen-devel] [PATCH, 3.2] fix race between scheduler and CPUs being offlined

["Jan Beulich" <jbeulich@xxxxxxxxxx>]

Since the credit scheduler depends on cpu_core_map and cpu_sibling_map
to be populated for all CPUs marked online in cpu_online_map (otherwise
csched_cpu_pick() can get into an endless loop due to nxt_idlers being
empty and hence no bit being cleared from cpus), sibling info must be
cleared *after* removing a CPU from cpu_online_map.

But that is only reducing the original race window - since the clearing
of the CPU maps happens on the dying CPU while the scheduler runs on an
active one (generally CPU0), the scheduler must also be enabled to deal
with the potential of finding empty nxt_idlers. While this change alone
would suffice to fix the race, clearing the maps in proper order still
seems like a reasonable thing to do.

Note that this is *not* applicable to 3.3 or -unstable, since there
scheduling doesn't happen anymore while CPUs are being brought down.

Signed-off-by: Jan Beulich <jbeulich@xxxxxxxxxx>

--- 2008-07-30.orig/xen/arch/x86/smpboot.c      2008-01-16 14:24:34.000000000 
+0100
+++ 2008-07-30/xen/arch/x86/smpboot.c   2008-12-15 15:38:37.000000000 +0100
@@ -1217,12 +1217,12 @@ int __cpu_disable(void)
 
        time_suspend();
 
-       remove_siblinginfo(cpu);
-
        cpu_clear(cpu, map);
        fixup_irqs(map);
        /* It's now safe to remove this processor from the online map */
        cpu_clear(cpu, cpu_online_map);
+       remove_siblinginfo(cpu);
+
        return 0;
 }
 
--- 2008-07-30.orig/xen/common/sched_credit.c   2008-01-16 14:24:35.000000000 
+0100
+++ 2008-07-30/xen/common/sched_credit.c        2008-12-15 16:31:59.000000000 
+0100
@@ -474,6 +474,16 @@ csched_cpu_pick(struct vcpu *vc)
             cpu = nxt;
             cpu_clear(cpu, cpus);
         }
+        else if ( unlikely(cpus_empty(nxt_idlers)) )
+        {
+            /*
+             * This can happen when CPUs are being brought down for S3
+             * or S5: cpu_{core,sibling}_map may have got cleared by
+             * the time we get here, while we may have found the CPU
+             * still set in cpu_online_map earlier.
+             */
+            cpu_clear(nxt, cpus);
+        }
         else
         {
             cpus_andnot(cpus, cpus, nxt_idlers);




_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel