[Xen-changelog] [xen-4.0-testing] sched_credit: Raise bar for inter-socket migrations on mostly-idle systems

["Xen patchbot-4.0-testing" <patchbot-4.0-testing@xxxxxxxxxxxxxxxxxxx>]

# HG changeset patch
# User Keir Fraser <keir.fraser@xxxxxxxxxx>
# Date 1285010305 -3600
# Node ID 67af28519aed5a68758f34cc37216c008faca9cb
# Parent  59917443fc5090cd4833a2381e96c96001007b21
sched_credit: Raise bar for inter-socket migrations on mostly-idle systems

The credit scheduler ties to keep work balanced, even on a mostly idle
system.  Unfortunately, if you have one VM burning cpu and another VM
idle, the effect is that the busy VM will flip back and forth between
sockets.

This patch addresses this, by only migrating to a different socket if
the number of idle processors is twice that of the socket the vcpu is
currently on.

This will only affect mostly-idle systems; as the system becomes more
busy, other load-balancing code will come into effect.

Signed-off-by: George Dunlap <george.dunlap@xxxxxxxxxxxxx>
xen-unstable changeset:   22180:0bc640853cfd
xen-unstable date:        Mon Sep 20 18:49:15 2010 +0100
---
 xen/common/sched_credit.c |   14 ++++++++++++--
 1 files changed, 12 insertions(+), 2 deletions(-)

diff -r 59917443fc50 -r 67af28519aed xen/common/sched_credit.c
--- a/xen/common/sched_credit.c Mon Sep 20 20:16:41 2010 +0100
+++ b/xen/common/sched_credit.c Mon Sep 20 20:18:25 2010 +0100
@@ -420,26 +420,36 @@ _csched_cpu_pick(struct vcpu *vc, bool_t
         cpumask_t cpu_idlers;
         cpumask_t nxt_idlers;
         int nxt, weight_cpu, weight_nxt;
+        int migrate_factor;
 
         nxt = cycle_cpu(cpu, cpus);
 
         if ( cpu_isset(cpu, per_cpu(cpu_core_map, nxt)) )
         {
+            /* We're on the same socket, so check the busy-ness of threads.
+             * Migrate if # of idlers is less at all */
             ASSERT( cpu_isset(nxt, per_cpu(cpu_core_map, cpu)) );
+            migrate_factor = 1;
             cpus_and(cpu_idlers, idlers, per_cpu(cpu_sibling_map, cpu));
             cpus_and(nxt_idlers, idlers, per_cpu(cpu_sibling_map, nxt));
         }
         else
         {
+            /* We're on different sockets, so check the busy-ness of cores.
+             * Migrate only if the other core is twice as idle */
             ASSERT( !cpu_isset(nxt, per_cpu(cpu_core_map, cpu)) );
+            migrate_factor = 2;
             cpus_and(cpu_idlers, idlers, per_cpu(cpu_core_map, cpu));
             cpus_and(nxt_idlers, idlers, per_cpu(cpu_core_map, nxt));
         }
 
         weight_cpu = cpus_weight(cpu_idlers);
         weight_nxt = cpus_weight(nxt_idlers);
-        if ( ( (weight_cpu < weight_nxt) ^ sched_smt_power_savings )
-                && (weight_cpu != weight_nxt) )
+        /* smt_power_savings: consolidate work rather than spreading it */
+        if ( ( sched_smt_power_savings
+               && (weight_cpu > weight_nxt) )
+             || ( !sched_smt_power_savings
+                  && (weight_cpu * migrate_factor < weight_nxt) ) )
         {
             cpu = cycle_cpu(CSCHED_PCPU(nxt)->idle_bias, nxt_idlers);
             if ( commit )

_______________________________________________
Xen-changelog mailing list
Xen-changelog@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-changelog