[Xen-changelog] [xen-unstable] sched_credit: Raise bar for inter-socket migrations on mostly-idle systems

[Xen patchbot-unstable <patchbot-unstable@xxxxxxxxxxxxxxxxxxx>]

# HG changeset patch
# User Keir Fraser <keir.fraser@xxxxxxxxxx>
# Date 1285004955 -3600
# Node ID 0bc640853cfd71b2fc6a68e95c45df2fd2d3a2f3
# Parent  b59f87f56b1ecf2c560efc41434cf79e94b1e59b
sched_credit: Raise bar for inter-socket migrations on mostly-idle systems

The credit scheduler ties to keep work balanced, even on a mostly idle
system.  Unfortunately, if you have one VM burning cpu and another VM
idle, the effect is that the busy VM will flip back and forth between
sockets.

This patch addresses this, by only migrating to a different socket if
the number of idle processors is twice that of the socket the vcpu is
currently on.

This will only affect mostly-idle systems; as the system becomes more
busy, other load-balancing code will come into effect.

Signed-off-by: George Dunlap <george.dunlap@xxxxxxxxxxxxx>
---
 xen/common/sched_credit.c |   14 ++++++++++++--
 1 files changed, 12 insertions(+), 2 deletions(-)

diff -r b59f87f56b1e -r 0bc640853cfd xen/common/sched_credit.c
--- a/xen/common/sched_credit.c Mon Sep 20 17:54:25 2010 +0100
+++ b/xen/common/sched_credit.c Mon Sep 20 18:49:15 2010 +0100
@@ -499,26 +499,36 @@ _csched_cpu_pick(const struct scheduler 
         cpumask_t cpu_idlers;
         cpumask_t nxt_idlers;
         int nxt, weight_cpu, weight_nxt;
+        int migrate_factor;
 
         nxt = cycle_cpu(cpu, cpus);
 
         if ( cpu_isset(cpu, per_cpu(cpu_core_map, nxt)) )
         {
+            /* We're on the same socket, so check the busy-ness of threads.
+             * Migrate if # of idlers is less at all */
             ASSERT( cpu_isset(nxt, per_cpu(cpu_core_map, cpu)) );
+            migrate_factor = 1;
             cpus_and(cpu_idlers, idlers, per_cpu(cpu_sibling_map, cpu));
             cpus_and(nxt_idlers, idlers, per_cpu(cpu_sibling_map, nxt));
         }
         else
         {
+            /* We're on different sockets, so check the busy-ness of cores.
+             * Migrate only if the other core is twice as idle */
             ASSERT( !cpu_isset(nxt, per_cpu(cpu_core_map, cpu)) );
+            migrate_factor = 2;
             cpus_and(cpu_idlers, idlers, per_cpu(cpu_core_map, cpu));
             cpus_and(nxt_idlers, idlers, per_cpu(cpu_core_map, nxt));
         }
 
         weight_cpu = cpus_weight(cpu_idlers);
         weight_nxt = cpus_weight(nxt_idlers);
-        if ( ( (weight_cpu < weight_nxt) ^ sched_smt_power_savings )
-                && (weight_cpu != weight_nxt) )
+        /* smt_power_savings: consolidate work rather than spreading it */
+        if ( ( sched_smt_power_savings
+               && (weight_cpu > weight_nxt) )
+             || ( !sched_smt_power_savings
+                  && (weight_cpu * migrate_factor < weight_nxt) ) )
         {
             cpu = cycle_cpu(CSCHED_PCPU(nxt)->idle_bias, nxt_idlers);
             if ( commit )

_______________________________________________
Xen-changelog mailing list
Xen-changelog@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-changelog