[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH] Yield to VCPU hcall, spinlock yielding





habanero@xxxxxxxxxxxxxxxxxxxxxxx wrote on 06/08/2005 03:11:06 PM:

> > In our original posting, we proposed that the Linux interrupt handler
> > for preemption notifications would create (or unblock) a
> > high-priority kernel thread which would then yield back to the
> > hypervisor.  To Linux on other CPUs, the de-scheduled CPU would
> > appear to be busy running the high-priority thread, and all real work
> > that that CPU had been doing would be eligible for stealing.
>
> IMO, I don't think this alone is enough to encourage task migration.  
> The primary motivator to steal is a 25% or more load imbalance, and one
> extra fake kernel thread will probably not be enough to trigger this.


The kernel thread is needed at the very least to ensure that all user programs on the de-scheduled CPU are available for migration.  In an important case, a program on the de-scheduled CPU holds a futex, and another CPU goes idle because its program blocks on the futex.  We'd want the idle CPU to pick up the futex holder, and I'm assuming (with very little actual knowledge) that the Linux scheduler would make that happen.

>
> To solve this and other issues, I believe we need an extra modifier to
> the Linux kernel cpus' load value, which Xen could modify to hint the
> kernel what cpus' relative processing power is.  The Linux kernel
> scheduler's per cpu load values would be something like (max_cpu_power
> / cpu_power * nr_running).  Xen could update cpu_power for a number of
> situations, a "long" preemption, a much faster alternative to a vcpu
> hot-unplug (don't unplug, just set cpu_power to 0), and to normalize
> load values for vcpus which have different time-slice lengths on the
> physical cpus.  
>
> I would hope something like this could also be used without Xen on Linux
> so it has wider appeal.  One thing that comes to mind is normalizing
> cpus' load when some cpus may be "speed stepped" down for power
> management or heat issues.
>
> -Andrew

I'd view your "cpu_power" proposal as orthogonal to (or perhaps complementary to) our ideas on preemption notification.  It's aimed more at load-balancing and fair scheduling than specifically at the problems that arise with the preemption of lock holders.  On the apparent CPU speed issue, does Linux account in any way for different interrupt loads on different processors?  Is a program just out of luck if it happens to get scheduled on a processor with heavy interrupt traffic, or will Linux notice that it's not making the same progress as its peers and shuffle things around?  It seems that your cpu_power proposal might have something to contribute here.

- Bryan

- Bryan
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.