Re: [Xen-devel] credit scheduler and HYPERVISOR_yield()

["George Dunlap" <gdunlap@xxxxxxxxxxxxx>]

On 10/14/07, Emmanuel Ackaouy <ackaouy@xxxxxxxxx> wrote:
> Doing the change that George proposed may help in your case
> but I suspect that, as I described in my previous post, it will cause
> problems for other workloads.
>
> I think it is reasonable for a yield() operation to yield to runnable
> VCPUs of equal or higher priority than the running VCPU. That
> is the behavior of the scheduler today. Maybe your problem can
> be addressed without changing the behavior of yield?

Part of the problem is that for the credit scheduler, the "priority"
is used a bit differently.  It changes, and it has no fundamental
relationship between more important work and less important work; it's
just a mechanism for implementing time allocations. (And a very clever
way, I might add.)

It's clear that "yield-I-really-mean-it" is useful for smp
synchronization issues (like yielding when waiting for a spinlock held
by scheduled-out vcpus, or waiting for a scheduled-out processor to
ACK an IPI).  But I can't really think of a situation where
"yield-to-other-cpus-that-haven't-used-all-their-credits-yet" is
particularly useful.  Can you think of an example?

Perhaps a better implementation of "yield-I-really-mean-it" would be:
* Reduce the priority only if there are no vcpus of the same priority
in the queue; and perhaps, only if there are no vcpus in the queue and
no work to steal.
* As soon as the vcpu in question is scheduled, raise its priority again.

That should avoid some of the problems you've pointed out with the
yield-reduces-priority technique.

> Arguably, a number of things need to be done in
> the Xen scheduler and synchronization primitives to improve
> the performance of SMP guests. It may be worthwhile to have
> a generic discussion about that on top of the specific problem
> you're encountering.

Here are some random ideas:
* Expose to the guest, via the shared-info page, which vcpus are
actively scheduled or not.
* Implement some kind of a yield or block primitive, like:
+ yield to a specific vcpu (i.e., the one holding the lock you want)
+ block with a vcpu mask.  The vcpu will then be blocked until each of
the vcpus in the mask has been scheduled at least once.

Thoughts?

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel