RE: [Xen-devel] credit scheduler

["Ian Pratt" <m+Ian.Pratt@xxxxxxxxxxxx>]

> In my most basic test I have a 4 socket dual core Intel box (Paxville)
with
> a
> uniprocessor dom0 and 7 uniprocessor domUs.  Each of the domUs is
pinned on
> its own core with the first core of the system left for dom0.  When
running
> the credit scheduler the dom0 VCPU will bounce around the system,
sometimes
> landing on the same thread as one of the domUs or sometimes on one of
the
> sibling hyperthreads (this appears to happen a majority of the time it
> moves).  This is less than ideal when considering cache warmth and the
> sharing of CPU resources when the first core of the system is always
> available in this configuration.  Does the credit scheduler have any
> awareness of cache warmth or CPU siblings when balancing?
> 
> I have also seen similar behavior when running tests in the domUs such
that
> each has its VCPU running at 100% utilization so I believe this
behavior to
> be fairly uniform.

The sometimes suboptimal use of hyperthreading is well understood and is
on the todo list. It hasn't been a priority as the current generation of
Intel Core/Core2 and Opteron CPUs don't have HT.

Apart from the hyperthreading sibling case, a dom0 vcpu should only ever
migrate to another CPU that has been idle for a complete rebalancing
period. Hence the rate of movement should be very low and thus the
overhead of priming the cache tiny.  It's conceivable one could
construct an argument that it's not worth the special case code in the
scheduler to fix...

BTW: The PhD thesis of our very own James Bulpin has lots of useful data
on cache warming times and how to optimize for HT systems:
http://www.cl.cam.ac.uk/TechReports/UCAM-CL-TR-619.pdf

Allowing dom0 vcpus to explicitly be pinned would certainly be a good
thing though, and is slated for post 3.0.3 -- see the thread on this
topic earlier today. Actually, in the interim it would be easy to add a
xen command line parameter to pin dom0 vcpus...

Ian


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel