Grouping domains for the purpose of sharing scheduler credits was the intent of the sched-groups project done by Mike Day at IBM:
A related project called Domain Groups (domgrps for short) also exists:
The primary difference is that domgrps is a general-purpose domain grouping framework with integration into both the VMM and userspace tools. It does group migration, moving domains b/w groups, etc..
To demonstrate the flexibility of the domgrps architecture, I merged domgrps and schedgrps:
It should be equally straightforward to adapt domgrps to accommodate this newly proposed pooling concept. I'd be happy to get involved if Keir suggests which grouping features are of interest. If we first need a discussion to define the potential grouping features that's OK too, but I'll delay that effort until I'm asked.
Cheers -Chris
On Dec 10, 2008, at 8:34 AM, Keir Fraser wrote: That was grouping domains to directly share scheduling credits, rather than grouping to share physical resources. -- Keir On 10/12/2008 13:21, "Tian, Kevin" < kevin.tian@xxxxxxxxx> wrote: I remember seeing some post before about domain group scheduler.
Not sure about its progress now, and maybe you can check that
thread to see anything useful?
Thanks,
Kevin
From:Juergen Gross
Sent: Wednesday, December 10, 2008 9:10 PM
Hi,
Currently the XEN credit scheduler has its pitfalls in
supporting weights of
domains together with cpu pinning (see the threads
http://lists.xensource.com/archives/html/xen-devel/2007-02/msg0
0006.html
http://lists.xensource.com/archives/html/xen-devel/2006-10/msg0
0365.html
http://lists.xensource.com/archives/html/xen-devel/2007-07/msg0
0303.html
which include a rejected patch).
We are facing this problem, too. We tried the above patch, but
it didn't solve
our problem completely, so we decided to start a new solution.
Our basic requirement is to limit a set of domains to a set of
physical cpus
while specifying the scheduling weight for each domain. The
general (and in my
opinion best) solution would be the introduction of a "pool"
concept in XEN.
Each physical cpu is dedicated to exactly one pool. At start
of XEN this is
pool0. A domain is member of a single pool (dom0 will always
be member of
pool0), there may be several domains in one pool. Scheduling
does not cross
pool boundaries, so the weight of a domain is only related to
the weight of
the other domains in the same pool. So it is possible to have
an own scheduler
for each pool.
What changes would be needed?
- The hypervisor must be pool-aware. It needs information
about the pool
configuration (cpu mask, scheduler) and the pool membership
of a domain.
The scheduler must restrict itself to its own pool only.
- There must be an interface to set and query the pool configuration.
- At domain creation the domain must be added to a pool.
- libxc must be expanded to support the new interfaces.
- xend and the xm command must support pools, defaulting to
pool0 if no pool
is specified
The xm commands could look like this:
xm pool-create pool1 ncpu=4 # create a pool with 4 cpus
xm pool-create pool2 cpu=1,3,5 # create a pool with
3 dedicated cpus
xm pool-list # show pools:
pool cpus sched domains
pool0 0,2,4 credit 0
pool1 6-9 credit 1,7
pool2 1,3,5 credit 2,3
xm pool-modify pool1 ncpu=3 # set new number of cpus
xm pool-modify pool1 cpu=6,7,9 # modify cpu-pinning
xm pool-destroy pool1 # destroy pool
xm create vm5 pool=pool1 # start domain in pool1
There is much more potential in this approach:
- add memory to a pool? Could be interesting for NUMA
- recent discussions on xen-devel related to scheduling
(credit scheduler for
client virtualization) show some demand for further work
regarding priority
and/or grouping of domains
- this might be an interesting approach for migration of
multiple related
domains (pool migration)
- move (or migrate?) a domain to another pool
- ...
Any comments, suggestions, work already done, ...?
Otherwise we will be starting our effort soon.
Juergen
--
Juergen Gross Principal Developer
IP SW OS6 Telephone: +49 (0) 89 636 47950
Fujitsu Siemens Computers e-mail:
juergen.gross@xxxxxxxxxxxxxxxxxxx
Otto-Hahn-Ring 6 Internet: www.fujitsu-siemens.com
D-81739 Muenchen Company details:
www.fujitsu-siemens.com/imprint.html
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
_______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxxhttp://lists.xensource.com/xen-devel
|
