[Xen-devel] [PATCH RFC 00/49] xen: add core scheduling support

[Juergen Gross <jgross@xxxxxxxx>]

This series is very RFC!!!!

Add support for core- and socket-scheduling in the Xen hypervisor.

Via boot parameter sched_granularity=core (or sched_granularity=socket)
it is possible to change the scheduling granularity from thread (the
default) to either whole cores or even sockets.

All logical cpus (threads) of the core or socket are always scheduled
together. This means that on a core always vcpus of the same domain
will be active, and those vcpus will always be scheduled at the same
time.

This is achieved by switching the scheduler to no longer see vcpus as
the primary object to schedule, but "schedule items". Each schedule
item consists of as many vcpus as each core has threads on the current
system. The vcpu->item relation is fixed.

I have done some very basic performance testing: on a 4 cpu system
(2 cores with 2 threads each) I did a "make -j 4" for building the Xen
hypervisor. With This test has been run on dom0, once with no other
guest active and once with another guest with 4 vcpus running the same
test. The results are (always elapsed time, system time, user time):

sched_granularity=thread, no other guest: 116.10 177.65 207.84
sched_granularity=core,   no other guest: 114.04 175.47 207.45
sched_granularity=thread, other guest:    202.30 334.21 384.63
sched_granularity=core,   other guest:    207.24 293.04 371.37

All tests have been performed with credit2, the other schedulers are
untested up to now.

Cpupools are not yet working, as moving cpus between cpupools needs
more work.

HVM domains do not work yet, there is a doublefault in Xen at the
end of Seabios. I'm currently investigating this issue.

This is x86-only for the moment. ARM doesn't even build with the
series applied. For full ARM support I might need some help with the
ARM specific context switch handling.

The first 7 patches have been sent to xen-devel already, I'm just
adding them here for convenience as they are prerequisites.

I'm especially looking for feedback regarding the overall idea and
design.


Juergen Gross (49):
  xen/sched: call cpu_disable_scheduler() via cpu notifier
  xen: add helper for calling notifier_call_chain() to common/cpu.c
  xen: add new cpu notifier action CPU_RESUME_FAILED
  xen: don't free percpu areas during suspend
  xen/cpupool: simplify suspend/resume handling
  xen/sched: don't disable scheduler on cpus during suspend
  xen/sched: fix credit2 smt idle handling
  xen/sched: use new sched_item instead of vcpu in scheduler interfaces
  xen/sched: alloc struct sched_item for each vcpu
  xen/sched: move per-vcpu scheduler private data pointer to sched_item
  xen/sched: build a linked list of struct sched_item
  xen/sched: introduce struct sched_resource
  xen/sched: let pick_cpu return a scheduler resource
  xen/sched: switch schedule_data.curr to point at sched_item
  xen/sched: move per cpu scheduler private data into struct
    sched_resource
  xen/sched: switch vcpu_schedule_lock to item_schedule_lock
  xen/sched: move some per-vcpu items to struct sched_item
  xen/sched: add scheduler helpers hiding vcpu
  xen/sched: add domain pointer to struct sched_item
  xen/sched: add id to struct sched_item
  xen/sched: rename scheduler related perf counters
  xen/sched: switch struct task_slice from vcpu to sched_item
  xen/sched: move is_running indicator to struct sched_item
  xen/sched: make null scheduler vcpu agnostic.
  xen/sched: make rt scheduler vcpu agnostic.
  xen/sched: make credit scheduler vcpu agnostic.
  xen/sched: make credit2 scheduler vcpu agnostic.
  xen/sched: make arinc653 scheduler vcpu agnostic.
  xen: add sched_item_pause_nosync() and sched_item_unpause()
  xen: let vcpu_create() select processor
  xen/sched: use sched_resource cpu instead smp_processor_id in
    schedulers
  xen/sched: switch schedule() from vcpus to sched_items
  xen/sched: switch sched_move_irqs() to take sched_item as parameter
  xen: switch from for_each_vcpu() to for_each_sched_item()
  xen/sched: add runstate counters to struct sched_item
  xen/sched: rework and rename vcpu_force_reschedule()
  xen/sched: Change vcpu_migrate_*() to operate on schedule item
  xen/sched: move struct task_slice into struct sched_item
  xen/sched: add code to sync scheduling of all vcpus of a sched item
  xen/sched: add support for multiple vcpus per sched item where missing
  x86: make loading of GDT at context switch more modular
  xen/sched: add support for guest vcpu idle
  xen/sched: modify cpupool_domain_cpumask() to be an item mask
  xen: round up max vcpus to scheduling granularity
  xen/sched: support allocating multiple vcpus into one sched item
  xen/sched: add a scheduler_percpu_init() function
  xen/sched: support core scheduling in continue_running()
  xen/sched: make vcpu_wake() core scheduling aware
  xen/sched: add scheduling granularity enum

 xen/arch/arm/domain.c                |   14 +
 xen/arch/arm/domain_build.c          |   13 +-
 xen/arch/arm/smpboot.c               |    6 +-
 xen/arch/x86/dom0_build.c            |   11 +-
 xen/arch/x86/domain.c                |  243 +++++--
 xen/arch/x86/hvm/dom0_build.c        |    9 +-
 xen/arch/x86/hvm/hvm.c               |    7 +-
 xen/arch/x86/hvm/viridian/viridian.c |    1 +
 xen/arch/x86/hvm/vlapic.c            |    1 +
 xen/arch/x86/hvm/vmx/vmcs.c          |    6 +-
 xen/arch/x86/hvm/vmx/vmx.c           |    5 +-
 xen/arch/x86/mm.c                    |   10 +-
 xen/arch/x86/percpu.c                |    3 +-
 xen/arch/x86/pv/descriptor-tables.c  |    6 +-
 xen/arch/x86/pv/dom0_build.c         |   10 +-
 xen/arch/x86/pv/domain.c             |   19 +
 xen/arch/x86/pv/emul-priv-op.c       |    2 +
 xen/arch/x86/pv/shim.c               |    4 +-
 xen/arch/x86/pv/traps.c              |    6 +-
 xen/arch/x86/setup.c                 |    2 +
 xen/arch/x86/smpboot.c               |    5 +-
 xen/arch/x86/traps.c                 |   10 +-
 xen/common/cpu.c                     |   61 +-
 xen/common/cpupool.c                 |  161 ++---
 xen/common/domain.c                  |   36 +-
 xen/common/domctl.c                  |   28 +-
 xen/common/keyhandler.c              |    7 +-
 xen/common/sched_arinc653.c          |  258 ++++---
 xen/common/sched_credit.c            |  704 +++++++++---------
 xen/common/sched_credit2.c           | 1143 +++++++++++++++---------------
 xen/common/sched_null.c              |  423 +++++------
 xen/common/sched_rt.c                |  538 +++++++-------
 xen/common/schedule.c                | 1292 ++++++++++++++++++++++++----------
 xen/common/softirq.c                 |    6 +-
 xen/common/wait.c                    |    5 +-
 xen/include/asm-x86/cpuidle.h        |    2 +-
 xen/include/asm-x86/dom0_build.h     |    3 +-
 xen/include/asm-x86/domain.h         |    3 +
 xen/include/xen/cpu.h                |   29 +-
 xen/include/xen/domain.h             |    3 +-
 xen/include/xen/perfc_defn.h         |   32 +-
 xen/include/xen/sched-if.h           |  276 ++++++--
 xen/include/xen/sched.h              |   40 +-
 xen/include/xen/softirq.h            |    1 +
 44 files changed, 3175 insertions(+), 2269 deletions(-)

-- 
2.16.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel