[Xen-devel] [RFC PATCH 00/49] New VGIC(-v2) implementation

[Andre Przywara <andre.przywara@xxxxxxxxxx>]

tl;dr: More preparatory patches from patch 07, actual new VGIC starting
at patch 20.
=============

During development of the Dom0 ITS MSI support last year we realised
that the existing GIC interrupt controller emulation has some shortcomings.
After some tries to fix those in the existing code, it was agreed upon
that the problems are fundamental and a new implementation based on the
"new VGIC" in KVM is the best choice.
This is the first drop of this new VGIC implementation. It lives in the
xen/arch/arm/vgic/ directory and is written to be a compile time option,
so people can choose whether to use the new VGIC or the existing
implementation. This is just for a transitional period, the old VGIC is
expected to be removed after confidence in the new implementation has grown.

This series starts with some GICv3 redistributor cleanup, which I posted
before. I need to incorporate the comments from the list, but for now I left
those patches as it from the previous post.

Starting with patch 07 there are some more cleanups and preparations for
the existing VGIC/GIC code. A big part of those patches are preparations to
properly support level triggered interrupts. This is one of the biggest
problems in the existing VGIC, which only correctly emulates edge triggered
IRQs. This affects both arch code and some users like the timer and the
event channel.

Starting with patch 20 we plumb in the new VGIC then. This is done in a
new directory, with all the files actually not wired into the build system
until the very last patch. The idea is to split the series into reviewable
chunks without resorting to nasty hacks to keep bisectability.
The code was forked from Linux' virt/kvm/arm/vgic/, as of 4.14-rc7, plus
some recent changes to improve support for level triggered and hardware
mapped interrupts, which is what we use heavily in Dom0. The code was
heavily adapted to fit into Xen, starting with using the Xen coding style
and using Xen structure and variable names (struct domain instead of
struct kvm, for instance). Where interfacing functions were similar enough,
they were changed over to the existing Xen name and prototypes (for instance
kvm_vgic_create() was renamed to domain_vgic_register()). As far as possible
the code layout and split was re-used from KVM, so patches in Linux should
be relatively easy to port into Xen. Due to the mentioned changes this can
not be done easily in an automatic way, but it should be not too complicated
to extract the gist of the patch and re-apply this to our code base.

The actual VGIC code splits into several parts:
- The core is the struct vgic_irq, which holds every information about a
virtual IRQ, including a per-IRQ lock. Also there is on (ordered) per-VCPU
list (ap_list), which links the interrupts to be considered by a VCPU.
There are functions to deal with queuing and removing IRQs from those lists
safely, obeying the locking order. (patches 20-23)
- There are functions to push vIRQs on a VCPU list to the list registers,
and handle their state changes. (patches 24-26)
- The distributor MMIO emulation is using separate functions per register,
also having read and write split. (patches 27-37)
- There are functions to deal with Xen specialities. (patches 38-43)
- The data structures and the wiring of the emulation into the hypervisor
  and the guests are done in vgic-init.c. (patches 44-47)
- Finally patch 49 enables the build of the new VGIC. This requires to
  increase the size limit for struct vcpu in patch 48.

Please consider this series the first post that it is. It was briefly tested
on a Juno and a Midway (with and without CONFIG_NEW_VGIC). It does not cover
GICv3 yet, though supporting this should be relatively straightforward.
In fact I have most of the code here already, but I wanted to wait for
initial feedback before applying similar transformations to the *-v3.c files.
Also this does not include ITS support, though the code itself is more ready
for that than the old VGIC ever was.
I did some semi-automatic translation of coding style, so I am sure there
are quite some leftovers that slipped through, including identifiers with
kvm_ in their name ;-)

I would appreciate if somehow could have a look at those patches, patch 20
and following are surely the most interesting ones to have a look at.

Cheers,
Andre

Andre Przywara (49):
  tools: ARM: vGICv3: avoid inserting optional DT properties
  ARM: vGICv3: drop GUEST_GICV3_RDIST_REGIONS symbol
  ARM: GICv3: use hardware GICv3 redistributor regions for Dom0
  ARM: GICv3: simplify GICv3 redistributor stride handling
  ARM: vGICv3: always use architected redist stride
  ARM: vGICv3: remove rdist_stride from VGIC structure
  ARM: VGIC: move gic_remove_from_lr_pending() prototype
  ARM: VGIC: move max_vcpus VGIC limit to struct arch_domain
  ARM: VGIC: change to level-IRQ compatible IRQ injection interface
  ARM: VGIC: carve out struct vgic_cpu and struct vgic_dist
  ARM: VGIC: reorder prototypes in vgic.h
  ARM: VGIC: introduce gic_get_nr_lrs()
  ARM: VGIC: Add hypervisor base address to vgic_v2_setup_hw()
  ARM: VGIC: extend GIC CPU interface definitions
  ARM: GIC: Allow tweaking the active state of an IRQ
  ARM: GIC: allow reading pending state of a hardware IRQ
  ARM: timer: Handle level triggered IRQs correctly
  ARM: evtchn: Handle level triggered IRQs correctly
  ARM: vPL011: Use the VGIC's level triggered IRQs handling if available
  ARM: new VGIC: Add data structure definitions
  ARM: new VGIC: Add acccessor to new struct vgic_irq instance
  ARM: new VGIC: Implement virtual IRQ injection
  ARM: new VGIC: Add IRQ sorting
  ARM: new VGIC: Add IRQ sync/flush framework
  ARM: new VGIC: Add GICv2 world switch backend
  ARM: new VGIC: Implement vgic_vcpu_pending_irq
  ARM: new VGIC: Add MMIO handling framework
  ARM: new VGIC: Add GICv2 MMIO handling framework
  ARM: new VGIC: Add CTLR, TYPER and IIDR handlers
  ARM: new VGIC: Add ENABLE registers handlers
  ARM: new VGIC: Add PENDING registers handlers
  ARM: new VGIC: Add ACTIVE registers handlers
  ARM: new VGIC: Add PRIORITY registers handlers
  ARM: new VGIC: Add CONFIG registers handlers
  ARM: new VGIC: Add TARGET registers handlers
  ARM: new VGIC: Add SGIR register handler
  ARM: new VGIC: Add SGIPENDR register handlers
  ARM: new VGIC: handle hardware mapped IRQs
  ARM: new VGIC: Add event channel IRQ handling
  ARM: new VGIC: Handle virtual IRQ allocation/reservation
  ARM: new VGIC: dump virtual IRQ info
  ARM: new VGIC: provide system register emulation stub
  ARM: new VGIC: Add preliminary stub implementations
  ARM: new VGIC: vgic-init: register VGIC
  ARM: new VGIC: vgic-init: implement vgic_init
  ARM: new VGIC: vgic-init: implement map_resources
  ARM: new VGIC: Add vgic_v2_enable
  ARM: allocate two pages for struct vcpu
  ARM: VGIC: wire new VGIC(-v2) files into Xen build system

 tools/libxl/libxl_arm.c           |   8 -
 xen/arch/arm/Kconfig              |   6 +-
 xen/arch/arm/Makefile             |  10 +-
 xen/arch/arm/domain.c             |  25 +-
 xen/arch/arm/gic-v2.c             |  20 +-
 xen/arch/arm/gic-v3-lpi.c         |   2 +-
 xen/arch/arm/gic-v3.c             |  72 ++--
 xen/arch/arm/gic-vgic.c           |  10 +-
 xen/arch/arm/gic.c                |  15 +
 xen/arch/arm/irq.c                |   2 +-
 xen/arch/arm/time.c               |  36 +-
 xen/arch/arm/traps.c              |   2 +
 xen/arch/arm/vgic-v2.c            |   3 +-
 xen/arch/arm/vgic-v3.c            |  45 +-
 xen/arch/arm/vgic.c               |  46 +-
 xen/arch/arm/vgic/vgic-init.c     | 259 ++++++++++++
 xen/arch/arm/vgic/vgic-mmio-v2.c  | 297 +++++++++++++
 xen/arch/arm/vgic/vgic-mmio.c     | 589 ++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic-mmio.h     | 194 +++++++++
 xen/arch/arm/vgic/vgic-v2.c       | 339 +++++++++++++++
 xen/arch/arm/vgic/vgic.c          | 871 ++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/vgic/vgic.h          |  69 +++
 xen/arch/arm/vpl011.c             |   6 +-
 xen/arch/arm/vtimer.c             |   4 +-
 xen/common/Makefile               |   1 +
 xen/common/list_sort.c            | 170 ++++++++
 xen/include/asm-arm/arm_vgic.h    | 269 ++++++++++++
 xen/include/asm-arm/domain.h      |  92 +---
 xen/include/asm-arm/event.h       |   1 +
 xen/include/asm-arm/gic.h         |  31 +-
 xen/include/asm-arm/gic_v3_defs.h |   5 +
 xen/include/asm-arm/vgic.h        | 170 ++++++--
 xen/include/public/arch-arm.h     |   3 -
 xen/include/xen/list_sort.h       |  11 +
 xen/include/xen/timer.h           |   2 +
 35 files changed, 3456 insertions(+), 229 deletions(-)
 create mode 100644 xen/arch/arm/vgic/vgic-init.c
 create mode 100644 xen/arch/arm/vgic/vgic-mmio-v2.c
 create mode 100644 xen/arch/arm/vgic/vgic-mmio.c
 create mode 100644 xen/arch/arm/vgic/vgic-mmio.h
 create mode 100644 xen/arch/arm/vgic/vgic-v2.c
 create mode 100644 xen/arch/arm/vgic/vgic.c
 create mode 100644 xen/arch/arm/vgic/vgic.h
 create mode 100644 xen/common/list_sort.c
 create mode 100644 xen/include/asm-arm/arm_vgic.h
 create mode 100644 xen/include/xen/list_sort.h

-- 
2.14.1


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel