Re: [Xen-devel] [RFC PATCH v2 00/26] arm64: Dom0 ITS emulation

[Vijay Kilari <vijay.kilari@xxxxxxxxx>]

Hi Andre,

On Thu, Dec 22, 2016 at 11:54 PM, Andre Przywara <andre.przywara@xxxxxxx> wrote:
> Hi,
>
> this is a reworked version of the Dom0 GICv3-ITS emulation series.
> This is still not fully where I want it and has some loose bits and
> pieces still, but since there are significant changes in the architecture
> I wanted to have an opinion before going ahead and replacing every single
> number with a named constant ;-) If that smells like a "send out before
> the end of the year", you are spot on.
>
> This series introduces ARM GICv3 ITS emulation, for now restricted to
> Dom0 only. The ITS is an interrupt controller widget providing a
> sophisticated way to deal with MSIs in a scalable manner.
> For hardware which relies on the ITS to provide interrupts for its
> peripherals this code is needed to get a machine booted into Dom0 at all.
> ITS emulation for DomUs is only really useful with PCI passthrough,
> which is not yet available for ARM. It is expected that this feature
> will be co-developed with the ITS DomU code. However this code drop here
> considered DomU emulation already, to keep later architectural changes
> to a minimum.
>
> Some generic design principles:
>
> * The current GIC code statically allocates structures for each supported
> IRQ (both for the host and the guest), which due to the potentially
> millions of LPI interrupts is not feasible to copy for the ITS.
> So we refrain from introducing the ITS as a first class Xen interrupt
> controller, also we don't hold struct irq_desc's or struct pending_irq's
> for each possible LPI.
> Fortunately LPIs are only interesting to guests, so we get away with
> storing only the virtual IRQ number and the guest VCPU for each allocated
> host LPI, which can be stashed into one uint64_t. This data is stored in
> a two-level table, which is both memory efficient and quick to access.
> We hook into the existing IRQ handling and VGIC code to avoid accessing
> the normal structures, providing alternative methods for getting the
> needed information (priority, is enabled?) for LPIs.
> For interrupts which are queued to or are actually in a guest we
> allocate struct pending_irq's on demand. As it is expected that only a
> very small number of interrupts is ever on a VCPU at the same time, this
> seems like the best approach. For now allocated structs are re-used and
> held in a linked list.
>
> * On the guest side we (later will) have to deal with malicious guests
> trying to hog Xen with mapping requests for a lot of LPIs, for instance.
> As the ITS actually uses system memory for storing status information,
> we use this memory (which the guest has to provide) to naturally limit
> a guest. For those tables which are page sized (devices, collections (CPUs),
> LPI properties) we map those pages into Xen, so we can easily access
> them from the virtual GIC code.
> Unfortunately the actual interrupt mapping tables are not necessarily
> page aligned, also can be much smaller than a page, so mapping all of
> them permanently is fiddly. As ITS commands in need to iterate those
> tables are pretty rare after all, we for now map them on demand upon
> emulating a virtual ITS command.
>
> * An obvious approach to handling some guest ITS commands would be to
> propagate them to the host, for instance to map devices and LPIs and
> to enable or disable LPIs.
> However this (later with DomU support) will create an attack vector, as
> a malicious guest could try to fill the host command queue with
> propagated commands.
> So in contrast to the previous RFC post this version now completely avoids
> this situation. For mapping devices and LPIs we rely on this being done
> via a hypercall prior to the actual guest run. For enabling and disabling
> LPIs we keep this bit on the virtual side and let LPIs always be enabled
> on the host side, dealing with the consequences this approach creates.
>
> This series is still a draft, with some known and many unknown issues.
> I made ITS support a Kconfig option, also it is only supported on arm64.
> This leads to some hideous constructs like an #ifdef'ed header file with
> empty function stubs, but I guess we can clean this up later in the
> upstreaming process.
>
> There are numerous changes compared to the last post, mainly affecting
> the now missing ITS command progagation. I also added locking to the
> "usual suspects" data structures.
> I picked some low hanging fruits from the review comments.
> Things I haven't addresses well is the whole memory management, in terms
> of marking pages r/o for a guest or allocating Xen memory from the proper
> bucket. This will be addresses with the next post.
>
> For now this code happens to boot Dom0 on an ARM fast model with ITS
> support. I still haven't had the chance to get hold of a Xen supported
> hardware platform with an ITS yet, so running on real hardware is a bit
> terra incognita.
>
> The code can also be found on the its/rfc-v2 branch here:
> git://linux-arm.org/xen-ap.git
> http://www.linux-arm.org/git?p=xen-ap.git;a=shortlog;h=refs/heads/its/rfc-v2

What is the kernel version that you tried to boot dom0?.
Have you tried with smmu and pci devices?.  Please share your kernel config.

>
> Cheers,
> Andre
>
> Andre Przywara (26):
>   ARM: GICv3 ITS: parse and store ITS subnodes from hardware DT
>   ARM: GICv3: allocate LPI pending and property table
>   ARM: GICv3 ITS: allocate device and collection table
>   ARM: GICv3 ITS: map ITS command buffer
>   ARM: GICv3 ITS: introduce ITS command handling
>   ARM: GICv3 ITS: introduce device mapping
>   ARM: GICv3 ITS: introduce host LPI array
>   ARM: GICv3 ITS: map device and LPIs to the ITS on physdev_op hypercall
>   ARM: GICv3: introduce separate pending_irq structs for LPIs
>   ARM: GICv3: forward pending LPIs to guests
>   ARM: GICv3: enable ITS and LPIs on the host
>   ARM: vGICv3: handle virtual LPI pending and property tables
>   ARM: vGICv3: Handle disabled LPIs
>   ARM: vGICv3: introduce basic ITS emulation bits
>   ARM: vITS: handle CLEAR command
>   ARM: vITS: handle INT command
>   ARM: vITS: handle MAPC command
>   ARM: vITS: handle MAPD command
>   ARM: vITS: handle MAPTI command
>   ARM: vITS: handle MOVI command
>   ARM: vITS: handle DISCARD command
>   ARM: vITS: handle INV command
>   ARM: vITS: handle INVALL command
>   ARM: vITS: create and initialize virtual ITSes for Dom0
>   ARM: vITS: create ITS subnodes for Dom0 DT
>   ARM: vGIC: advertising LPI support
>
>  xen/arch/arm/Kconfig              |  20 +
>  xen/arch/arm/Makefile             |   2 +
>  xen/arch/arm/efi/efi-boot.h       |   1 -
>  xen/arch/arm/gic-its.c            | 934 
> ++++++++++++++++++++++++++++++++++++++
>  xen/arch/arm/gic-v3.c             |  92 +++-
>  xen/arch/arm/gic.c                |   9 +-
>  xen/arch/arm/physdev.c            |  24 +
>  xen/arch/arm/vgic-its.c           | 842 ++++++++++++++++++++++++++++++++++
>  xen/arch/arm/vgic-v3.c            | 274 +++++++++--
>  xen/arch/arm/vgic.c               |  71 ++-
>  xen/include/asm-arm/bitops.h      |   2 +
>  xen/include/asm-arm/cache.h       |   4 +
>  xen/include/asm-arm/domain.h      |  13 +-
>  xen/include/asm-arm/gic-its.h     | 217 +++++++++
>  xen/include/asm-arm/gic_v3_defs.h |  67 ++-
>  xen/include/asm-arm/irq.h         |   8 +
>  xen/include/asm-arm/vgic.h        |  15 +
>  17 files changed, 2558 insertions(+), 37 deletions(-)
>  create mode 100644 xen/arch/arm/gic-its.c
>  create mode 100644 xen/arch/arm/vgic-its.c
>  create mode 100644 xen/include/asm-arm/gic-its.h
>
> --
> 2.9.0
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel