Re: [PATCH] xen/arm: acpi: Support memory reserve configuration table

[Julien Grall <julien@xxxxxxx>]

Hi,

On 25/08/2022 08:59, Leo Yan wrote:
> On Fri, Aug 19, 2022 at 01:10:10PM +0100, Marc Zyngier wrote:
>
> [...]
>
> > > > > In the context of Xen, dom0 doesn't have direct access to the host ITS
> > > > > because we are emulating it. So I think it doesn't matter for us because we
> > > > > can fix our implementation if it is affected.
> > > > >
> > > > > That said, kexec-ing dom0 (or any other domain) on Xen on Arm would require
> > > > > some work to be supported. OOI, @leo is it something you are investigating?
> > > >
> > > > Now I am working on automative reference platform; the first thing for
> > > > me is to resolve the kernel oops.
> > > >
> > > > For long term, I think the kexec/kdump would be important to be
> > > > supported, to be clear, so far supporting kexec/kdump for Xen/Linux is
> > > > not priority for my work.
> > > >
> > > > Also thanks a lot for Ard and Mark's replying. To be honest, I missed
> > > > many prerequisites (e.g. redistributor configurations for GIC in
> > > > hypervisor) and seems Xen uses a different way by emulating GICv3
> > > > controller for guest OS.  So now I am bit puzzle what's for next step
> > > > or just keep as it is?
> > > If i understand Julien's remark correctly, the dom0 GICv3 is emulated,
> > > and so it should not suffer from the issue that we are working around
> > > here.
> Before proceeding discussion, I would like step back to get clear for
> the GIC implementation in Xen, otherwise, it's really hard for me to
> catch up the dicussion :)
>
> For me it's clear that Xen emulates GICv3 for DomU, but I am still
> confused how GICv3 works for Dom0.
>
> Xen directly passes ACPI MADT table from UEFI to Linux kernel to Dom0,
> (see functions acpi_create_madt() and gic_make_hwdom_madt()), which
> means the Linux kernel Dom0 uses the same ACPI table to initialize GICv3
> driver, but since Linux kernel Dom0 accesses GIC memory region as IPA,
> it still trap to Xen in EL2 for stage 2 translation, so finally Xen
> can emulate the GICv3 device for Dom0.
In the default setup, dom0 is also the hardware domain. So it owns all 
of the devices but the ones used by Xen (e.g. interrupt controller, SMMU).
Therefore, dom0 will use the same memory layout as the host. At which 
point, it is a lot more convenient to re-use the host ACPI tables and 
rewrite only what's necessary.
> This is quite different from DomU.  Xen prepares a DT node for GICv3
> rather than directly passing ACPI table, so DomU kernel initializes
> GICv3 driver based on the DT binding.
DomUs memory layout is defined by Xen. So we need to create the 
Device-Tree and ACPI tables (both are supported) from scrartch.
> Simply to say, no matter Dom0 using ACPI table or DomU using DT
> binding, at the end Xen emulates GICv3 device for all of them.
Correct. In both situations the GICv3 will be owned by Xen and we will 
emulate the bits that are not virtualized by the CPUs (e.g. 
re-distributors).
> Another thing is not clear for me is that I can see Xen allocates
> redistributor pending page (see gicv3_lpi_set_pendtable()), after Dom0
> or DomU kernel boots up, kernel allocates another RD pending page; so
> the question is how these two different pending pages co-work
> together.
Xen allocates the pending pages that will be used by the host. Dom0/DomU 
will be allocating pending pages that will be used by the virtual GICv3.
> In other words, let's assume the Dom0 kernel panic and its secondary
> kernel is launched by kexec, is it necessarily for the secondary
> kernel to reuse the primary kernel's RD pending page?
No.

>  Or in this case
> it's no matter for the RD pending page in Dom0 and it's safe for Xen
> always maintains its own RD pending page in EL2?
Dom0 doesn't have direct access to the host GICv3 (this will be 
controlled by Xen). What we expose to dom0 is a virtual GICv3.
So effectively we have two different GICv3 and each of them will require 
their own set of pending table.
> > The problem is that there is no way to distinguish a system that
> > suffers from GICR LPI tables being immutable from one that allows the
> > LPI configuration being changed (either because the HW allows it or
> > because the hypervisor plays other games).
> Let me ask a stupid question.  Seems to me, GICR LPI tables can be
> configured as below options
>
> - The hypervisor pre-allocates GICR LPI tables and pass the memory
>    region info to Dom0 kernel;
>
> - The hypervisor doesn't allocate GICR LPI tables, and then Dom0
>    kernel allocates GICR LPI tables for the virtual GICv3, and Dom0
>    directly write data to the GICR LPI tables and the table is used by
>    physical GICv3;
>
> - The hypervisor pre-allocates GICR LPI tables for phycial GICv3 and
>    Dom0 kernel allocates another GICR LPI tables for virtual GICv3,
>    and Xen needs to sync between these two tables.
>
> To be clear, all of above three options are hypothesis.  So please
> correct me if anything is wrong (or even total are wrong?!).
I will defer this question to Marc.

Cheers,

--
Julien Grall