[Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 6

[Shannon Zhao <zhaoshenglong@xxxxxxxxxx>]

This document is going to explain the design details of Xen booting with
ACPI on ARM. Any comments are welcome.

Changes v5->v6:
* add a new node "uefi" under /hypervisor to pass UEFI informations to
  Dom0 instead of the nodes under /chosen.
* change creation of MADT table, get the information from
  domain->arch.vgic struct
* Reuse grant table region which will not be used by Dom0 when booting
  through ACPI to store the new created ACPI tables.

Changes v4->v5:
* change the description of section 4 to make it more generic
* place EFI and ACPI tables at non-RAM space of Dom0

Changes v3->v4:
* add explanation for minimal DT and the properties
* drop "linux," prefix of the properties
* add explanation for the event channel flag
* create RSDP table since the "xsdt_physical_address" is changed
* since it uses hypervisor_id introduced by ACPI 6.0 to notify Dom0 the
  hypervisor ID, so it needs to limit minimum supported ACPI version for
  Xen on ARM to 6.0.

Changes v2->v3:
* remove the two HVM_PARAMs for grant table and let linux kernel use
  xlated_setup_gnttab_pages() to setup grant table.
* don't modify GTDT table
* add definition of event-channel interrupt flag
* state that route all Xen unused interrupt to Dom0
* state that reusing existing PCI bus_notifier for PCI devices MMIO
* mapping

To Xen itself booting with ACPI, this is similar to Linux kernel except
that Xen doesn't parse DSDT table. So I'll skip this part and focus on
how Xen prepares ACPI tables for Dom0 and how Xen passes them to Dom0.

1. Create minimal DT to pass required informations to Dom0
----------------------------------------------------------
When booting in UEFI mode on ARM64, it needs to pass some UEFI
informations to Dom0. The necessary informations is the address of EFI
System table and EFI Memory Descriptor table, the size of EFI Memory
Descriptor table, the size of EFI Memory Descriptor and the version of
EFI Memory Descriptor. Here it passes these informations through the
"uefi" node under hypervisor of this minimal DT. Dom0 should parse this
DT to get Xen UEFI informations like the way Linux kernel getting normal
UEFI informations. Also, it should check if the DT contains only the
/hypervisor and /chosen nodes to know whether it boots with DT or ACPI.

In addition, Dom0 should parse DT to know whether it runs on Xen
hypervisor, then it should execute a Xen UEFI specific routine to
initialize UEFI.

An example of the minimal DT:
/ {
    #address-cells = <2>;
    #size-cells = <2>;
    hypervisor {
        compatible = "xen,xen-4.3", "xen,xen";
        reg = <0 0xb0000000 0 0x20000>;  /* Only need for booting
without ACPI */
        interrupts = <1 15 0xf08>; /* Only need for booting without ACPI */
        uefi {
            xen,uefi-system-table = <0xXXXXXXXX>;
            xen,uefi-mmap-start = <0xXXXXXXXX>;
            xen,uefi-mmap-size = <0xXXXXXXXX>;
            xen,uefi-mmap-desc-size = <0xXXXXXXXX>;
            xen,uefi-mmap-desc-ver = <0xXXXXXXXX>;
        };
    };

    chosen {
        bootargs = "kernel=Image console=hvc0 earlycon=pl011,0x1c090000
root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force";
        linux,initrd-start = <0xXXXXXXXX>;
        linux,initrd-end = <0xXXXXXXXX>;
    };
};

For details loook at(this will be updated by a patch of Linux kernel)
https://github.com/torvalds/linux/blob/master/Documentation/devicetree/bindings/arm/xen.txt

2. Copy and change some EFI and ACPI tables
-------------------------------------------
a) Create EFI_SYSTEM_TABLE table
Create a new EFI System table. Copy the table header from host original
EFI System table. Change the value of HeaderSize, CRC32 and Revision
fields in this EFI System table header. Assign new values for
FirmwareVendor and FirmwareRevision fields of EFI System table. Create
one ConfigurationTable and assign the value of VendorGuid field to
ACPI_20_TABLE_GUID, the value of VendorTable field to the address of
ACPI RSDP table. This EFI System Table will be passed to Dom0 through
the property "uefi-system-table" in the above minimal DT. So Dom0 could
get ACPI root table address through the ConfigurationTable.

b) Create EFI_MEMORY_DESCRIPTOR table
It needs to notify Dom0 where are the RAM regions. Add memory start and
size information of Dom0 in this table. It's passed to Dom0 through the
properties "uefi-mmap-start", "uefi-mmap-size", "uefi-mmap-desc-size"
and "uefi-mmap-desc-ver" of the minimal DT. Then Dom0 will get the
memory information through this EFI table.

c) Create FADT table
Firstly copy the contents of host FADT table to the new created FADT
table. Then change the value of arm_boot_flags to enable PSCI and HVC.

d) Create MADT table
It needs to change MADT table to restrict the number of vCPUs.
Firstly copy the contents of host MADT table except the interrupt
controller structures to the new created MADT table. For GICv2, it needs
to add dom0_max_vcpus number of GICC entries and one GICD entry. For
GICv3, it needs to add one GICD entry and domain->arch.vgic.nr_regions
number of GICR entries. These information could be got from
domain->arch.vgic struct.

e) Create STAO table
This table is a new added one that's used to define a list of ACPI
namespace names that are to be ignored by the OSPM in Dom0. Currently
we use it to tell OSPM that it should ignore UART defined in SPCR table.
Look at below url for details:
http://wiki.xenproject.org/mediawiki/images/0/02/Status-override-table.pdf

f) Create XSDT table
Create a new XSDT table and copy the contents of host XSDT table. Then
add a new table entry for STAO and change the values of other table's
entries since we create new ones.

g) Create RSDP table
Create a new RSDP table and copy the contents of host RSDP table. Then
change the value of xsdt_physical_address field in RSDP table. As we
create a new XSDT table and the address of XSDT is changed, so it needs
to update the value of "xsdt_physical_address" field in RSDP. So Dom0
could get the right XSDT table rather than the old one.

h) The rest of tables are not new created or changed. They are reused
including DSDT, SPCR, GTDT, etc. It doesn't mask EL2 resources for Dom0
because the Linux kernel would not use EL2 resources when it boots at
EL1.

All above tables will be mapped to Dom0 non-RAM space. Since when
booting through ACPI it doesn't need the grant table region(see below
section 3), it could use this region to store the tables or use the same
way to find one memory region to store them.

3. Dom0 gets grant table and event channel irq information
-----------------------------------------------------------
The OS will have to find a place himself in the memory map for the grant
table region. For instance, Linux can make usage of
xlated_setup_gnttab_pages.

To event channel interrupt, reuse HVM_PARAM_CALLBACK_IRQ and add a new
delivery type to get it.
val[63:56] == 3: val[15:8] is flag: val[7:0] is a PPI (ARM and ARM64
only)
The definition of flag reusing the definition of xenv table. Bit 0
stands interrupt mode(1: edge triggered, 0: level triggered) and bit 1
stands interrupt polarity(1: active low, 0: active high).

4. Map MMIO regions
-------------------
Add a new XENMAPSPACE "XENMAPSPACE_dev_mmio" for mapping mmio region.
Dom0 could use the hypercall XENMEM_add_to_physmap or
XENMEM_add_to_physmap_batch to map the mmio regions of devices. The
usage of the hypercall's parameters:
- domid: DOMID_SELF.
- space: XENMAPSPACE_dev_mmio.
- size: Number of pages to go through.
- idxs: native physical addresses.
- gpfns: guest physical addresses where the mapping should appear.

For example, Linux could register a bus_notifier for platform and amba
bus. Within the notifier, check if the device is newly added, if so call
the hypercall to map the mmio regions. And for PCI bus devices, Linux
could reuse the existing PCI bus_notifier like X86.

5. Route device interrupts to Dom0
----------------------------------
Route all the SPI interrupts to Dom0 before Dom0 booting, except the
interrupts used by Xen. For uart, it could get the interrupt from SPCR
table and exclude it. For SMMU, it could get the interrupts from IORT
table and exclude them as well.

Note: The above is the end of this design doc text. Below words are
going to illustrate what changes will be made to Linux kernel especially
for UEFI and ACPI initialization.

Regarding the initialization flow of Linux kernel, it needs to move
xen_early_init() before efi_init(). Then xen_early_init() will check
whether it runs on Xen through the /hypervisor node and efi_init() will
call a new function, e.g. fdt_find_xen_uefi_params(), to parse those
xen,uefi-* parameters just like the existing efi_get_fdt_params().

And in arm64_enable_runtime_services() it will check whether it runs on
Xen and call another new function, e.g. xen_efi_runtime_setup() to setup
runtime service instead of efi_native_runtime_setup(). The
xen_efi_runtime_setup() will assign the runtime function pointers with
the functions of driver/xen/efi.c.

And since we pass a /hypervisor node and a /chosen node to Dom0, it
needs to check whether the DTS only contains a /hypervisor node and a
/chosen node in acpi_boot_table_init().

-- 
Shannon


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel