Re: [RFC v2 04/12] xen/arm32: head: Remove restriction where to load Xen

[Luca Fancellu <Luca.Fancellu@xxxxxxx>]

> On 22 Oct 2022, at 16:04, Julien Grall <julien@xxxxxxx> wrote:
> 
> From: Julien Grall <jgrall@xxxxxxxxxx>
> 
> At the moment, bootloaders can load Xen anywhere in memory but the
> region 2MB - 4MB. While I am not aware of any issue, we have no way
> to tell the bootloader to avoid that region.
> 
> In addition to that, in the future, Xen may grow over 2MB if we
> enable feature like UBSAN or GCOV. To avoid widening the restriction
> on the load address, it would be better to get rid of it.
> 
> When the identity mapping is clashing with the Xen runtime mapping,
> we need an extra indirection to be able to replace the identity
> mapping with the Xen runtime mapping.
> 
> Reserve a new memory region that will be used to temporarily map Xen.
> For convenience, the new area is re-using the same first slot as the
> domheap which is used for per-cpu temporary mapping after a CPU has
> booted.
> 
> Furthermore, directly map boot_second (which cover Xen and more)
> to the temporary area. This will avoid to allocate an extra page-table
> for the second-level and will helpful for follow-up patches (we will
> want to use the fixmap whilst in the temporary mapping).
> 
> Lastly, some part of the code now needs to know whether the temporary
> mapping was created. So reserve r12 to store this information.
> 
> Signed-off-by: Julien Grall <jgrall@xxxxxxxxxx>
> ----
> 
>    Changes in v2:
>        - Patch added
> ---

Hi Julien,

I’m hitting an assert with this one, tested on qemu and fvp:

Xen 4.17-rc
(XEN) Xen version 4.17-rc (user@hostname) (arm-poky-linux-gnueabi-gcc (GCC) 
11.3.0) debug=y Tue Oct 25 10:51:06 UTC 2022
(XEN) Latest ChangeSet:
(XEN) build-id: ab143b13f4394ced5331d6ff1cedebdb2ffadc07
(XEN) Processor: 412fc0f1: "ARM Limited", variant: 0x2, part 0xc0f,rev 0x1
(XEN) 32-bit Execution:
(XEN)   Processor Features: 00001131:00011001
(XEN)     Instruction Sets: AArch32 A32 Thumb Thumb-2 ThumbEE Jazelle
(XEN)     Extensions: GenericTimer
(XEN)   Debug Features: 02010555
(XEN)   Auxiliary Features: 00000000
(XEN)   Memory Model Features: 10201105 20000000
(XEN)                          01240000 02102211
(XEN)   ISA Features: 02101110 13112111 21232041
(XEN)                 11112131 10011142 00000000
(XEN) Using SMC Calling Convention v1.0
(XEN) Using PSCI v0.2
(XEN) SMP: Allowing 4 CPUs
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 62500 KHz
(XEN) GICv2m extension register frame:
(XEN)         gic_v2m_addr=0000000008020000
(XEN)         gic_v2m_size=0000000000001000
(XEN)         gic_v2m_spi_base=80
(XEN)         gic_v2m_num_spis=64
(XEN) GICv2 initialization:
(XEN)         gic_dist_addr=0000000008000000
(XEN)         gic_cpu_addr=0000000008010000
(XEN)         gic_hyp_addr=0000000008030000
(XEN)         gic_vcpu_addr=0000000008040000
(XEN)         gic_maintenance_irq=25
(XEN) GICv2: 288 lines, 4 cpus (IID 00000000).
(XEN) XSM Framework v1.0.1 initialized
(XEN) Initialising XSM SILO mode
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Allocated console ring of 32 KiB.
(XEN) VFP implementer 0x41 architecture 4 part 0x30 variant 0xf rev 0x0
(XEN) CPU0: Guest atomics will try 1 times before pausing the domain
(XEN) Bringing up CPU1
(XEN) Assertion '!lpae_is_valid(*entry)' failed at arch/arm/domain_page.c:69
(XEN) ----[ Xen-4.17-rc  arm32  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) PC:     00269af4 init_domheap_mappings+0x178/0x17c
(XEN) CPSR:   2000005a MODE:Hypervisor
(XEN)      R0: 002ffe48 R1: 0000007f R2: bfff8f7d R3: 00400000
(XEN)      R4: 000bfff8 R5: 43ffa010 R6: 00000001 R7: 43ffa000
(XEN)      R8: 43ff8000 R9: 00000000 R10:00000000 R11:002ffe6c R12:000bfff8
(XEN) HYP: SP: 002ffe44 LR: 00269a2c
(XEN)
(XEN)   VTCR_EL2: 00000000
(XEN)  VTTBR_EL2: 0000000000000000
(XEN)
(XEN)  SCTLR_EL2: 30cd187f
(XEN)    HCR_EL2: 00000038
(XEN)  TTBR0_EL2: 000000004014a000
(XEN)
(XEN)    ESR_EL2: 00000000
(XEN)  HPFAR_EL2: 00000000
(XEN)      HDFAR: 00000000
(XEN)      HIFAR: 00000000
(XEN)
(XEN) Xen stack trace from sp=002ffe44:
(XEN)    00000000 bfff8f7d 00400000 002ffe6c 43ffa000 00000001 002a6000 0033b224
(XEN)    002a2ec8 002ffe84 002700cc 00000001 00000000 002a6000 0033b224 002ffeb4
(XEN)    00276338 00000001 002a6000 0033b224 00000001 00000000 002a6000 0033b224
(XEN)    002a2ec8 00000000 00000000 002ffecc 0020185c 002ffecc 002ad978 00000001
(XEN)    00000001 002fff54 002c7d80 11112131 10011142 00000000 002a4744 00000000
(XEN)    00000000 00008f20 00004000 48000000 002e01f0 00000000 c0000000 00000000
(XEN)    40000000 00000001 002e0208 002a6e84 002a6e80 002afaa0 002e31f0 00000000
(XEN)    c0000000 00000003 ffffffff 00000000 002a9000 0020005c 00000000 00000000
(XEN)    48000000 40010000 3fe10000 00000000 00200064 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) Xen call trace:
(XEN)    [<00269af4>] init_domheap_mappings+0x178/0x17c (PC)
(XEN)    [<00269a2c>] init_domheap_mappings+0xb0/0x17c (LR)
(XEN)    [<002700cc>] init_secondary_pagetables+0x60/0x168
(XEN)    [<00276338>] __cpu_up+0x38/0x1d8
(XEN)    [<0020185c>] cpu_up+0xe0/0x114
(XEN)    [<002c7d80>] start_xen+0xd40/0x1138
(XEN)    [<00200064>] head.o#primary_switched+0x8/0x18
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion '!lpae_is_valid(*entry)' failed at arch/arm/domain_page.c:69
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...