Re: Xen Booting Problem on ARM Machine

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Thu, 14 Oct 2021, Sai Kiran Kumar Reddy Y wrote:
> On Thu, Oct 14, 2021 at 5:45 AM Stefano Stabellini <sstabellini@xxxxxxxxxx> 
> wrote:
>       On Wed, 13 Oct 2021, Sai Kiran Kumar Reddy Y wrote:
>       > On Fri, Oct 1, 2021 at 8:17 AM Stefano Stabellini 
> <sstabellini@xxxxxxxxxx> wrote:
>       >       Yes there are other ways but without serial is going to be 
> difficult
>       >       because you are not going to see anything until everything 
> works.
>       >
>       >       How do you boot Xen and Dom0 exactly from EDK2? Are you using 
> GRUB or
>       >       loading Xen directly from the EDK2 prompt? Please provide as 
> many
>       >       details as possible so that I might be able to spot any errors.
>       >
>       > I am using GRUB to load Xen. In the GRUB menu, I see two options. 
>       > Option 1: Debian 11 with latest Linux Kernel
>       > Option 2: Debian 11(with Xen hypervisor) with latest Kernel
>       >  
>       >       Can you provide the Device Tree you are using? If you are not 
> passing
>       >       any Device Tree  binary explicitely, then it might be passed
>       >       automatically from EDK2 to Linux/Xen. In that case, just boot 
> from Linux
>       >       then do the following to retrieve the Device Tree:
>       >
>       >       dtc -I fs -O dts /proc/device-tree > host.dts
>       >
>       >       Then please attach host.dts to this email thread.
>       >
>       > Yeah, you are right. It looks like LInux is booting from ACPI. In the 
> bootloader menu, "Automatic ACPI configuration" is
>       disabled. So, I
>       > thought that Linux may be booting from Device Tree. I have tried the 
> "dtc" command you mentioned. But it looks like there's
>       no device-tree
>       > under "/proc". I also tried to get DT info, from 
> "/sys/firmware/devicetree/base" . But, there's no info. under devicetree
>       folder. I am not
>       > quite sure how to get the DT info, if the Linux is booting from ACPI. 
> I am attaching .dsl files, that contain the acpi info.
> 
>       OK, so it is pretty clear that even if "Automatic ACPI configuration" is
>       disabled, it is still booting with ACPI.
> 
> 
>       >       Also for your information it looks like Linux actually booted 
> from ACPI,
>       >       not from Device Tree, as you can see from all the "ACPI" 
> messages in the
>       >       kernel logs.
>       >
>       >       If you need to boot from ACPI, then you need to enable ACPI 
> support in
>       >       Xen, which is disabled by default. You can do that using make
>       >       menuconfig.
>       >
>       > In the make menuconfig of Xen, I do not see any option to enable 
> ACPI.  
> 
>       You definitely need to enable ACPI support in Xen, if you are booting
>       from ACPI, otherwise nothing is going to work.
> 
>       On the latest staging (https://gitlab.com/xen-project/xen) you can
>       enable ACPI as follows:
> 
> 
>       # export CROSS_COMPILE=/path/to/cross-compiler
>       # export XEN_TARGET_ARCH=arm64
>       # cd xen.git/xen
>       # make menuconfig
>       #   --> Configure UNSUPPORTED features
>       #   --> Architecture Features --> ACPI
>       # make
> 
> 
> Hi
> 
> I got the code from Gitlab and installed it, enabling ACPI support in Xen. As 
> I reboot the system, I am able to see 2 options like before. 
> Option 1: Debian with latest kernel
> Option 2 : Debian with Xen
> 
> I have selected Option 2. I did not see any bootinfo membanks error. However, 
> there is the following error in GRUB(just for a fraction of a
> second). 
> 
> "Using modules provided by boot loader in FDT
>   Xen 4.16-unstable (c/s Wed Oct 13 13 13:28:43 2021 -0700 git:4cfab4425d) 
> EFI Loader
>   Couldn't obtain the File System Protocol Interface: ErrCode: 
> 0x8000000000000002"
> 
> I have enabled earlyprintk. I do not see any messages in the Serial. There 
> seems to be some problem with the gitlab version of Xen.

The error comes from xen/common/efi/boot.c:get_parent_handle

Xen is booted as EFI binary and it is trying to load other binaries
using the EFI File System Protocol Interface which is one of the EFI
Boot Services.

A wild guess is that somehow Grub is calling ExitBootServices, which
closes all Boot Services interfaces, before executing Xen. It should not
happen if Grub is executing Xen as EFI binary. I cannot explain how it
is possible. It looks like a bug in Grub.

Can you post the Grub config file that you are using?


Usually before Grub there is the proper EFI bootloader, tipically EDK2.
You should be able to boot Xen directly from the EFI bootloader too,
from its prompt, just by executing "xen". You need to place the xen
binary in the first FAT partition together with a xen.cfg config file as
follows:

---
options=console=com1 com1=115200 loglvl=all noreboot
kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
ramdisk=initrd-3.0.31-0.4-xen
---

options is to specify the Xen command line.
kernel is to specify the Linux kernel to use and its command line.
ramdisk is to specify the Linux ramdisk.

Both kernel and ramdisk needs to be on the FAT partition too.


That way you skip Grub and you might be able to skip any related
problems with ExitBootServices.