Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI boot

[Luca Fancellu <luca.fancellu@xxxxxxx>]

> On 16 Sep 2021, at 21:16, Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote:
> 
> On Thu, 16 Sep 2021, Jan Beulich wrote:
>> On 16.09.2021 17:07, Luca Fancellu wrote:
>>> I explain here my understanding on dom0less, this feature is used to start 
>>> domUs at
>>> Xen boot in parallel, the name is misleading but it doesn’t require dom0 to 
>>> be absent.
>>> 
>>> So if you have a dom0 kernel embed in the image, it's completely fine to 
>>> start it and it 
>>> doesn’t have to be skipped.
>>> 
>>> Here the possible user cases:
>>> 1) start only dom0 [dom0 modules on xen.cfg or embedded in Xen image]
>>> 2) start only domUs, true dom0less [no dom0 modules on xen.cfg or embedded 
>>> in Xen image, domUs on DT]
>>> 3) start dom0 and domUs [(dom0 modules on xen.cfg or embedded in Xen image) 
>>> and domUs on DT]
>> 
>> If that's the intention - fine. Stefano?
> 

Hi Stefano,

> What do you mean by dom0 modules embedded in the Xen image? I am not
> familiar with it, but I imagine it doesn't involve any multiboot,module
> nodes in device tree, right?
> 
> Putting aside "dom0 modules embedded in Xen image" that I didn't fully
> understand, there are three ways to load Dom0:
> 
> - dom0 kernel (and ramdisk, optional) on xen.cfg
> - dom0 kernel (and ramdisk, optional) on device tree using the "reg" property
> - dom0 kernel (and ramdisk, optional) on device tree using the "uefi,binary" 
> property

True for the #1 and #2, the last one is not implemented. The uefi,binary 
property
for now is only used to load domU modules.

> 
> Then, the other use cases are:
> 
> - true dom0less, domUs on device tree using the "reg" property
> - true dom0less, domUs on device tree using the "uefi,binary" property
> 
> And of course all the possible combinations between Dom0 and DomU
> loading.
> 
> 
> Currently, patch #1 checks for the presence of a Dom0 kernel node and, if
> present, it decides not to proceed with xen.cfg. So if the Dom0 kernel
> node is *not* present, efi_arch_use_config_file returns true.
> 
> However, this could be a true dom0less configuration without any Dom0
> kernel. If so, you might not want to load xen.cfg at all because it is
> not needed. In a true dom0less configuration, we probably want
> efi_arch_use_config_file to return false.

In a true dom0less configuration we might need to read xen.cfg to retrieve the
Xen command line, but following the actual implementation of the common code
there is more. I’m going to explain.

What efi_arch_use_config_file really does is not only choosing to read xen.cfg
or not. Following the common code (xen/common/efi/boot.c) and what its result 
activate
along the path, it basically decides if the UEFI stub is used as a loader from 
filesystem
or not. We need the UEFI stub as a loader to be 100% UEFI and load our modules.

The original check basically says “if there are multiboot,module in the DT, 
then some
bootloader has loaded in memory the required modules so I’m not gonna load 
anything
from the filesystem because I assume it puts everything in place for me to 
boot.”

>From misc/efi.txt:
When booted as an EFI application, Xen requires a configuration file as 
described below unless a bootloader,
such as GRUB, has loaded the modules and describes them in the device tree 
provided to Xen. If a bootloader
provides a device tree containing modules then any configuration files are 
ignored, and the bootloader is
responsible for populating all relevant device tree nodes.

What I’m doing in patch #1 is restricting that check to just the 
multiboot,module that are
Dom0 module, why? Because with the introduction of dom0less we need to specify
multiboot,modules for domUs, but the presence or not of dom0 modules is the only
Information we need to understand if the user decided to start Xen with 
everything
in places (modules in memory, xen command line, dtb) or if the job is demanded 
to the
UEFI stub and its configuration file.

By the configuration file you can also load in memory the Xen dtb, so Xen can
be started as an EFI application without the DTB and then load it using the EFI 
stub.

I’m not against this new property “xen,cfg-loading”, I just think it is not 
needed because
we have all the information we need without it and in any case we need to read 
the
configuration file because otherwise we won’t have access to the Xen command 
line.

Now I’m going to show you examples of all use cases that are valid with the 
introduction
of this serie:

1) Start Xen as EFI application and load only Dom0

    Xen.cfg:
    [global]
    default=xen_dom0

    [xen_dom0]
    options=<Xen command line>
    kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
    ramdisk=initrd-3.0.31-0.4-xen
    dtb=<xen DTB>

    DT:
    {no modification}

2) Start Xen as EFI application to load a true dom0less setup

    Xen.cfg:
    [global]
    default=xen_true_dom0less

    [xen_true_dom0less]
    options=<Xen command line>
    dtb=<xen DTB>

    DT:
    chosen {
        #size-cells = <0x1>;
        #address-cells = <0x1>;

        domU1 {
            #size-cells = <0x1>; 
            #address-cells = <0x1>;
            compatible = "xen,domain”;
            cpus = <1>;
            memory = <0 0xC0000>;
           
            module@1 {
                compatible = "multiboot,kernel", "multiboot,module”;
                bootargs = "console=ttyAMA0 root=/dev/ram0 rw”;
                uefi,binary = “domU_kernel1.bin"
            };

            module@2 {
                compatible = “multiboot,ramdisk", "multiboot,module”;
                uefi,binary = “domU_ramdisk1.bin"
            };

            module@3 {
                compatible = "multiboot,device-tree", "multiboot,module”;
                uefi,binary = “domU_passthrough1.dtb"
            }; 
        };
        
        domU2 { <as above> };
    }

3) Start Xen as EFI application to load Dom0 and DomUs

    Xen.cfg:
    [global]
    default=xen_dom0_domUs

    [xen_dom0_domUs]
    options=<Xen command line>
    kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
    ramdisk=initrd-3.0.31-0.4-xen
    dtb=<xen DTB>

    DT:
    chosen {
        #size-cells = <0x1>;
        #address-cells = <0x1>;

        domU1 {
            #size-cells = <0x1>; 
            #address-cells = <0x1>;
            compatible = "xen,domain”;
            cpus = <1>;
            memory = <0 0xC0000>;
           
            module@1 {
                compatible = "multiboot,kernel", "multiboot,module”;
                bootargs = "console=ttyAMA0 root=/dev/ram0 rw”;
                uefi,binary = “domU_kernel1.bin"
            };

            module@2 {
                compatible = “multiboot,ramdisk", "multiboot,module”;
                uefi,binary = “domU_ramdisk1.bin"
            };

            module@3 {
                compatible = "multiboot,device-tree", "multiboot,module”;
                uefi,binary = “domU_passthrough1.dtb"
            }; 
        };
        
        domU2 { <as above> };
    }

So as you see every case is covered without the introduction of the
property.

Please let me know what do you think.

Cheers,
Luca

> 
> 
> In general, loading xen.cfg or not loading xen.cfg doesn't seem like it
> should be dependent on whether there is or there is not a dom0 kernel
> node in device tree. I think there are too many possible combinations to
> guess correctly based on the presence of the dom0 kernel node. Instead,
> I think it would be better to have an explicit flag.
> 
> Today, if a "multiboot,module" node is present, efi_arch_use_config_file
> returns false. So I suggested to extend it so that if a
> "multiboot,module" node is present we look into a specific
> xen.cfg-loading property and if present return true otherwise false.
> This way, we are backward compatible.