[PATCH 0/2] templates/linux_xen: Properly handle multiple initrd files

[Mauricio Faria de Oliveira <mfo@xxxxxxxxxxxxx>]

Adding the xen-devel list as requested in grub-devel [0]:

On Thu, Aug 11, 2022 at 1:51 PM Daniel Kiper <dkiper@xxxxxxxxxxxx> wrote:
> In general patches LGTM. However, I would prefer to hear an opinion from Xen
> folks too. So, please repost the patch set and add 
> xen-devel@xxxxxxxxxxxxxxxxxxxx
> to the list of recipients.

[0] https://lists.gnu.org/archive/html/grub-devel/2022-08/msg00115.html

Thanks,
Mauricio

...

The linux_xen template seems to be broken for multiple initrd files.
Linux fails to boot when it needs a real initrd but early/microcode
initrd(s) are found by grub-mkconfig.

Patch 1 allows initrd(s) other than the first early/microcode initrd
to be loaded at all, fixing an implementation error for multiboot[2].

Patch 2 allows Linux to get the real initrd as initrd (vs. an early/
microcode-only initrd), thus being able to boot if it needs a initrd.

Synthetic tests:
---

    # touch /boot/xen /boot/microcode.cpio

Original:

    # grub-mkconfig 2>/dev/null | grep -P '^\t(multiboot|module)'
            multiboot       /boot/xen ...
            module  /boot/vmlinuz-5.4.0-122-generic ...
            module  --nounzip   /boot/microcode.cpio 
/boot/initrd.img-5.4.0-122-generic

Patch 1:

    # grub-mkconfig 2>/dev/null | grep -P '^\t(multiboot|module)'
            multiboot      /boot/xen ...
            module  /boot/vmlinuz-5.4.0-122-generic ...
            module  --nounzip   /boot/microcode.cpio
            module  --nounzip   /boot/initrd.img-5.4.0-122-generic

Patch 2:

    # grub-mkconfig 2>/dev/null | grep -P '^\t(multiboot|module)'
            multiboot      /boot/xen ...
            module  /boot/vmlinuz-5.4.0-122-generic ...
            module  --nounzip   /boot/initrd.img-5.4.0-122-generic
            module  --nounzip   /boot/microcode.cpio

    With fake multiboot2 header:

    # echo -en '\xd6\x50\x52\xe8\x00\x00\x00\x00' > /boot/xen
    # echo -en '\x00\x00\x00\x00\x2a\xaf\xad\x17' >> /boot/xen
    # echo -en '\x00\x00\x00\x00\x00\x00\x00\x00' >> /boot/xen
    # echo -en '\x00\x00\x00\x00\x00\x00\x00\x00' >> /boot/xen

    # grub-file --is-x86-multiboot2 /boot/xen; echo $?
    0
    
    # grub-mkconfig 2>/dev/null | grep -P '^\t(multiboot|module)'
            multiboot2      /boot/xen ...
            module2 /boot/vmlinuz-5.4.0-122-generic ...
            module2 --nounzip   /boot/initrd.img-5.4.0-122-generic
            module2 --nounzip   /boot/microcode.cpio

Regular tests:
---

Debug patch:

    __start_xen() @ xen.git:xen/arch/x86/setup.c
    ...
    +for (i = 0; i < mbi->mods_count; i++)
    +   printk("MODULE %d size %d string %s\n",
    +          i, mod[i].mod_end - mod[i].mod_start, 
(char*)__va(mod[i].string));
    +
     bitmap_fill(module_map, mbi->mods_count);
     __clear_bit(0, module_map); /* Dom0 kernel is always first */

For /boot/microcode.cpio:

    $ sudo apt install microcode-initrd # on Ubuntu

    $ du --bytes /boot/...
    13660416 /boot/vmlinuz-5.4.0-122-generic
    33062542 /boot/initrd.img-5.4.0-122-generic
    5045248  /boot/microcode.cpio

Original:

    (XEN) MODULE 0 size 13660416 string placeholder 
root=UUID=74dd579c-a377-487d-b8f7-bc7c6df13ba1 ro console=ttyS0
    (XEN) MODULE 1 size 5045248 string /boot/initrd.img-5.4.0-122-generic
    ...
    [    2.505207] Kernel panic - not syncing: VFS: Unable to mount root fs on 
unknown-block(0,0)

Patch 1:

    (XEN) MODULE 0 size 13660416 string placeholder 
root=UUID=74dd579c-a377-487d-b8f7-bc7c6df13ba1 ro console=ttyS0
    (XEN) MODULE 1 size 5045248 string
    (XEN) MODULE 2 size 33062542 string
    ...
    [    1.890498] Freeing initrd memory: 4928K
    ...
    [    2.710948] Kernel panic - not syncing: VFS: Unable to mount root fs on 
unknown-block(0,0)

Patch 2:

    (XEN) MODULE 0 size 13660416 string placeholder 
root=UUID=74dd579c-a377-487d-b8f7-bc7c6df13ba1 ro console=ttyS0
    (XEN) MODULE 1 size 33062542 string
    (XEN) MODULE 2 size 5045248 string
    ...
    [    1.968578] Freeing initrd memory: 32288K
    ...
    [    2.844889] Run /init as init process
    [    2.916532] systemd-udevd[148]: Starting version 245.4-4ubuntu3.17

Xen code:
---

Key code path for this patchset:

    @ xen.git:xen/arch/x86/setup.c:

    void __init noreturn __start_xen(unsigned long mbi_p)
    {
    ...
        unsigned int initrdidx, ...
        multiboot_info_t *mbi;
        module_t *mod;
        unsigned long ..., module_map[1];
    ...
        mbi = __va(mbi_p);
    ...
        mod = __va(mbi->mods_addr);
    ...
        bitmap_fill(module_map, mbi->mods_count);
        __clear_bit(0, module_map); /* Dom0 kernel is always first */
    ...
        microcode_grab_module(module_map, mbi);
           # ucode=number can __test_and_clear_bit(number, module_map),
           # ucode=scan can't.
    ...
        initrdidx = find_first_bit(module_map, mbi->mods_count);
        if ( bitmap_weight(module_map, mbi->mods_count) > 1 )
        printk(XENLOG_WARNING
               "Multiple initrd candidates, picking module #%u\n",
               initrdidx);
    ...
        /*
         * ... The second module, if present, is an initrd ramdisk.
         */
        dom0 = create_dom0(mod, modules_headroom,
                           initrdidx < mbi->mods_count ? mod + initrdidx : NULL,
                           kextra, loader);
    ...

Mauricio Faria de Oliveira (2):
  templates/linux_xen: Properly load multiple initrd files
  templates/linux_xen: Properly order the multiple initrd files

 util/grub.d/20_linux_xen.in | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

-- 
2.34.1