Re: [Xen-devel] [Xen 4.12 bug] HVM/PVH boot confusion

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

Adding Anthony who likely knows more about all this since it's closely
related to QEMU.

On Tue, Jan 29, 2019 at 07:49:51PM +0000, Andrew Cooper wrote:
> Hello,
> 
> Given the following vm.cfg file:
> 
> name="vm"
> type="hvm"
> 
> vcpus=4
> memory=1024
> 
> firmware_override="/root/xen-syms"
> 
> kernel="/boot/vmlinuz-4.4-xen"
> ramdisk="/boot/initrd-4.4.0+10.img"

I know very little about direct kernel booting with HVM, but I assume
using firmware_override gets rid of hvmloader and the BIOS/UEFI
payload plus the option rom to direct boot into a Linux kernel?

So basically the module list is `mod[0] = kernel` and `mod[1] =
ramdisk`?

> 
> cmdline="console=xen,pv dom0=pv --- earlyprintk=xen"
> 
> Xen crashes with the following trace:
> 
> (d15) (XEN) Xen BUG at pvh-boot.c:82
> (d15) (XEN) ----[ Xen-4.12.0-rc  x86_64  debug=y   Not tainted ]----
> (d15) (XEN) CPU:    0
> (d15) (XEN) RIP:    e008:[<ffff82d0804331f2>] pvh_init+0x27d/0x2fe
> <snip>
> (d15) (XEN) Xen call trace:
> (d15) (XEN)    [<ffff82d0804331f2>] pvh_init+0x27d/0x2fe
> (d15) (XEN)    [<ffff82d080429000>] __start_xen+0x14c/0x28f6
> (d15) (XEN)    [<ffff82d0802000f3>] __high_start+0x53/0x55
> (d15) (XEN)
> (d15) (XEN)
> (d15) (XEN) ****************************************
> (d15) (XEN) Panic on CPU 0:
> (d15) (XEN) Xen BUG at pvh-boot.c:82
> (d15) (XEN) ****************************************
> 
> The problem is that Xen is started at its PVH entrypoint (contrary to
> the instructions in the vm config file), and Xen unconditionally expects
> RSDP to be passed.
> 
> There are at least two bugs here.
> 
> 1) RSDP was a late addition to the PVH boot protocol.  Xen's PVH
> entrypoint must not mandate its existence, because there are releases of
> the domain builder which don't provide it.

Right, I think it should be fine to attempt to boot without a rsdp
hint, Xen can always resort to scanning the low 1MB in order to
attempt to find the rsdp.

The problem here would be that the toolstack is not going to create
any ACPI tables because it's a HVM guest and thus the toolstack
expects the firmware to create such tables (hvmloader). Since in the
above example you skip the firmware, you won't get any ACPI tables at
all.

> 2) The HVM/PVH boot confusion.  This think this is a still-outstanding
> bug around the broken assumption that the hvmloader binary speaks the
> PVH protocol without advertising itself appropriately (I really regret
> not objecting to those patches before they went in).  At the least, that
> needs fixing by putting a proper ELF note in hvmloader, and the domain
> builder needs to be updated to build all PVH-boot-ABI images consistently.

Hm, booting hvmloader using the hvm_start_info has always been kind of
a hack, that relied on the toolstack being in full control of the
firmware and the user not playing with it.

I think part of the problem here also comes from the fact that the
loader list (xc_dom_loader *first_loader) is shared between
all the guest types, thus allowing a guest in a hvm container to be
booted using the pvh entry point.

I think a proper fix to this mess involves the HVM and the PVH domain
creation paths being exactly the same, ie: ACPI tables always created
by the toolstack for example. The only difference between a PVH and a
HVM guests from the toolstack PoV should be the emulated devices that
it gets.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel