Re: [Xen-users] Questions on hvmloader, direct kernel boot and simulated BIOS

[George Dunlap <dunlapg@xxxxxxxxx>]

On Fri, Sep 9, 2016 at 10:54 PM, Brett Stahlman <brettstahlman@xxxxxxxxx> wrote:
> Hello,
> I have several questions on how xen handles booting HVM guests. Any
> answers, insights or links to pertinent documentation would be greatly
> appreciated...
>
> IIUC, leaving "kernel" and "ramdisk" options unset in your xl.cfg file
> means you do *not* want "direct kernel boot": rather, you want the
> boot to proceed using "simulated firmware": i.e., firmware code that
> xen loads into guest memory as a single, contiguous binary blob. As I
> understand it, xen actually loads the selected BIOS binary blob from
> disk to a specific memory location in the guest (usually 0xF0000) and
> then, after transitioning to real mode, jumps to the reset vector
> 0xFFFF0 (presumably in non-root "guest" mode) to initiate the boot
> process. I haven't been able to find much documentation on this, but
> I'm assuming that the simulated firmware entry point of 0xFFFF0
> ensures that the HVM guest will perform POST and do the other things
> the BIOS would normally do after reset, and that it will do all this
> in "guest" (non-root) mode. In addition to POST, I'm assuming this
> simulated BIOS code will attempt to load an MBR from one of the
> virtual disks specified with the "disk" option in xl.cfg (unless the
> BIOS selected is ovmf, in which case, I'm assuming the simulated ovmf
> firmware will be looking for an EFI System Partition and a suitable
> .efi file). Am I on the right track so far, or have I misunderstood
> something fundamental?
>
> One of the things that's confusing me is "hvmloader". I see that this
> is built as a standalone executable under "tools" in the xen source.
> Looking at the inline assembly at the top of hvmloader.c, I see that
> there's a call to hvmloader main, which contains a call to
> bios_load(), presumably to perform the aforementioned load of a
> firmware blob to 0xF0000. Following return from hvmloader main, the
> inline assembly transitions back to 16-bit real mode, and ultimately
> jumps to the reset vector (0xFFFF0), presumably to execute the
> BIOS/UEFI firmware blob loaded by hvmloader main. This makes sense,
> but I'm missing some of the overall context: for instance, on an Intel
> processor, will we be in non-root (guest) mode when hvmloader runs?
> Who starts hvmloader and how? I've seen code in
> tools/libxl/libxl_dom.c (specifically, the call to xc_dom_kernel_file
> in function libxl__domain_firmware), which appears to be loading it
> into memory, but from there it gets a bit fuzzy... Are hypercalls used
> to start up the guest DomU? Is hvmloader's _start label the
> entry-point for each non-direct kernel boot HVM guest?
>
> Also, how does the VMX entry/exit logic in
> xen/arch/x86/hvm/vmx/entry.S fit into the picture here? I'm assuming
> that code is running only on the VMM in root mode, and that it somehow
> includes a mechanism for switching between the various guests.

You're mostly on the right track.  A couple of points:

* When doing a direct boot, you don't start at a fixed location (as
you would on reset in real hardware); the domain builder running in
dom0 has to read the ELF data structures and find the appropriate
entry point, I believe in the right paging mode as well (i.e., not
real mode).

* hvmloader runs in guest (non-root) mode.  In current releases of Xen
it has SeaBIOS or OVMF baked into it (i.e., a single binary contains
both hvmloader and the BIOS).  We're working on changing this so that
the domain builder will load up both hvmloader and the appropriate
BIOS.

Also, FYI, this sort of question about the Xen internals is probably
better off on xen-devel (more people who know the answer reading it on
a regular basis)

 -George

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
https://lists.xen.org/xen-users