Re: [Xen-devel] [RFC] OVMF on PVH

[Anthony PERARD <anthony.perard@xxxxxxxxxx>]

On Thu, Aug 02, 2018 at 06:21:31PM +0200, Roger Pau Monné wrote:
> On Thu, Aug 02, 2018 at 12:24:35PM +0100, Anthony PERARD wrote:
> > ## New binary, separated from QEMU/KVM one.
> > 
> > Right now, the rules to build the firmware are located in
> > `OvmfPkg/OvmfPkgX64.dsc`, a platform. I've created a new platform, without
> > KVM support, and would like to retire the Xen support from the current
> > platform.  For now, I have created OvmfPkg/XenOvmf.dsc. (The change to a
> > new platform had been proposed by upstream.)
> 
> Is it my understanding that after this there will be a generic
> platform (for QEMU, KVM and everything else) and a platform for Xen?

It's not really generic. As far as I can tell, it's only QEMU/KVM (and
Xen now). But yes, there would be a separate platform (or file when
built) for Xen.

It would be possible to keep Xen support in the current "generic"
platform, but that would mean that code would be duplicated.

I think the OVMF maintainer would be happier if the Xen support was
separated, even now without PVH.

> > That would simplify Xen support in OVMF, and allow to diverge more from
> > OVMF where needed. That would mostly be useful during the initial boot
> > phase where there lots of `if xen do that, else do that`.
> 
> I fear this might be dangerous because most developers will only test
> against the generic platform, so breakage could be introduced more
> easily to the Xen platform without upstream noticing?

That would not change anything. Upstream maintainers doesn't test booting
OVMF in a Xen guest so there are sometime where only Xen is broken, even
if they try hard to not break it.

As for build, I actually don't know if that would change anything.
Anyway, we've got osstest.

> > We can also choose some drivers that can work on both PVH and HVM, for
> > example, use a LAPIC timer instead of ACPI Timer.
> > 
> > All this mean that building OVMF for Xen would need to be change.
> 
> And distros will likely have to provide two different packages, or a
> single package with the generic binary and the Xen specific one.

Yes. But it is already an issue sometime. For example, I cann't use the
ovmf package from Arch Linux because it is cut in two part (vars and
code), and Xen only works when both part are together (/bin/cat vars
code).

> > 
> > ## ELF header
> > 
> > Adding an ELF header to OVMF so the blob can be loaded by libxl/libxc
> > without modifying those libs.
> > 
> > That replace a section of the OVMF binary called VarStore by that ELF
> > header.  It's empty and libxl doesn't know how to use it. (QEMU/KVM would
> > have a flash device loading the store, so it can be written to.)
> 
> Can't you just add the header without replacing anything? The ELF
> header is quite small, so I don't think size should be a problem (if
> that's why it's placed on top of the VarStore).

I don't think hvmloader can cope with a different size right now. And
that section doesn't contain anything useful, it would need to be
separated in order to be useful.

> > With the ELF header, I can test OVMF by simply adding this in the config
> > file:
> > 
> >     type='pvh'
> >     kernel='/path/to/OVMF.fd'
> 
> Yes, that's the expectation and is what firmware='ovmf|uefi' should
> do behind the scenes.
> 
> > We could use a modified `hvmloader` to load OVMF on PVH or teach libxc to
> > load it at a hardcoded location, but I don't see it as necessary, and if
> > we have one less firmware to load, it's probably better.
> > 
> > With the use of an ELF header comes another issue, which as to do with
> > were the binary needs to be loaded initially.
> > 
> > ## Loading binary at 0x10000 (1MB, like hvmloader on HVM)
> 
> That works well for hvmloader because after it has finished running
> the memory can be overwritten without issues. The same is not true for
> ovmf which needs to keep data around so that boot services and runtime
> services can continue working.
> 
> > Right now, the OVMF blob is loaded (by hvmloader) just below 4GB, with
> > hvmloader moving some RAM around to allow that.  But with the use of an
> > ELF header, and let libxc load the blob, it was not possible to load the
> > blob just below 4GB. (Or at least without modification of the toolstack I
> > guess.)
> 
> libxc/libelf will load the binary in the position specified by the ELF
> program headers PhysAddr field (also called LMA). If you execute
> `readelf -l <ovmf binary>` you should see the program headers and the
> address where they should be loaded.
> 
> You can easily modify the LMA in the linker script using the AT
> keyword.

I don't know what to answer to that. OVMF isn't an ELF. There isn't an
easy way to tell where it should be loaded. There is no linker script
(at least that link everything together), but a collection python script
and some program, and configuration. I don't need to run `readelf`
because it either doesn't work or it would print the value that I've set
manually.

The issue it that, if I set LMA to 0xffe00000, libxc fails.

To reuse names from elf.h, what my header would be if I try to load ovmf
where it is currently (just bellow 4G), that would be (with only
addr/size, not the flags):
Elf32_Ehdr hdr = {
.e_entry = 0xffffffd0,
};
Elf32_Phdr phdr = {
.p_paddr = 0xffe00000,
.p_vaddr = 0xffe00000;
.p_filesz = 0x00200000,
.p_memsz = 0x00200000,
};

> > So I've attempted to have OVMF been started from a different place
> > in memory. Then I had to hack a bit more in order to be able to start OVMF
> > from two different location (the current one for HVM guest, and a new one
> > that can be used for PVH). That works fine, I'll just have to find out
> > what upstream think about that.
> 
> I think the address where ovmf is loaded should be the same for HVM or
> PVH if possible. We know that loading at this address works for HVM,
> so I think we should keep it for PVH, it's less likely to cause issues
> in the future if both are kept in sync.

Indeed, but that doesn't work well, and I find it strange that we have
to move RAM around for something that is only use very early and never
used again.

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel