Re: [Xen-devel] [Bug] Intel RMRR support with upstream Qemu

[Alexey G <x1917x@xxxxxxxxx>]

Hi,

On Mon, 24 Jul 2017 10:53:16 +0100
Igor Druzhinin <igor.druzhinin@xxxxxxxxxx> wrote:
> > [Zhang, Xiong Y] Thanks for your suggestion.
> > Indeed, if I set mmi_hole >= 4G - RMRR_Base, this could fix my issue.
> > For this I still have two questions, could you help me ?
> > 1) If hvmloader do low memory relocation, hvmloader and qemu will see a
> > different guest memory layout . So qemu ram maybe overlop with mmio,
> > does xen have plan to fix this ? 
> 
> hvmloader doesn't do memory relocation - this ability is turned off by
> default. The reason for the issue is that libxl initially sets the size
> of lower MMIO hole (based on the RMRR regions present and their size)
> and doesn't communicate it to QEMU using 'max-ram-below-4g' argument.
> 
> When you set 'mmio_hole' size parameter you basically forces libxl to
> pass this argument to QEMU.
> 
> That means the proper fix would be to make libxl to pass this argument
> to QEMU in case there are RMRR regions present.

I tend to disagree a bit. 
What we lack actually is some way to perform a 'dynamical' physmem
relocation, when a guest domain is running already. Right now it works only
in the 'static' way - i.e. if memory layout was known for both QEMU and
hvmloader before starting a guest domain and with no means of arbitrarily
changing this layout at runtime when hvmloader runs.

But, the problem is that overall MMIO hole(s) requirements are not known
exactly at the time the HVM domain being created. Some PCI devices will be
emulated, some will be merely passed through and yet there will be some
RMRR ranges. libxl can't know all this stuff - some comes from the host,
some comes from DM. So actual MMIO requirements are known to hvmloader at
the PCI bus enumeration time.

libxl can be taught to retrieve all missing info from QEMU, but this way
will require to perform all grunt work of PCI BARs allocation in libxl
itself - in order to calculate the real MMIO hole(s) size, one needs to
take into account all PCI BARs sizes and their alignment requirements
diversity + existing gaps due to RMRR ranges... basically, libxl will
need to do most of hvmloader/pci.c's job.

My 2kop opinion here is that we don't need to move all PCI BAR allocation to
libxl, or invent some new QMP-interfaces, or introduce new hypercalls or
else. A simple and somewhat good solution would be to implement this missing
hvmloader <-> QEMU interface in the same manner how it is done in real
hardware.

When we move some part of guest memory in 4GB range to address space above
4GB via XENMEM_add_to_physmap, we basically perform what chipset's
'remap' (aka reclaim) does. So we can implement this interface between
hvmloader and QEMU via providing custom emulation for MCH's
remap/TOLUD/TOUUD stuff in QEMU if xen_enabled().

In this way hvmloader will calculate MMIO hole sizes as usual, relocate
some guest RAM above 4GB base and communicate this information to QEMU via
emulated host bridge registers -- so then QEMU will sync its memory layout
info to actual physmap's.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel