Re: [Xen-devel] [Bug] Intel RMRR support with upstream Qemu

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 24/07/17 17:42, Alexey G wrote:
> Hi,
>
> On Mon, 24 Jul 2017 10:53:16 +0100
> Igor Druzhinin <igor.druzhinin@xxxxxxxxxx> wrote:
>>> [Zhang, Xiong Y] Thanks for your suggestion.
>>> Indeed, if I set mmi_hole >= 4G - RMRR_Base, this could fix my issue.
>>> For this I still have two questions, could you help me ?
>>> 1) If hvmloader do low memory relocation, hvmloader and qemu will see a
>>> different guest memory layout . So qemu ram maybe overlop with mmio,
>>> does xen have plan to fix this ? 
>> hvmloader doesn't do memory relocation - this ability is turned off by
>> default. The reason for the issue is that libxl initially sets the size
>> of lower MMIO hole (based on the RMRR regions present and their size)
>> and doesn't communicate it to QEMU using 'max-ram-below-4g' argument.
>>
>> When you set 'mmio_hole' size parameter you basically forces libxl to
>> pass this argument to QEMU.
>>
>> That means the proper fix would be to make libxl to pass this argument
>> to QEMU in case there are RMRR regions present.
> I tend to disagree a bit. 
> What we lack actually is some way to perform a 'dynamical' physmem
> relocation, when a guest domain is running already. Right now it works only
> in the 'static' way - i.e. if memory layout was known for both QEMU and
> hvmloader before starting a guest domain and with no means of arbitrarily
> changing this layout at runtime when hvmloader runs.
>
> But, the problem is that overall MMIO hole(s) requirements are not known
> exactly at the time the HVM domain being created. Some PCI devices will be
> emulated, some will be merely passed through and yet there will be some
> RMRR ranges. libxl can't know all this stuff - some comes from the host,
> some comes from DM. So actual MMIO requirements are known to hvmloader at
> the PCI bus enumeration time.
>
> libxl can be taught to retrieve all missing info from QEMU, but this way
> will require to perform all grunt work of PCI BARs allocation in libxl
> itself - in order to calculate the real MMIO hole(s) size, one needs to
> take into account all PCI BARs sizes and their alignment requirements
> diversity + existing gaps due to RMRR ranges... basically, libxl will
> need to do most of hvmloader/pci.c's job.
>
> My 2kop opinion here is that we don't need to move all PCI BAR allocation to
> libxl, or invent some new QMP-interfaces, or introduce new hypercalls or
> else. A simple and somewhat good solution would be to implement this missing
> hvmloader <-> QEMU interface in the same manner how it is done in real
> hardware.
>
> When we move some part of guest memory in 4GB range to address space above
> 4GB via XENMEM_add_to_physmap, we basically perform what chipset's
> 'remap' (aka reclaim) does. So we can implement this interface between
> hvmloader and QEMU via providing custom emulation for MCH's
> remap/TOLUD/TOUUD stuff in QEMU if xen_enabled().
>
> In this way hvmloader will calculate MMIO hole sizes as usual, relocate
> some guest RAM above 4GB base and communicate this information to QEMU via
> emulated host bridge registers -- so then QEMU will sync its memory layout
> info to actual physmap's.

Qemu isn't the only entity which needs to know.  There is currently an
attack surface via Xen by virtue of the fact that any hole in the p2m
gets emulated and forwarded to qemu.  (Two problems caused by this are a
qemu segfault and qemu infinite loop.)

The solution is to have Xen know which gfn ranges are supposed to be
MMIO, and terminate the access early if the guest frame falls outside of
the MMIO range.

Doing this by working it out statically at domain creation time is far
more simple for all components involved.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel