Re: [Xen-devel] [PATCH RFC] libxl: set 1GB MMIO hole for PVH

[Juergen Gross <jgross@xxxxxxxx>]

On 10/05/18 10:33, Roger Pau Monné wrote:
> On Wed, May 09, 2018 at 06:12:28PM +0200, Juergen Gross wrote:
>> On 09/05/18 18:07, Roger Pau Monne wrote:
>>> This prevents page-shattering, by being able to populate the RAM
>>> regions below 4GB using 1GB pages, provided the guest memory size is
>>> set to a multiple of a GB.
>>>
>>> Note that there are some special and ACPI pages in the MMIO hole that
>>> will be populated using smaller order pages, but those shouldn't be
>>> accessed as often as RAM regions.
>>
>> Would it be possible somehow to put a potential firmware into that
>> 1GB region, too, if it needs any memory in high memory? Seabios e.g.
>> is taking the last RAM page of the guest for its hypercall page, which
>> will again shatter GB mappings.
> 
> I know this comment is related to HVM guests, but I'm not sure I see
> how setting the hypercall page shatters GB mappings. Setting the
> hypercall page doesn't involve changing any p2m mappings, but just
> filling a guest RAM page with some data.

The problem is that any memory reserved by firmware will be added as
"Reserved" in the E820 map. This will in turn result to the OS mapping
it read only or not at all, so it can't use a GB mapping even for the
physical memory mapping any longer. Seabios e.g. is using the last
memory page of the guest below 4GB for that purpose. Linux tends to
put memory management structure accessed very often (e.g. struct page
or numa node data) at the end of a memory region, so performance is
degraded. With memory management intensive workloads I've seen
performance going up about 3% in a HVM guest with a small Xen patch
adding a single additional page to the guest which was used by Seabios
for the hypercall page, resulting in the guest using a GB mapping for
the last GB of its memory.


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel