Re: [Xen-devel] [PATCH 5/5] x86/dom0: take alignment into account when populating p2m in PVH mode

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Thu, Dec 27, 2018 at 08:14:40PM +0000, Andrew Cooper wrote:
> On 27/12/2018 15:26, Roger Pau Monne wrote:
> > Current code that allocates memory and populates the p2m for PVH Dom0
> > doesn't take the address alignment into account, this can lead to high
> > order allocations that start on a non-aligned address to be broken
> > down into lower order entries on the p2m page tables.
> >
> > Fix this by taking into account the p2m page sizes and alignment
> > requirements when allocating the memory and populating the p2m.
> >
> > Reported-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
> > Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
> 
> I've committed patches 1-4 because they are trivial.
> 
> This patch does fix the alignment issue, but does so at a rather large cost.
> 
> The sequence (without any of the E820 series) is now:
> 
> (XEN) *** Building a PVH Dom0 ***
> (XEN) pvh_populate_memory_range(d0, 0, a0)
> (XEN) guest_physmap_add_page(d0, 00000, 100be1, 0 = 4kB)
> ...
> (XEN) guest_physmap_add_page(d0, 0009f, 100b42, 0 = 4kB)
> (XEN) pvh_populate_memory_range(d0, 100, 360)
> (XEN) guest_physmap_add_page(d0, 00100, 100b41, 0 = 4kB)
> ...
> (XEN) guest_physmap_add_page(d0, 001ff, 100a42, 0 = 4kB)
> (XEN) guest_physmap_add_page(d0, 00200, 100800, 9 = 2048kB)
> (XEN) guest_physmap_add_page(d0, 00400, 100a41, 0 = 4kB)
> ...
> (XEN) guest_physmap_add_page(d0, 0045f, 1007e2, 0 = 4kB)
> 
> 
> So overall, the 2M superpage is created in the middle, but all the 4k
> entries are made with a single call to guest_physmap_add_page() at a
> time, which is going to be very inefficient with the P2M lock.
> 
> Ideally, we should make a single order 8 allocation at 1M, then the
> order 9 at 2M.

Right, this new approach limit the allocation orders to the orders of
the page, super page or huge pages, thus preventing using the orders
in the middle.

> I think it might be better to try and crib from the HVM domainbuilder
> code, which copes with collecting together the head and tail of
> non-aligned allocations while still making an order > 0 requests where
> possible.

I find the HVM p2m domain builder code extremely difficult to follow,
and hence I would like to avoid introducing such complexity here.

I've reworked the patch so that all possible orders are used when
populating the p2m, let me post that version.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel