[Xen-devel] [PATCH 0/7] x86: improve handling of sparse physical memory maps

["Jan Beulich" <JBeulich@xxxxxxxxxx>]

Significant holes in the physical memory map weren't handled gracefully
so far in Xen: The overhead of setting up a fully linear 1:1 mapping as
well as a fully linear frame table could so high that Xen wouldn't boot
on extreme systems.

The following adjustments are being made in each patch of this series:

1) For x86-64, the virtual range reserved for the 1:1 mapping gets
extended to cover 5Tb, and the virtual size of the frame table gets
changed to match whatever the 1:1 table can cover.

2) The largest range of contiguous bits which are zero for all valid
MFNs gets removed from the MFN representation used to index into the
1:1 mapping and frame table, thereby cutting the virtual range these
tables must cover approximately by half with each bit removed.

3) The reduced in size frame table gets mapped sparsely, i.e. holes of
MFN ranges spanning a sufficiently large range will no longer have
struct page_info instances allocated for them.

4) The non-reducible (in virtual size) M2P table gets allocated
sparsely in the same manner (just requiring larger holes for a table
chunk to be omitted).

This ends the portion of the patch series that deal with overhead
reduction. On a partner system with suitable memory layout, this patch
series has been shown to reduce Xen memory overhead from 14.2% of total
memory to 1.8%, thus getting reasonably close to the theoretically
minimal overhead of 1.3% (counting just frame table, 1:1 mapping, and
M->P tables).

Since the more complex calculations resulted in not insignificant code
bloat, the following non-essential patches represent possible attempts
to reduce the code size overhead, and should be considered RFC:

5) Introduce a variant of map_domain_page() directly getting passed a
struct page_info * argument, based on the observation that in many
places the argument to this function so far simply was the result of
page_to_mfn(). This is meaningful for the x86-64 case where
map_domain_page() really just is an invocation of mfn_to_virt(), and
hence the combined mfn_to_virt(page_to_mfn()) now represents a
needless roundtrip conversion compressed -> uncompressed ->
compressed of the MFN representation.

6) The grant table code had some particularly frequent repetitions of
mfn_to_page() on each time the same input arguments. To help the
compiler (which can do only a limited job on CSE), this adds explicit
caching of the transformation result in a few places.

7) This one is likely the most questionable part: Realizing that the
compiler can do a much better job in terms of CSE when the variables
needed are constant, this introduces and extension to the read-mostly
variables, allowing variables to be marked as being written just
during boot initialization.

In the event of transmission problems with the patches (our IS&T
folks, while now being aware of the patch corruption issues seen in
the past, refuse to address the root cause of the problem for the
time being), the full set of patches is attached here.

Signed-off-by: Jan Beulich <jbeulich@xxxxxxxxxx>

Attachment: xen-x86_64-5Tb.patch
Description: Text document

Attachment: x86_64-fold-discontig-mem.patch
Description: Text document

Attachment: xen-x86-sparse-frame-table.patch
Description: Text document

Attachment: xen-x86-sparse-m2p-table.patch
Description: Text document

Attachment: xen-map-domain-page-page-param.patch
Description: Text document

Attachment: xen-fold-discontig-mem-CSE.patch
Description: Text document

Attachment: xen-write-once.patch
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel