Semantics of XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

Hello,

A recent ABI change in Xen caused a total breakage under the Xapi
toolstack, and the investigation had lead to this.

First of all, the memory pool really needs renaming, because (not naming
names) multiple developers were fooled into thinking that the bug was
caused by a VM being unexpectedly in shadow mode.

Second, any MB value >= 0x1000000 will truncate to 0 between
{hap,shadow}_domctl() and {hap,shadow}_set_allocation().


But for the main issue, passing 0 in at the hypercall level is broken.

hap_enable() forces a minimum of 256 pages.  A subsequent hypercall
trying to set 0 frees {tot 245, free 244, p2m 11} all the way down to
{tot 1, free 0, p2m 11} before failing with -ENOMEM because there are no
more free pages to free.  Getting -ENOMEM from this kind of operation
isn't really correct.


Passing 0 cannot possibly function.  There are non-zero p2m frames by
the time createdomain completes, as we allocate the top of the p2m, and
they cannot be freed without the teardown logic which releases memory in
the correct order.

In fact, passing anything lower than the current free size is guaranteed
to fail.  Continuations also mean that you can't pick a value which is
guaranteed not to fail, because even a well (poorly?) placed foreign map
in dom0 could change the properties of the pool.


The shadow side rejects hypercall attempts using 0 (but can be bypassed
with the above truncation bug), and will try to drop shadows to get down
to the limit.  This represents a difference vs HAP, and in practice 1M
granularity is probably enough to ensure that you can't fail to set the
shadow allocation that low.  There is also a reachable BUG() somewhere
in this path as reported several times on xen-devel, but I still haven't
figured out how to tickle it.


There is no code or working usecase for reducing the size of the shadow
pool, except on domain destruction.  I think we should prohibit the
ability to shrink the shadow pool, and defer most of this mess to anyone
who turns up with a plausible usecase.

~Andrew