Re: [Xen-devel] [PATCH] Disallow setting maxmem to higher value than total physical memory size

[Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx>]

On Wed, 2010-09-01 at 19:15 +0100, Jeremy Fitzhardinge wrote:
> On 09/01/2010 10:56 AM, Ian Campbell wrote:
> > On Wed, 2010-09-01 at 17:53 +0100, Jeremy Fitzhardinge wrote:
> >> On 09/01/2010 05:44 AM, Ian Campbell wrote:
> >>> On Wed, 2010-09-01 at 13:31 +0100, Michal Novotny wrote:
> >>>> Hi,
> >>>> this is the patch to disallow changing the maxmem value to higher value 
> >>>> than total physical memory size since without this patch I was able to 
> >>>> set dom0 maxmem to higher (invalid) value which is not correct.
> >>> I think it is allowable for a domU though. Consider the scenario where
> >>> you have two hosts, one of which has more physical RAM than the other.
> >>> You may which to boot a domain on the smaller host, (i.e. booting
> >>> ballooned with a current_pages suitable for the small host) and then
> >>> migrate it to the large machine where you then want to be able to
> >>> balloon to a value larger than was even possible on the previous
> >>> machine.
> >> But max-mem can change between hosts; on the small host it needn't have
> >> a maxmem larger than the host's memory.  (The domain itself may have a
> >> larger notion of maxmem internally, but that's separate.)
> > It's not separate, this guest configuration item precisely informs the
> > guest how large it can expect it's memory map to ever need to be (the
> > setting is also called the static-max in both xend and xapi).
> 
> No, that is separate.  There are three values:
> 
>    1. the domain's initial memory allocation
>    2. the max size the domain will ever grow to
>    3. the max size Xen will allow the domain to grow to right now
> 
> At domain build time, the domain needs to know 1 and 2, but 3 is
> irrelevant (so long it is larger than 1). 
> 
> 2 can be arbitrarily large.  The domain may not need to know about 2 at
> all if it can dynamically add memory at runtime via, say, memory
> hotplug.  It only matters if it needs to statically allocate space in
> its memory map for future balloonings.

Yes, so it only matters for all PV Linux guests which currently support
ballooning up from their initial size...

> There's no need for 3 to ever be larger than the host's physical memory,
> and it can be changed at will (if the host memory size changes due to
> hotplug memory, save/restore or migrate).
> 
> AFAICT, "static-max" is completely useless, at least for PV Linux
> domains, because the kernel needs to get that value when its
> constructing its basic memory mappings, way before it has any chance to
> talk to xenstore.

It's not only in xenstore, it is also in the result of the
XENMEM_get_memory_map hypercall.

Old-style PV Linux domains really do use static-max in this way. It may
well be that there are potentially better ways for guests to implement
this in the future but we have a deployed base of guests which work this
way.

> > For a PV guest the value is pushed down into the hypervisor by the
> > toolstack via XENMEM_set_memory_map and this controls the memory map
> > returned to the guest from XENMEM_get_memory_map, which in turn informs
> > the guest's choice of maxmem value (for PV guests which can boot
> > ballooned that is).
> 
> OK, I need to investigate that.  But the maxmem size for the domain
> should be something derived from the domain's config file,

I thought we were talking about clamping the value from the domain's
config file, weren't we? At least that's what I thought the proposed
patch would end up doing, which is why I was objecting.

It may be that xend uses the same variable names in various structures
(and/or otherwise conflates the two concepts of maxmem) and we are
talking at cross purposes here but from what I can tell
xc_domain_set_memmap_limit() is called from
X86_Linux_ImageHandler::buildDomain with the result of
XendDomainInfo::getMemoryMaximum which I think is the same value as is
stored by XendDomainInfo::setMemoryMaximum (one of the functions being
patched here). This field ultimately corresponds to the maxmem field in
the domain configuration file, although when it is translated from cfg
file to XendDomainInfo it appears to be set without using the setter
method so maybe the patch only has an impact in some obscure set of
circumstances. With xend being the twisty maze that it is it is hard to
be sure what's going on.

I don't think it has been shown that the two maxmems are completely
disconnected in xend's mind or what the impact this change is on the
maxmem which is exposed to guests.

It's also not entirely clear what actual problem this patch is solving
(apart from allusions to something being wrong in "xm list -v") but it
seems to me that if it is clamping an invalid value for domain 0 into
something value then surely that value came from _somewhere_ and that is
the place which really needs fixing.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel