[Xen-devel] Ballooning up

[Jeremy Fitzhardinge <jeremy@xxxxxxxx>]

 I finally got around to implementing "ballooning up" in the pvops
kernels.  Now if you start a domain with "memory=X maxmem=Y", the domain
will start with X MB of memory, but you can use "x[ml] mem-set" to
expand the domain up to Y.

This relies on the toolstack setting the E820 map for the domain with an
E820_RAM region which goes beyond xen_start_info->nr_pages.  When the
domain starts up and sees this, then it adds the extra pages to the
kernel's E820 map, but marks them reserved.  This causes the kernel to
allocate page structures for that memory, but it doesn't attempt to
allocate or use them.  When the balloon driver starts, it adds those
pages to the list of ballooned out pages, and everything works as
expected from there.

This also means that you can fail to boot if Y is many times larger than
X, because the kernel's memory gets filled with page structures.  This
can be particularly acute on 32-bit domains, as the page structures must
be in low memory.

As a side-effect, it also works for dom0.  If you set dom0_mem on the
Xen command line, then nr_pages is limited to that value, but the kernel
can still see the system's real E820 map, and therefore adds all the
system's memory to its own balloon driver, potentially allowing dom0 to
expand up to take all physical memory.

However, this may caused bad side-effects if your system memory is much
larger than your dom0_mem, especially if you use a 32-bit dom0.  I may
need to add a kernel command line option to limit the max initial
balloon size to mitigate this...

Also, any unused pages released at boot time (because they fall into
holes between E820 regions) are also added to the balloon, so they can
be ballooned back in again (this doesn't happen automatically, however).

(Konrad, the infrastructure put in place also makes it very easy for the
kernel to punch a PCI hole in its own E820 maps to leave space for
passthrough devices if we want.  Or if the tools pass in an E820 map
with a suitable hole, then it will be automatically honoured.)

These changes are in xen/next-2.6.32 for the moment.  I'll merge them
into xen/stable-2.6.32.x if they don't cause too many problems.

Also, there's currently a bug in the xl toolset which causes it to
ignore the maxmem domain parameter.  Stefano has a pending patch to fix
this.

I haven't tested this with xm/xend, XCP, Xen Client or Xen Server -
please let me know how it goes.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel