RE: [Xen-devel] [PATCH 0 of 9] v2: xl/libxl autoballooning

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

Hi Stefano --

I'm glad you have been working on this for xl since this dom0
capability is also the foundation for ensuring domain creation works
properly when tmem is both enabled on Xen and enabled on one or more
running guests.  In that situation, tmem absorbs much (or even
all) of the free memory in the system, much like dom0 starts
with all the memory and must surrender it to start other guests.
One big difference though is that dom0 doesn't asynchronously
grow and claim the memory back, which can happen with tmem.
Tmem has functionality to cover this case though.

I'm not able to work on this right now... it will probably
be a few weeks before I can, due to various constraints.  But
since you have just implemented this, I'm sure it is fresh
in your brain and you may be able to check or update your
patch to ensure it will easily support tmem as well.
Perhaps you can at least insert comments at the points
where you think the tmem hooks would be necessary?  Or if
you insert actual code that works with tmem disabled,
I can try/debug it later with tmem enabled?  Either way
would make it much easier for me to finish the job later.

There are three important tmem concepts that apply here.
They are:

1) freeable memory
2) tmem freeze/thaw
3) tmem flush

All of these funnel through xc_tmem_control.

Freeable memory (see tmem_query_freeable_mb in XendNode.py)
is memory that tmem can immediately free when requested.
The implementation only queries how much is available in MB.

Tmem freeze guarantees that tmem will not absorb any additional
memory until tmem thaw.  Thus freeable memory can only
increase during a freeze.  (Note that if tmem is not frozen,
"free", "freeable", and their sum may increase or decrease.)
See TMEMC_FREEZE and TMEMC_THAW in xend/balloon.py.
Note that it is critical that any freeze be soon
paired with a matching thaw... tmem is essentially non-
functional during freeze which can affect the performance
of other guests.

Tmem flush (see TMEMC_FLUSH in balloon.py) releases the
amount of memory requested (or all available freeable
memory if the request exceeds freeable memory).

Ideally, freeable memory should be checked to ensure
there is sufficient total memory for the domain before
a flush is done... balloon.py does not currently implement
this, so xend domain creation may unnecessarily flush memory
useful for other guest domains, only to find out that
there was insufficient total memory.

So the algorithm for domain creation should be:
1) If there is enough free memory, create the domain.
   If successful, go to (8).   
   If not, or creation fails due to insufficient memory...
2) Freeze tmem, if ret < 0 (tmem is disabled) goto (8)
3) Check total memory = free + freeable memory
4) If not enough total memory, go to (7)
5) Flush enough tmem memory to create the domain
6) Create the domain
7) Thaw tmem
8) Done
   
Thanks,
Dan

> -----Original Message-----
> From: Stefano Stabellini [mailto:stefano.stabellini@xxxxxxxxxxxxx]
> Sent: Friday, August 27, 2010 7:50 AM
> To: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: [Xen-devel] [PATCH 0 of 9] v2: xl/libxl autoballooning
> 
> Hi all,
> this patch series adds some new memory management functions to libxl
> that are used by xl to autoballoon dom0 down by the amount of memory
> needed by the new domain build.
> In order to avoid conflicts with other xl instances during the memory
> accounting operation I am introducing a global filelock.
> I am also introducing a global configuration file for xl to
> enable/disable autoballooning.
> 
> The only changes compared to v1 are related to paths of xl.conf and the
> lockfile: the paths are now set from the system dependent global
> variables.
> 
> Cheers,
> 
> Stefano
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel