Re: [Xen-devel] [PATCH] RFC: initial libxl support for xenpaging

["Andres Lagar-Cavilla" <andres@xxxxxxxxxxxxxxxx>]

> On Mon, Feb 27, 2012 at 2:45 PM, Andres Lagar-Cavilla
> <andres@xxxxxxxxxxxxxxxx> wrote:
>>> On Fri, 2012-02-24 at 17:12 +0000, Andres Lagar-Cavilla wrote:

I'm gonna top-post (gasp!). At the Hackathon, Ian C, Adin and I got
together and held a brief pow-wow on the memory management interface. This
is a rough summary. Comments highly welcome!

Please keep in mind that this is a sketch of ideas, not an edict on naming
and syntax rules. So pretty much everything below is changeable. Also note
that xl is used liberally below, without distinction between xl and libxl.
Partly due to a 1-1 match between the two in terms of the functionality
explained, and partly due to ignorance :)

The new interface has a master key for the domain memory footprint, and a
policy key that allows fine-grained manipulation of how to achieve this
footprint.

xl mem-set foo
  This is a value in KiB. It will translate to writing the
  /local/domain/X/memory-policy/target key
  *note is does not write the traditional memory/target key. The reason
   will become clear later*

xl mem-policy foo
  This is a string. It will translate to writing the
  /local/domain/X/memory-policy/actor key

The per-domain xl daemon watches memory-policy/target. When this is
written to it wakes up, reads the actor key, and if the actor key does not
refer to xl, it will simply not care. Multiple actors can have the same
behavior, and in this way only the right actor is awakened.

By default, the actor key points to xl. But it could point to an external
actor. For example, xenpaging itself could watch this, and get to act if the
specified actor is 'xenpaging'.

The xl policy can then be controlled with "power user" xl knobs. We won't
deatil the specific syntax here. By default, the policy is "first try to
achieve the target via ballooning. If it can't get there fast enough,
resort to paging". The "power user" xl-specific knobs are:
- policy: an enum of known policies
 -- first balloon then paging (default)
 -- only balloon
 -- only paging
 -- first paging then balloon
- primary timeout: how long to wait before triggering the second option.

Note that libxl will then be in charge of writing memory/target (to
trigger the balloon), memory/target-tot_pages (xenpaging), launch
xenpaging, etc. All internal bits no one outside of libxl have to care
about.

Note that this interface does not preclude sharing or tmem or "other", but
doesnot incorporate them right now. The aim of the interface is to be
extensible, so that after release 4.2 tmem/sharing/foo are added without
breaking anything.

Another goal is for global host memory controllers to get to manage
domains' footprints via this interface with minimal effort. Something like
squeezed/xenballoond could set per domain actor values (to match today's
behavior, to the equivalent of "xl:balloon"), and then just call libxl
mem-set.

So, this is a bit of a 10K foot view in order to get the primary
guidelines "right", and then dive again to hammer out the fine-grained
details.

Thanks,
Andres

>>>> Sharing + balloon is mostly a bad idea. It's not forbidden or broken
>>>> from the hypervisor angle, but it's a lose-lose game. Ballooning a
>>>> shared page gains you nothing. The ballooner can't know what is shared
>>>> and what isn't, in order to make an educated decision. And the sharer
>>>> can't foresee what the balloon will victimize.
>>>
>>> Doesn't ballooning generally evict unused pages, because it allocates
>>> new _free_ pages, even if they were shared is there any benefit to
>>> doing
>>> so?
>>
>> Certainly the balloon will pick free pages. The sharing daemon should
>> not
>> have shared those, but it's not unlikely that it will have.
>>
>> It's a classic semantic gap problem, and we were discussing this in the
>> context of paging ("the pager should not have paged out page table
>> pages")
>>
>> Beyond free pages, a prime target for sharing are read-only disk buffers
>> in the page cache. Those are victim #2 for the balloon.
>
> Not exactly -- victim #2 would be read-only disk buffers *which have
> not been read recently*.  Buffers which are in active use will not be
> evicted.  So although evicting these pages from the guests' cache
> doesn't buy the system any more memory, it doesn't have a major impact
> on the guest either.
>
> In any case, if the guest experiences its own internal memory
> pressure, these pages will be the first to go anyway.  After that, it
> will go for trying to evict infrequently-used read-write pages --
> which, if the pager is active, will already have been paged out to
> disk; thus we'll end up with the double-paging problem.  This will
> have a much larger impact on performance than uselessly evicting
> little-used read-only pages.
>
> So I think that even though sharing+balloon will lead to some
> occasional sub-optimal behavior, it's still a lot better than
> sharing+paging and no ballooning.  Remember that ballooning was a
> technique introduced in the paper by VMWare that talked about page
> sharing -- they obviously thought sharing+ballooning was better than
> sharing+paging.
>
>  -George
>



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel