Re: [Xen-devel] Serious issues with xenpaging

[Olaf Hering <olaf@xxxxxxxxx>]

On Mon, Dec 23, Russell Pavlicek wrote:

> 22-Dec: [S:@:S]lars_kurth [S:@:S]RCPavlicek Hey guys, I wrote down as much as 
> I
> could https://piratenpad.de/p/Ik3lOBLniq1L5TEM    (since I'm on holiday and 
> not
> constant online)


> [I've taken the liberty of removing the colorful expletive from the final 
> post]

Thanks for that.

Quoting the above document:

> Docs issues:
> Most accessible documentation is from SUSE manual
> 
>      written for Xen 4.1/xm toolstack

The work was done for SLES11SP2.

>     xenpaging at least on alpine isn't in $PATH,

Yes, its a helper for the toolstack, just like qemu-dm.
 
>     actual path /usr/lib/xen/bin/xenpaging is weird

Its not because its not supposed to be called manually.
 
> Google turns up the SLES docs, the commit and the "how does one test this" 
> thread
> SLES docs don't mention experimental, but mailing list threads etc, do.

I did some testing during that time and xenpaging works well enough with SP2
and SP3 and its xend based toolstack.

> two years after release it would be a good time to have docs + testing sorted

Yes, but that did not happen due to lack of time, resources and lack of
integration into libxl.

> Looking for sources (last time, at least) doesn't show acticity post-2012
> issues:
>  incoherent:       
> 
>     specify a KB "size" of the paging region
> 
>     but specify a number of pages to hold back
> 
> Decide for one thing, not two. Actually, let us specify any of it?

> I bet the system page size is queryable, so why not KB/MB/GB

The current pagesize can be obtained by syscalls.
Its up to the toolstack to drive xenpaging, so the actual admin
interface should be at this level. The values passed to the helper are
an internal detail of the toolstack.

> /var/lib/xen/xenpaging Path is a bit weird, too. 
> Should it rather be /var/lib/xen/paging in line with /var/lib/xen/save ...

Thats a minor detail. I decided on a default path, so lets stick with
it.

> Where's docs how to define it in domU.cfg for XL stack?

There are no docs because there is no code and not even a usable
proposal how to specify the memory related properties for a domU. What
exists today is either populate-on-demand or a fixed amount of memory.
There was some discussion about two years ago how paging, sharing, PoD
and maybe even tmem could be described in a domU.cfg. Nothing was
decided.

> Sles docs say domU.cfg for xm stack was:
> xenpaging = NUMPAGES
> Sles docs basically say for cli use post-domU-creation
> xenpaging domID NUM
> Actual usage is:
> /usr/lib/xen/bin/xenpaging -f /var/lib/xen/xenpaging/freebsd2.page -
> d 79  -m 524288 

I dont think so. The docs say 'memory=N ; actmem=M' and mem-swap-target
to adjust actmem at runtime.
Old upstream docs/misc/xenpaging.txt is certainly wrong, simply because
no code exists to integrate it into the toolstack. 4.2+ is slightly
better.

> Q's:
> - Obviously, the file name handling when not manually setting it up

That did not parse.

> - How does one pull an overview of current xenpaging consumers?

xenpaging is not called by anything.

> - Is the paging file autodiscarded at domU termination?

It is supposed to be removed, xenpaging will get an event when the domid
disappears. In my testing this part works well.

> Testing / doc:
> - what happens if someone messes up and overcommits the xenpaging area

What does that question mean? Does it mean lack of diskspace in dom0?

> - is there any protection against it?
> - will it block or crash?

The overall OOM handling is not very good. Once a guest starts the full
amount of memory for a given domU is required. So if the whole system is
in overcommit state and all guests do a restart an OOM situation occurs
and some guests will not start anymore due to lack of memory.

> Nested paging:
> Would be nice to document it's needed. This should be the like line 2
> of the docs actually. 

What is "nested paging"?

> Same goes for a note that it's only for HVM domUs, btw.

Thats written down in docs/misc/xenpaging.txt.

> Us users don't really use HVM whenever possible, so this is an
> relevant info. It's moot to see the cool feature, mess around long
> since NODOCS and then find out the one use case you got won't work.
> Would be nicer to have s/w workaround for nested page tables if it's
> not available.  Why? On more modern (basically DDR3-Era) hosts we tend
> to be CPU-bound these days and have TONs of Ram.  For that I'd rather
> use tmem to blink out dup's and have compression fancyness instead of
> paging. (if that were tested & documented and known stable ;) If you'd
> use xenpaging on a current vm host you'd probably topside it if the
> memory is ever really allocated.  (think, 128GB ram, 16GB paging,
> allocate 16GB ... ouch, and it just makes no sense since RAM and
> Raid-1 SSD prices are not much different) It's the 4-core 8GB boxes
> and embedded where the paging is a good last resort (methinks) - but
> there you don't have nested page tables.

xenpaging is not a solution but rather a workaround. It can be used to
quickly swapout parts of a running guest to free host memory. Its not
predictable what pages a guest will access next, so it comes with a high
performance cost (even if the pagefile is in dom0 /dev/shm/).

Olaf

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel