Re: [Xen-devel] QEMU bumping memory bug analysis

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

On Mon, Jun 08, 2015 at 04:20:48PM +0100, George Dunlap wrote:
> On 06/08/2015 03:53 PM, Konrad Rzeszutek Wilk wrote:
> > On Mon, Jun 08, 2015 at 01:11:38PM +0100, Ian Campbell wrote:
> >> On Mon, 2015-06-08 at 12:40 +0100, Stefano Stabellini wrote:
> >>
> >>>>> I disagree that libxl should be the arbitrator of a property that is
> >>>>> stored, maintained and enforced by Xen. Xen should be the arbitrator.
> >>>>
> >>>> That's not what "arbitrator" means, an arbitrator decides what the value
> >>>> should be, but that doesn't necessarily imply that it either stores,
> >>>> maintains nor enforces that value. 
> >>>
> >>> The way you describe it, it looks like some kind of host wide memory
> >>> policy manager,
> >>
> >> Not necessarily.
> >>
> >> We need to make a distinction between the entity which enforces
> >> maxpages, that (or those) which decide on what maxpages should be and
> >> tells Xen.
> >>
> >> Xen is the enforcer. It is also therefore by necessity the holder of the
> >> canonical value for the maxpages, since it has to be.
> >>
> >> Xen cannot be the second, it simply doesn't have the necessary
> >> information to make a decision about a domain's memory limit, it just
> >> blindly does whatever it is told by the last thing to tell it something.
> >>
> >> There are lots of ways the desired value of maxpages can be decided on,
> >> given a bunch of entities all with some level of responsibility for the
> >> maxpages of a given domain:
> >>
> >>      1. One single entity which decides on the limit and tells Xen. I
> >>         think this is the xapi model (not squeezed/balloond, they just
> >>         ask xapi to do things AFAIK, so are a red-herring). The unwary
> >>         might think "libxl" was in this category, but it isn't since it
> >>         is in fact multiple instances of the library.
> >>      2. A group of entities which cooperate as a distributed system such
> >>         that any one of them can recalculate (from from the shared
> >>         state, or by intra entity communication) the current desired
> >>         value of max pages and propagate it to Xen. Prior to QEMU
> >>         starting to play with the max pages the multiple libxl's on a
> >>         system fell into this case, coordinating via xenstore
> >>         transactions and libxl userdata.
> >>      3. A group of entities which operate in isolation by only ever
> >>         increasing or descreasing the max pages according to their own
> >>         requirements, without reference to anyone else. When QEMU
> >>         entered the fray, and with the various libxl fixes since, you
> >>         might think we are implementing this model, but we aren't
> >>         because the hypervisor interface is only "set", not
> >>         "increment/decrement" and so there is a racy read/modify/write
> >>         cycle in every entity now.
> >>      4. Xen exposes multiple "maxpages" variables corresponding to the
> >>         various entities which have a need to control this (or multiple
> >>         per entity). Xen adds all of those up to come up with the actual
> >>         maxpages.
> >>      5. I thought there wasw one more but I've forgotten what it was.
> > 
> > claim hypercall kind of fits in this category. It sticks an stake in the 
> > ground
> > so that the 'memory' available for a guest will be available when QEMU,libxl
> > etc go up and down.
> > 
> > And when libxl is done it unclaims the memory (which hopefully at this
> > point has all been given to the guest).
> 
> Right -- in an ideal world maxpages and claim would be set at the same
> time, so that you wouldn't have to worry (for instance) about qemu
> failing because although it was *allowed* to have more memory to
> allocate ROMs, it wan't able to due to lack of available host memory.
> 
> But deciding exactly how big either maxpages or the claim needs to be
> involves either having a central place where all the values are stored
> and calculated (1, 2, or 4), or having each of them just increment or
> decrement the total based on their own need (3), with no way of
> accounting for who asked for what.


Right. I was just pointing out that with claim call you have an semi
atomic mechanism to deal with the amount of memory you need - so you
don't have to worry about the race aspect (another guest taking away
the memory you want to use).

So what is the next step? (I concur BTW with your assesment - just not
sure which group [1,2, or 4 vs 3] is the right direction).

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel