Re: [Xen-devel] Xen balloon driver improvement (version 1)

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Thu, Oct 23, 2014 at 01:27:45PM +0100, Ian Campbell wrote:
> On Thu, 2014-10-23 at 13:17 +0100, Wei Liu wrote:
> > On Thu, Oct 23, 2014 at 12:59:17PM +0100, Ian Campbell wrote:
> > > On that subject, how do you handle alloc_xenballooned_pages calls of
> > > non-2M alignment? Would it be best to do a 2M balloon and queue the rest
> > > for use on future similar allocations?
> > > 
> > > If so then I'm wondering if it might make sense to keep the spare 4K
> > > pages from doing this on a separate queue to the normal 4K queue, in
> > > order to keep these sorts pages isolated into 2M regions -- because I
> > > expect that they cannot be compacted without cooperation with the driver
> > > which allocated them (which I expect won't even be possible in many
> > > cases).
> > > 
> > 
> > Yes, it requires cooperation from the driver, and I don't think it's a
> > good idea because that would mean drivers need to do weird things which
> > hinder performance and increase complexity. 
> 
> I have a feeling it may even be impossible in some cases.
> 
> > I intend to not touch them, just leave them in separate queue.
> 
> i.e. a separate one from the "unusued ballooned 4k"?
> 

Yes. A separate one -- if the "unused ballooned 4k" queue refers to the
queue that holds balloon pages which are subject to balloon page
compaction.

> > > On both here and decrease there are attempts to allocate (from either
> > > the queue or the kernel, depending) 2M which could fail. I wonder if it
> > > is worth inserting "kick THP and give it a chance"? It's a question of
> > > tradeoffs in the latency of a ballooning operation vs the efficiency
> > > with which we can use 2M allocations.
> > > 
> > 
> > No, THP is not involved. I think you mean balloon page compaction.
> 
> I meant whichever global kernel facility exists to try and increase the
> supply of 2M pages for allocations, which certainly includes balloon
> page compaction, but also includes all the other types of compaction
> which can be done, no? Maybe THP isn't the right name, or perhaps I
> think the kernel has more functionality than it does in reality?
> 
> > I'm not sure if it's a good idea because we're now introducing latency
> > that we can't control.
> 
> By "give it a chance" I meant "wait some period which we control", if it
> can't succeed in that timescale then give up. So the latency would be
> bound by us (or our supplied control knob, or whatever) not by the "THP"
> side of things.
> 
> >  At the very least, if guest admin really cares he
> > / she should be able to kick off compaction voluntarily.
> 
> If the kernel can compact e.g. normal user pages then it would be better
> to do that than to balloon and compact those later, I think? (which is
> contingent on my not having overestimated what the kernel is capable
> of...)
> 

OK, I think there's some misunderstanding here. When kernel tries to
allocate high order page, it already kicks of compaction (including
normal page and balloon page compaction) when fast path fails.

So I was thinking about something like calling compact_zone or rolling
our own implementation when I saw your reply. That's a very time
consuming operation and time varies depending on kernel parameters and
the status of memory fragmentation.

Wei.

> Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel