Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

["Edgar E. Iglesias" <edgar.iglesias@xxxxxxxxxx>]

On Wed, Feb 22, 2017 at 09:22:25AM -0800, Stefano Stabellini wrote:
> On Wed, 22 Feb 2017, Edgar E. Iglesias wrote:
> > On Tue, Feb 21, 2017 at 07:20:29PM +0000, Julien Grall wrote:
> > > 
> > > 
> > > On 21/02/2017 18:30, Stefano Stabellini wrote:
> > > >On Tue, 21 Feb 2017, Julien Grall wrote:
> > > >>Hi Stefano,
> > > >>
> > > >>On 21/02/17 17:49, Stefano Stabellini wrote:
> > > >>>On Tue, 21 Feb 2017, Dario Faggioli wrote:
> > > >>>>On Tue, 2017-02-21 at 13:46 +0000, George Dunlap wrote:
> > > >>>>Oh, actually, if --which I only now realize may be what you are
> > > >>>>referring to, since you're talking about "guest burning its credits"--
> > > >>>>you let the vCPU put the pCPU to sleep *but*, when it wakes up (or 
> > > >>>>when
> > > >>>>the scheduler runs again for whatever reason), you charge to it for 
> > > >>>>all
> > > >>>>the time the the pCPU was actually idle/sleeping, well, that may
> > > >>>>actually  not break scheduling, or cause disruption to the service of
> > > >>>>other vCPUs.... But indeed I'd consider it rather counter intuitive a
> > > >>>>behavior.
> > > >>>
> > > >>>How can this be safe? There could be no interrupts programmed to wake 
> > > >>>up
> > > >>>the pcpu at all. In fact, I don't think today there would be any, 
> > > >>>unless
> > > >>>we set one up in Xen for the specific purpose of interrupting the pcpu
> > > >>>sleep.
> > > >>>
> > > >>>I don't know the inner working of the scheduler, but does it always 
> > > >>>send
> > > >>>an interrupt to other pcpu to schedule something?
> > > >>
> > > >>You still seem to assume that WFI/WFE is the only way to get a vCPU
> > > >>unscheduled. If that was the case it would be utterly wrong because you 
> > > >>cannot
> > > >>expect a guest to use them.
> > > >>
> > > >>>
> > > >>>What if there are 2 vcpu pinned to the same pcpu? This cannot be fair.
> > > >>
> > > >>Why wouldn't it be fair? This is the same situation as a guest vCPU not 
> > > >>using
> > > >>WFI/WFE.
> > > >
> > > >I read your suggestion as trapping WFI in Xen, then, depending on
> > > >settings, executing WFI in the Xen trap handler to idle the pcpu. That
> > > >doesn't work. But I take you suggested not trapping wfi (remove
> > > >HCR_TWI), executing the instruction in guest context. That is what we
> > > >used to do in the early days (before a780f750). It should be safe and
> > > >possibly even quick. I'll rerun the numbers and let you know.
> > > 
> > > My first suggestion was to emulate WFI in Xen, which I agree is not safe 
> > > :).
> > > 
> > > I think not trapping WFI will have the best performance but may impact the
> > > credit of the vCPU as mentioned by Dario and George.
> > 
> > I agree, wfi in guest context or at least with everything prepared to 
> > return to
> > the current guest would be great.
> 
> And the new numbers look good:
> 
>          AVG        MAX     WARM_MAX
> credit1  1850         2650    1950
> credit2  1850         2950    1840 
> 
> We are hitting the same levels as the non-WFI case. Nice!

Yeah, very nice :-)

Thanks!
Edgar

> 
> 
> > An option to enable this would work fine for our use-cases. Or if we could
> > at runtime detect that it's the best approach given scheduling (i.e
> > exclusive vCPU/pCPU pinning) even better.
> 
> The option is easy and we can do it today. We might be able to do
> automatic enablement once we have a "nop" scheduler.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel