Re: [Xen-devel] xen/arm: Domain not fully destroyed when using credit2

[Dario Faggioli <dario.faggioli@xxxxxxxxxx>]

On Tue, 2017-01-24 at 13:04 +0000, Julien Grall wrote:
> On 24/01/17 12:53, Dario Faggioli wrote:
> > Do you have a script and/or some more info for letting me try to
> > reproduce it (e.g., you say some otf the vCPUs are pinned, which
> > one?
> > etc)?
> 
> That was mentioned in my first e-mail :). My configuration is:
>       - ARM platform with 6 cores
>       - staging Xen with credit2 enabled by default
>       - DOM0 using 2 pinned vCPUs
>       - Guest using 2 vCPUs (not pinned)
> 
Yeah, but some of the details were either missing, or not clear to
me... Sorry for bothering and thanks for re-stating this here. :-)

How are Dom0 vCPUs pinned, exclusively (i.e., there are 2 pCPUs on
which _only_ Dom0 and _no_ DomU can run)?

> The script is really simple:
> 
> for i in `seq 1 10`; do
>       sudo xl create ~/works/guest/guest.cfg;
>       sudo xl destroy guest;
> done
> 
Ok.

> > I'm a bit curious about why you're saying this is being exposed by
> > using Credit2.
> 
> It is been exposed by Credit2 because compared to Credit1 there is
> no 
> interrupt traffic made by the scheduler. 
>
So, when you say "no interrupt traffic", do you perhaps mean that
SCHEDULE_SOFTIRQ is rarely (never!) raised for idle pCPUs? Or are you
really talking about actual interrupts (either inter-processor or not)?

> On ARM with credit2 the 
> interrupt traffic is reduced to none for idle pCPU.
> 
Yes, but _iff_ we're talking about SCHEDULE_SOFTIRQ events, for a truly
idle pCPU (e.g., if I use vcpu-pin to *forbid* every vCPU to execute
there), that's _zero_ also for Credit1, at least on x86 (I've just
tried)!

Perhaps this is too extreme/unrealistic of an idle situation, but I'm
trying to understand the problem. :-)

> In fact:
> > 
> >  1) I've power-cycled quite a few domains in these last months,
> > while
> >     under Credit2, and I don't think I have encountered it on x86;
> 
> AFAIU, IPI is often the only way to broadcast some instruction on
> x86. 
> So compare to ARM, you have likely an higher interrupt traffic.
> 
Right.

> Also, the problem is not obvious to spot unless you look at the free 
> memory (via xl info) before and after. Another solution is printing
> a 
> message in both domain_destroy and complete_domain_destroy.
> 
> You will spot the first message directly. The latter may never be
> printed.
> 
Yep, I was already instrumenting the code like this... I'll let you
know.

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel