Re: Recent upgrade of 4.13 -> 4.14 issue

[Dario Faggioli <dfaggioli@xxxxxxxx>]

On Sat, 2020-10-31 at 03:54 +0100, marmarek@xxxxxxxxxxxxxxxxxxxxxx
wrote:
> On Sat, Oct 31, 2020 at 02:34:32AM +0000, Dario Faggioli wrote:
> (XEN) *** Dumping CPU7 host state: ***
> (XEN) Xen call trace:
> (XEN)    [<ffff82d040223625>] R _spin_lock+0x35/0x40
> (XEN)    [<ffff82d0402233cd>] S on_selected_cpus+0x1d/0xc0
> (XEN)    [<ffff82d040284aba>] S vmx_do_resume+0xba/0x1b0
> (XEN)    [<ffff82d0402df160>] S context_switch+0x110/0xa60
> (XEN)    [<ffff82d04024310a>] S core.c#schedule+0x1aa/0x250
> (XEN)    [<ffff82d040222d4a>] S softirq.c#__do_softirq+0x5a/0xa0
> (XEN)    [<ffff82d040291b6b>] S vmx_asm_do_vmentry+0x2b/0x30
> 
> And so on, for (almost?) all CPUs.
> 
Yes, you're right.

So, I indeed saw the spin_lock() calls, but I somehow thought I had
seen them in the in the guests' contextes (for which, we probably don't
even print the callstak! :-O). Instead they're there in the host ones.

Sorry for the oversight.

> Note the '*' output is (I think) from a different instances of the
> freeze, so cannot be correlated with other outputs...
> 
> > Maybe they're stuck in the kernel, not in Xen? Thoughs ?
> 
> Given the above spin locks, I don't think so. But also, even if they
> are
> stuck in the kernel, it clearly happened after 4.13 -> 4.14
> upgrade...
> 
Right. So, it seems like a live (I would say) lock. It might happen on
some resource which his shared among domains. And introduced (the
livelock, not the resource or the sharing) in 4.14.

Just giving a quick look, I see that vmx_do_resume() calls
vmx_clear_vmcs() which calls on_selected_cpus() which takes the
call_lock spinlock.

And none of these seems to have received much attention recently.

But this is just a really basic analysis!

Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

Attachment: signature.asc
Description: This is a digitally signed message part