Re: [Xen-devel] Investigating a SLES11 guest crash

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 13.09.13 at 12:49, Andrew Bennieston <andrew.bennieston@xxxxxxxxxx> 
>>> wrote:
> I'm trying to get to the bottom of a SLES11 guest crash (stack trace
> below). This is an early page fault on a 64-bit SLES11 SP0 guest, and
> only appears in debug builds of Xen.
> 
> An automated git bisect run in xen.git pointed at the following commit.
> I realise that this is a debug commit, and is therefore likely not the
> culprit, but it probably exposes the real bug, which was hiding silently
> before.
> 
> commit f1bde87fc08ce8c818a1640a8fe4765d48923091
> Author: Jan Beulich <jbeulich@xxxxxxxx>
> Date:   Fri Feb 8 11:06:04 2013 +0100
> 
>      x86: debugging code for testing 16Tb support on smaller memory systems
> 
>      Signed-off-by: Jan Beulich <jbeulich@xxxxxxxx>
>      Acked-by: Keir Fraser <keir@xxxxxxx>

This by itself indeed is rather unlikely to have introduced the bug.
However, I'd also have thought that it's unlikely for it to introduce
a guest side crash.

> I wondered if anyone had advice on the likely cause, or how to go about
> debugging from here? In particular, are there any bits of code that are
> likely culprits, or commits that might be candidates? One option is to
> rerun the bisect, and apply this debug patch on top of each point...
> I'll investigate that in the absence of alternatives.

There isn't that big of a window, since the patch here matters
only after the non-trivial map_domain_page() et al got
re-introduced.

> (XEN) Pagetable walk from ffff8800027da000:
> (XEN)  L4[0x110] = 000000022ca5a067 00000000000015d0
> (XEN)  L3[0x000] = 000000022ca59067 00000000000015d1
> (XEN)  L2[0x013] = 000000022ca45067 00000000000015e5
> (XEN)  L1[0x1da] = 801000022b850065 00000000000027da
> (XEN) domain_crash_sync called from entry.S
> (XEN) Domain 8 (vcpu#0) crashed on cpu#7:
> (XEN) ----[ Xen-4.3-unstable  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    7
> (XEN) RIP:    e033:[<ffffffff80217b91>]
> (XEN) RFLAGS: 0000000000000246   EM: 1   CONTEXT: pv guest
> (XEN) rax: ffff880000000000   rbx: ffff8800027da000   rcx: 0000000000000023
> (XEN) rdx: 00003ffffffff000   rsi: 00000000027da000   rdi: 00000000deadbeef
> (XEN) rbp: ffff8800027d9fd8   rsp: ffffffff80709e60   r8:  0000000000000000
> (XEN) r9:  00000000027da000   r10: ffff880001871680   r11: 0000000000000100
> (XEN) r12: ffff880000000000   r13: 00003ffffffff000   r14: ffffffffff600000
> (XEN) r15: 0000000229523065   cr0: 000000008005003b   cr4: 00000000000026f0
> (XEN) cr3: 000000022d329000   cr2: ffff8800027da000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> (XEN) Guest stack trace from rsp=ffffffff80709e60:
> (XEN)    0000000000000023 0000000000000100 0000000000000003 ffffffff80217b91
> (XEN)    000000010000e030 0000000000010046 ffffffff80709ea8 000000000000e02b
> (XEN)    ffffffff80217988 0000000000b90000 0000000229523065 ffffffffff600000
> (XEN)    000000000082e000 0000000000b90000 ffff880000000000 0000000000000000
> (XEN)    ffffffff8021c6bf 0000000000040000 0000000040000000 00000000013be000
> (XEN)    ffffffff80714233 0000000000000000 0000000000000000 0000000000000000
> (XEN)    ffffffff80463720 ffffffff00000008 ffffffff80709f88 ffffffff80709f48
> (XEN)    ffffffff8071d80f 0000000000000002 0000000000000000 0000000000000000
> (XEN)    ffffffffffffffff ffffffff80733eb0 0000000000000000 0000000000000000
> (XEN)    ffffffff807106ff ffffffff7fffffff ffffffff807100d2 ffffffff807362e0
> (XEN)    ffffffff815be000 0000000000000000 0000000000000000 0000000000000000
> (XEN)    ffffffff807101b0 ffff800000000000 ffff804000000000 00000007ffffffff
> (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN)    ffffffff807102d2 ffffffff807106b4 ffffffff807106bc ffffffff807106c4
> (XEN)    ffffffff8020906a ffffffff8020911a ffffffff80209e34 ffffffff80209ed9
> (XEN)    ffffffff80209fba ffffffff80209fcd ffffffff80209fd4 ffffffff80209ff1
> (XEN)    ffffffff8020a0c9 ffffffff8045854b ffffffff8020ab35 ffffffff8020acb6
> (XEN)    ffffffff8020ae8d ffffffff80466182 ffffffff804662a5 ffffffff804663e9
> (XEN)    ffffffff80466409 ffffffff8020d9e1 ffffffff8020da1e ffffffff8020da73
> (XEN)    ffffffff8020dcbf ffffffff8020e670 ffffffff80713fb0 ffffffff80713fb8

Did you try taking this apart, to understand what it is in the guest
that crashes?

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel