Re: [Xen-devel] Dom0 kernel 4.14 with SMP randomly crashing

[Rishi <2rushikeshj@xxxxxxxxx>]

Alright, I got the serial console and following is the crash log. Thank you for pointing that out.

[  133.594852] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [ksoftirqd/2:22]

[  133.599232] Kernel panic - not syncing: softlockup: hung tasks

[  133.602275] CPU: 2 PID: 22 Comm: ksoftirqd/2 Tainted: G             L    4.19.1 #1

[  133.606620] Hardware name: Xen HVM domU, BIOS 4.4.1-xs132257 12/12/2016

[  133.610376] Call Trace:

[  133.611827]  <IRQ>

[  133.612942]  dump_stack+0x5a/0x73

[  133.614676]  panic+0xe8/0x249

[  133.616183]  watchdog_timer_fn+0x200/0x230

[  133.618366]  ? softlockup_fn+0x40/0x40

[  133.620561]  __hrtimer_run_queues+0x133/0x270

[  133.623056]  hrtimer_interrupt+0xfb/0x260

[  133.625580]  xen_timer_interrupt+0x1b/0x30

[  133.628011]  __handle_irq_event_percpu+0x69/0x1a0

[  133.630642]  handle_irq_event_percpu+0x30/0x70

[  133.633826]  handle_percpu_irq+0x34/0x50

[  133.636012]  generic_handle_irq+0x1e/0x30

[  133.638573]  __evtchn_fifo_handle_events+0x163/0x1a0

[  133.641444]  __xen_evtchn_do_upcall+0x41/0x70

[  133.643772]  xen_evtchn_do_upcall+0x27/0x50

[  133.645868]  xen_do_hypervisor_callback+0x29/0x40

[  133.648231]  </IRQ>

[  133.649279] RIP: e030:xen_hypercall_xen_version+0xa/0x20

[  133.652259] Code: 51 41 53 b8 10 00 00 00 0f 05 41 5b 59 c3 cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc 51 41 53 b8 11 00 00 00 0f 05 <41> 5b 59 c3 cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc

[  133.662574] RSP: e02b:ffffc900400dbd18 EFLAGS: 00000246

[  133.665557] RAX: 0000000000040007 RBX: ffff88003315a898 RCX: ffffffff8100122a

[  133.670005] RDX: ffffc900400080ff RSI: 0000000000000000 RDI: 0000000000000000

[  133.674606] RBP: ffff88003315a890 R08: 0000000000000000 R09: 0000000000000000

[  133.678567] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000080000600

[  133.682552] R13: 000000000000002a R14: 00000000000002a0 R15: 000000000000002a

[  133.686851]  ? xen_hypercall_xen_version+0xa/0x20

[  133.689681]  ? xen_force_evtchn_callback+0x9/0x10

[  133.692189]  ? check_events+0x12/0x20

[  133.694227]  ? xen_restore_fl_direct+0x1f/0x20

[  133.696878]  ? _raw_spin_unlock_irqrestore+0x14/0x20

[  133.699894]  ? cp_rx_poll+0x427/0x4d0 [8139cp]

[  133.702421]  ? net_rx_action+0x171/0x3a0

[  133.704693]  ? __do_softirq+0x11e/0x295

[  133.706672]  ? run_ksoftirqd+0x1e/0x40

[  133.708867]  ? smpboot_thread_fn+0x13c/0x160

[  133.711320]  ? kthread+0x115/0x120

[  133.713350]  ? sort_range+0x20/0x20

[  133.715421]  ? __kthread_cancel_work+0x80/0x80

[  133.717969]  ? ret_from_fork+0x35/0x40

[  133.720238] Kernel Offset: disabled

I'll try to solve it ahead.

On Mon, Nov 5, 2018 at 4:28 PM Wei Liu <wei.liu2@xxxxxxxxxx> wrote:

On Mon, Nov 05, 2018 at 04:26:03PM +0530, Rishi wrote:
> Nothing is getting logged. I suspect its because the kernel just goes in
> halt state.
>
> /var/log/kern.log /var/log/messages /var/log/xen/hypervisor.log
>
> All of these files have nothing appended.
>
> I can fetch info from kernel debugfs, if you point me to required area,
>
>
> The dom0 kernel works fine without spilt of SMP code or with "nosmp" option
> at xen command line.

I think you will need to set up a serial console. I wouldn't expect to
see anything in /var/log if the system has crashed.

Without serial log there is nothing we can do to help.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel