[Xen-devel] xen-unstable: [error_code=0000] , IN INTERRUPT CONTEXT

[Bruce Edge <bruce.edge@xxxxxxxxx>]

With top of xen-unstable and pv-ops 2.6.32.x, loading some of our
custom PCI drivers in a domU kills off xen.
Here's the xen serial console output:

0 kaan-18  ~ #> [  786.689775] pciback 0000:07:00.0: enabling device
(0000 -> 0003)
[  786.689834] xen: registering gsi 32 triggering 0 polarity 1
[  786.689855] xen_allocate_pirq: returning irq 32 for gsi 32
[  786.689874] xen: --> irq=32
[  786.689888] Already setup the GSI :32
[  786.689904] pciback 0000:07:00.0: PCI INT A -> GSI 32 (level, low) -> IRQ 32
[  786.689959] pciback 0000:07:00.0: setting latency timer to 64
[  786.690523] pciback 0000:07:00.0: Driver tried to write to a
read-only configuration space field at offset 0x62, size 2. This may
be harmless, but if you have problems with your device:
[  786.690525] 1) see permissive attribute in sysfs
[  786.690526] 2) report problems to the xen-devel mailing list along
with details of your device obtained from lspci.
[  786.690741]   alloc irq_desc for 478 on node 0
[  786.690764]   alloc kstat_irqs on node 0
[  786.762089] pciback 0000:07:00.1: enabling device (0000 -> 0003)
[  786.762128] xen: registering gsi 42 triggering 0 polarity 1
[  786.762168] xen_allocate_pirq: returning irq 42 for gsi 42
[  786.762187] xen: --> irq=42
[  786.762201] Already setup the GSI :42
[  786.762218] pciback 0000:07:00.1: PCI INT B -> GSI 42 (level, low) -> IRQ 42
[  786.762254] pciback 0000:07:00.1: setting latency timer to 64
[  786.762729] pciback 0000:07:00.1: Driver tried to write to a
read-only configuration space field at offset 0x62, size 2. This may
be harmless, but if you have problems with your device:
[  786.762731] 1) see permissive attribute in sysfs
[  786.762731] 2) report problems to the xen-devel mailing list along
with details of your device obtained from lspci.
[  786.762935]   alloc irq_desc for 477 on node 0
[  786.762958]   alloc kstat_irqs on node 0
[  786.824345] pciback 0000:07:00.2: enabling device (0000 -> 0003)
[  786.824400] xen: registering gsi 47 triggering 0 polarity 1
[  786.824421] xen_allocate_pirq: returning irq 47 for gsi 47
[  786.824440] xen: --> irq=47
[  786.824453] Already setup the GSI :47
[  786.824469] pciback 0000:07:00.2: PCI INT C -> GSI 47 (level, low) -> IRQ 47
[  786.824505] pciback 0000:07:00.2: setting latency timer to 64
(XEN)
(XEN)
(XEN) NMI - MEMORY ERROR
(XEN) ----[ Xen-4.1-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82c480105e97>] evtchn_set_pending+0x1b/0x197
(XEN) RFLAGS: 0000000000000082   CONTEXT: hypervisor
(XEN) rax: ffff83018b802e90   rbx: ffff83018f2e0000   rcx: 0000000000000001
(XEN) rdx: 0000000000000000   rsi: 000000000000004c   rdi: ffff8300df586000
(XEN) rbp: ffff82c48028fcc8   rsp: ffff82c48028fca8   r8:  00000000000000d9
(XEN) r9:  0000000000000001   r10: ffff8800142b23f0   r11: ffff82c480201e90
(XEN) r12: 000000000000004c   r13: 0000000000000000   r14: ffff830195c10060
(XEN) r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000000026f0
(XEN) cr3: 00000001373b9000   cr2: ffff88001429bda8
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff82c48028fca8:
(XEN)    ffff82c48028fcd8 ffff83018f2e0000 000000000000007f 0000000000000000
(XEN)    ffff82c48028fcd8 ffff82c480106162 ffff82c48028fd88 ffff82c48015efd0
(XEN)    000000000d9c90a0 5000000000000000 ffff82f6031b3840 000000000018d9c2
(XEN)    0000000000000042 ffff83019ff021b4 0000004200000002 ffff82c48028ff18
(XEN)    000000008028ff18 ffff82c48028fef8 ffff83019ff02180 0000000000000108
(XEN)    ffff82c48028fd88 0000000100007ff0 ffff83019e660018 ffff83019ff01880
(XEN)    0000000000000030 ffff82c48028fef8 0000000000001800 ffff82c4802a4080
(XEN)    00007d3b7fd70247 ffff82c480157fe6 ffff82c4802a4080 0000000000001800
(XEN)    ffff82c48028fef8 0000000000000030 ffff82c48028fee8 ffff83019ff01880
(XEN)    ffff82c480201e90 ffff8800142b23f0 0000000000000001 0000000000000000
(XEN)    0000000000000000 ffff82c48028fea8 ffff82c48028ff18 0000000000000082
(XEN)    ffff83019ff018b4 000000d900000000 ffff82c48012072a 000000000000e008
(XEN)    0000000000000246 ffff82c48028fe40 000000000000e010 ffff82c48015f194
(XEN)    ffff82c48028ff18 ffff82c48028ff18 ffff82c48028fea8 ffff82c48028ff18
(XEN)    0000000000000030 ffff83019ff018b4 0000003000000000 ffff83019fed5fb0
(XEN)    ffff82c48028fef8 0000000000000000 0000000000000000 0000000000000246
(XEN)    ffffffffff60004d 0000000000000100 ffffffffff60004d ffff8800142b23f0
(XEN)    ffff88001e7fd310 ffffffff8123b00f 0000000000000001 0000000000000200
(XEN)    00007d3b7fd700e7 ffff82c480157fe6 0000000000000200 0000000000000001
(XEN)    ffffffff8123b00f ffff88001e7fd310 ffff8800142459c0 ffff8800142b23f0
(XEN) Xen call trace:
(XEN)    [<ffff82c480105e97>] evtchn_set_pending+0x1b/0x197
(XEN)    [<ffff82c480106162>] send_guest_pirq+0x54/0x56
(XEN)    [<ffff82c48015efd0>] do_IRQ+0x3c9/0x65e
(XEN)    [<ffff82c480157fe6>] common_interrupt+0x26/0x30
(XEN)    [<ffff82c48012072a>] _spin_unlock_irq+0x22/0x23
(XEN)    [<ffff82c480157fe6>] common_interrupt+0x26/0x30
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL TRAP: vector = 2 (nmi)
(XEN) [error_code=0000] , IN INTERRUPT CONTEXT
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

These are the PCI devices passed through to the domU:

0 kaan-18-dpm  ~ #> lspci
07:00.0 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 08)
07:00.1 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 08)
07:00.2 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 08)
07:00.3 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 08)

I was tailing kern.log on the domU at the same time and got this from the domU:
It's possibly unrelated as we're early in the pvops port of our
drivers, but it may shed some light so I'm including it.

2010-09-15T16:39:39.308892-07:00 kaan-18-dpm [  599.301862] WARNING:
at mm/page_alloc.c:1820 __alloc_pages_nodemask+0x669/0x7c0()
2010-09-15T16:39:39.308904-07:00 kaan-18-dpm [  599.301865] Modules
linked in: dpm_css(+) dpm_kstat dpm_hpi dpm_sid dpm_klib ipv6 lp
parport xen_kbdfront xen_netfront xen_pcifront xen_blkfront
2010-09-15T16:39:39.308915-07:00 kaan-18-dpm [  599.301882] Pid: 1256,
comm: modprobe Not tainted 2.6.32.21-1 #1
2010-09-15T16:39:39.308926-07:00 kaan-18-dpm [  599.301884] Call Trace:
2010-09-15T16:39:39.308937-07:00 kaan-18-dpm [  599.301888]
[<ffffffff8111cf79>] ? __alloc_pages_nodemask+0x669/0x7c0
2010-09-15T16:39:39.308948-07:00 kaan-18-dpm [  599.301894]
[<ffffffff8106bf70>] warn_slowpath_common+0x80/0xd0
2010-09-15T16:39:39.308959-07:00 kaan-18-dpm [  599.301898]
[<ffffffff8106bfd4>] warn_slowpath_null+0x14/0x20
2010-09-15T16:39:39.308971-07:00 kaan-18-dpm [  599.301901]
[<ffffffff8111cf79>] __alloc_pages_nodemask+0x669/0x7c0
2010-09-15T16:39:39.308982-07:00 kaan-18-dpm [  599.301907]
[<ffffffff810ab3bb>] ? __lock_acquire+0x80b/0x17a0
2010-09-15T16:39:39.308994-07:00 kaan-18-dpm [  599.301912]
[<ffffffff8103b976>] ? pvclock_clocksource_read+0x56/0xc0
2010-09-15T16:39:39.309005-07:00 kaan-18-dpm [  599.301918]
[<ffffffff8101094d>] ? xen_force_evtchn_callback+0xd/0x10
2010-09-15T16:39:39.309035-07:00 kaan-18-dpm [  599.301922]
[<ffffffff81011202>] ? check_events+0x12/0x20
2010-09-15T16:39:39.309047-07:00 kaan-18-dpm [  599.301927]
[<ffffffff81150b4c>] alloc_pages_current+0x8c/0xe0
2010-09-15T16:39:39.309058-07:00 kaan-18-dpm [  599.301930]
[<ffffffff8111b754>] __get_free_pages+0x14/0x60
2010-09-15T16:39:39.309070-07:00 kaan-18-dpm [  599.301939]
[<ffffffffa0085368>] klib_malloc_by_type+0x1a8/0x2f0 [dpm_klib]
2010-09-15T16:39:39.309081-07:00 kaan-18-dpm [  599.301952]
[<ffffffffa00fdad0>] ? init_module+0x0/0x270 [dpm_css]
2010-09-15T16:39:39.309106-07:00 kaan-18-dpm [  599.301958]
[<ffffffffa00855a6>] klib_malloc+0xf6/0x250 [dpm_klib]
2010-09-15T16:39:39.309119-07:00 kaan-18-dpm [  599.301962]
[<ffffffff810a7c86>] ? lockdep_init_map+0x56/0x150
2010-09-15T16:39:39.309130-07:00 kaan-18-dpm [  599.301974]
[<ffffffffa010fd0b>] csspsco_inittbl+0x4b/0x230 [dpm_css]
2010-09-15T16:39:39.309141-07:00 kaan-18-dpm [  599.301986]
[<ffffffffa00f3d7b>] ? kst_initpool+0x2b/0x130 [dpm_css]
2010-09-15T16:39:39.309152-07:00 kaan-18-dpm [  599.301998]
[<ffffffffa00fdad0>] ? init_module+0x0/0x270 [dpm_css]
2010-09-15T16:39:39.309162-07:00 kaan-18-dpm [  599.302010]
[<ffffffffa00fe85b>] css_init+0xfb/0x840 [dpm_css]
2010-09-15T16:39:39.309192-07:00 kaan-18-dpm [  599.302015]
[<ffffffff815f400b>] ? _spin_unlock+0x2b/0x40
2010-09-15T16:39:39.309204-07:00 kaan-18-dpm [  599.302021]
[<ffffffffa0086b8a>] ? tmeter_reg+0x8a/0x110 [dpm_klib]
2010-09-15T16:39:39.309215-07:00 kaan-18-dpm [  599.302033]
[<ffffffffa00fdad0>] ? init_module+0x0/0x270 [dpm_css]
2010-09-15T16:39:39.309226-07:00 kaan-18-dpm [  599.302044]
[<ffffffffa00fdad0>] ? init_module+0x0/0x270 [dpm_css]
2010-09-15T16:39:39.309237-07:00 kaan-18-dpm [  599.302056]
[<ffffffffa00fdb59>] init_module+0x89/0x270 [dpm_css]
2010-09-15T16:39:39.309261-07:00 kaan-18-dpm [  599.302067]
[<ffffffffa00fdad0>] ? init_module+0x0/0x270 [dpm_css]
2010-09-15T16:39:39.309274-07:00 kaan-18-dpm [  599.302072]
[<ffffffff8100a04d>] do_one_initcall+0x3d/0x1d0
2010-09-15T16:39:39.309285-07:00 kaan-18-dpm [  599.302076]
[<ffffffff810bd038>] sys_init_module+0xe8/0x270
2010-09-15T16:39:39.309296-07:00 kaan-18-dpm [  599.302081]
[<ffffffff810151f2>] system_call_fastpath+0x16/0x1b
2010-09-15T16:39:39.309308-07:00 kaan-18-dpm [  599.302083] ---[ end
trace b968ffb3af48abf3 ]---

-Bruce

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel