Re: [Xen-devel] xen-unstable: [error_code=0000] , IN INTERRUPT CONTEXT

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

On Wed, Sep 15, 2010 at 04:49:51PM -0700, Bruce Edge wrote:
> With top of xen-unstable and pv-ops 2.6.32.x, loading some of our
> custom PCI drivers in a domU kills off xen.
> Here's the xen serial console output:
> 
> 0 kaan-18  ~ #> [  786.689775] pciback 0000:07:00.0: enabling device
> (0000 -> 0003)
> [  786.689834] xen: registering gsi 32 triggering 0 polarity 1
> [  786.689855] xen_allocate_pirq: returning irq 32 for gsi 32
> [  786.689874] xen: --> irq=32
> [  786.689888] Already setup the GSI :32
> [  786.689904] pciback 0000:07:00.0: PCI INT A -> GSI 32 (level, low) -> IRQ 
> 32
> [  786.689959] pciback 0000:07:00.0: setting latency timer to 64
> [  786.690523] pciback 0000:07:00.0: Driver tried to write to a
> read-only configuration space field at offset 0x62, size 2. This may
> be harmless, but if you have problems with your device:
> [  786.690525] 1) see permissive attribute in sysfs
> [  786.690526] 2) report problems to the xen-devel mailing list along
> with details of your device obtained from lspci.
> [  786.690741]   alloc irq_desc for 478 on node 0
> [  786.690764]   alloc kstat_irqs on node 0
> [  786.762089] pciback 0000:07:00.1: enabling device (0000 -> 0003)
> [  786.762128] xen: registering gsi 42 triggering 0 polarity 1
> [  786.762168] xen_allocate_pirq: returning irq 42 for gsi 42
> [  786.762187] xen: --> irq=42
> [  786.762201] Already setup the GSI :42
> [  786.762218] pciback 0000:07:00.1: PCI INT B -> GSI 42 (level, low) -> IRQ 
> 42
> [  786.762254] pciback 0000:07:00.1: setting latency timer to 64
> [  786.762729] pciback 0000:07:00.1: Driver tried to write to a
> read-only configuration space field at offset 0x62, size 2. This may
> be harmless, but if you have problems with your device:
> [  786.762731] 1) see permissive attribute in sysfs
> [  786.762731] 2) report problems to the xen-devel mailing list along
> with details of your device obtained from lspci.
> [  786.762935]   alloc irq_desc for 477 on node 0
> [  786.762958]   alloc kstat_irqs on node 0
> [  786.824345] pciback 0000:07:00.2: enabling device (0000 -> 0003)
> [  786.824400] xen: registering gsi 47 triggering 0 polarity 1
> [  786.824421] xen_allocate_pirq: returning irq 47 for gsi 47
> [  786.824440] xen: --> irq=47
> [  786.824453] Already setup the GSI :47
> [  786.824469] pciback 0000:07:00.2: PCI INT C -> GSI 47 (level, low) -> IRQ 
> 47
> [  786.824505] pciback 0000:07:00.2: setting latency timer to 64
> (XEN)
> (XEN)
> (XEN) NMI - MEMORY ERROR
> (XEN) ----[ Xen-4.1-unstable  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    0
> (XEN) RIP:    e008:[<ffff82c480105e97>] evtchn_set_pending+0x1b/0x197
> (XEN) RFLAGS: 0000000000000082   CONTEXT: hypervisor
> (XEN) rax: ffff83018b802e90   rbx: ffff83018f2e0000   rcx: 0000000000000001
> (XEN) rdx: 0000000000000000   rsi: 000000000000004c   rdi: ffff8300df586000
> (XEN) rbp: ffff82c48028fcc8   rsp: ffff82c48028fca8   r8:  00000000000000d9
> (XEN) r9:  0000000000000001   r10: ffff8800142b23f0   r11: ffff82c480201e90
> (XEN) r12: 000000000000004c   r13: 0000000000000000   r14: ffff830195c10060
> (XEN) r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000000026f0
> (XEN) cr3: 00000001373b9000   cr2: ffff88001429bda8
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen stack trace from rsp=ffff82c48028fca8:
> (XEN)    ffff82c48028fcd8 ffff83018f2e0000 000000000000007f 0000000000000000
> (XEN)    ffff82c48028fcd8 ffff82c480106162 ffff82c48028fd88 ffff82c48015efd0
> (XEN)    000000000d9c90a0 5000000000000000 ffff82f6031b3840 000000000018d9c2
> (XEN)    0000000000000042 ffff83019ff021b4 0000004200000002 ffff82c48028ff18
> (XEN)    000000008028ff18 ffff82c48028fef8 ffff83019ff02180 0000000000000108
> (XEN)    ffff82c48028fd88 0000000100007ff0 ffff83019e660018 ffff83019ff01880
> (XEN)    0000000000000030 ffff82c48028fef8 0000000000001800 ffff82c4802a4080
> (XEN)    00007d3b7fd70247 ffff82c480157fe6 ffff82c4802a4080 0000000000001800
> (XEN)    ffff82c48028fef8 0000000000000030 ffff82c48028fee8 ffff83019ff01880
> (XEN)    ffff82c480201e90 ffff8800142b23f0 0000000000000001 0000000000000000
> (XEN)    0000000000000000 ffff82c48028fea8 ffff82c48028ff18 0000000000000082
> (XEN)    ffff83019ff018b4 000000d900000000 ffff82c48012072a 000000000000e008
> (XEN)    0000000000000246 ffff82c48028fe40 000000000000e010 ffff82c48015f194
> (XEN)    ffff82c48028ff18 ffff82c48028ff18 ffff82c48028fea8 ffff82c48028ff18
> (XEN)    0000000000000030 ffff83019ff018b4 0000003000000000 ffff83019fed5fb0
> (XEN)    ffff82c48028fef8 0000000000000000 0000000000000000 0000000000000246
> (XEN)    ffffffffff60004d 0000000000000100 ffffffffff60004d ffff8800142b23f0
> (XEN)    ffff88001e7fd310 ffffffff8123b00f 0000000000000001 0000000000000200
> (XEN)    00007d3b7fd700e7 ffff82c480157fe6 0000000000000200 0000000000000001
> (XEN)    ffffffff8123b00f ffff88001e7fd310 ffff8800142459c0 ffff8800142b23f0
> (XEN) Xen call trace:
> (XEN)    [<ffff82c480105e97>] evtchn_set_pending+0x1b/0x197
> (XEN)    [<ffff82c480106162>] send_guest_pirq+0x54/0x56
> (XEN)    [<ffff82c48015efd0>] do_IRQ+0x3c9/0x65e
> (XEN)    [<ffff82c480157fe6>] common_interrupt+0x26/0x30
> (XEN)    [<ffff82c48012072a>] _spin_unlock_irq+0x22/0x23
> (XEN)    [<ffff82c480157fe6>] common_interrupt+0x26/0x30
> (XEN)
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) FATAL TRAP: vector = 2 (nmi)
> (XEN) [error_code=0000] , IN INTERRUPT CONTEXT
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
> 
> These are the PCI devices passed through to the domU:
> 
> 0 kaan-18-dpm  ~ #> lspci
> 07:00.0 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 08)
> 07:00.1 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 08)
> 07:00.2 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 08)
> 07:00.3 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 08)
> 
> I was tailing kern.log on the domU at the same time and got this from the 
> domU:
> It's possibly unrelated as we're early in the pvops port of our
> drivers, but it may shed some light so I'm including it.
> 
> 2010-09-15T16:39:39.308892-07:00 kaan-18-dpm [  599.301862] WARNING:
> at mm/page_alloc.c:1820 __alloc_pages_nodemask+0x669/0x7c0()

Well, that looks like an allocation for a huge swath of memory failed. Did your
driver check that it failed to allocate this chunk of memory?

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel