[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] PCI passthrough - pci interrupt gets disabled



Hi there,

I'm running Xen 3.2.1-rc1-pre on a Ubuntu 8.0.4 server (kernel 2.6.24-19-xen).

I have some network cards which are passed through to the DomUs. 'Normally' everything works quite well. but sometimes when a DomU crashes or has to be shutdown, the pci interrupt of that DomU gets disabled in Dom0. After that happend there is no way (at least I haven't figured out how to do it) to use that network card again in any DomU or Dom0. Even rebooting the Dom0 does not help. It has to be turned off and on again to revive that pci interrupt.

root@XenDom:/boot/grub# lspci -nn
00:00.0 Host bridge [0600]: Intel Corporation 5000P Chipset Memory Controller Hub [8086:25d8] (rev b1)
00:02.0 PCI bridge [0604]: Intel Corporation 5000 Series Chipset PCI Express x8 Port 2-3 [8086:25f7] (rev b1)
00:04.0 PCI bridge [0604]: Intel Corporation 5000 Series Chipset PCI Express x8 Port 4-5 [8086:25f8] (rev b1)
00:06.0 PCI bridge [0604]: Intel Corporation 5000 Series Chipset PCI Express x8 Port 6-7 [8086:25f9] (rev b1)
00:08.0 System peripheral [0880]: Intel Corporation 5000 Series Chipset DMA Engine [8086:1a38] (rev b1)
00:10.0 Host bridge [0600]: Intel Corporation 5000 Series Chipset FSB Registers [8086:25f0] (rev b1)
00:10.1 Host bridge [0600]: Intel Corporation 5000 Series Chipset FSB Registers [8086:25f0] (rev b1)
00:10.2 Host bridge [0600]: Intel Corporation 5000 Series Chipset FSB Registers [8086:25f0] (rev b1)
00:11.0 Host bridge [0600]: Intel Corporation 5000 Series Chipset Reserved Registers [8086:25f1] (rev b1)
00:13.0 Host bridge [0600]: Intel Corporation 5000 Series Chipset Reserved Registers [8086:25f3] (rev b1)
00:15.0 Host bridge [0600]: Intel Corporation 5000 Series Chipset FBD Registers [8086:25f5] (rev b1)
00:16.0 Host bridge [0600]: Intel Corporation 5000 Series Chipset FBD Registers [8086:25f6] (rev b1)
00:1c.0 PCI bridge [0604]: Intel Corporation 631xESB/632xESB/3100 Chipset PCI Express Root Port 1 [8086:2690] (rev 09)
00:1d.0 USB Controller [0c03]: Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #1 [8086:2688] (rev 09)
00:1d.1 USB Controller [0c03]: Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #2 [8086:2689] (rev 09)
00:1d.2 USB Controller [0c03]: Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #3 [8086:268a] (rev 09)
00:1d.7 USB Controller [0c03]: Intel Corporation 631xESB/632xESB/3100 Chipset EHCI USB2 Controller [8086:268c] (rev 09)
00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev d9)
00:1f.0 ISA bridge [0601]: Intel Corporation 631xESB/632xESB/3100 Chipset LPC Interface Controller [8086:2670] (rev 09)
00:1f.1 IDE interface [0101]: Intel Corporation 631xESB/632xESB IDE Controller [8086:269e] (rev 09)
00:1f.2 IDE interface [0101]: Intel Corporation 631xESB/632xESB/3100 Chipset SATA IDE Controller [8086:2680] (rev 09)
00:1f.3 SMBus [0c05]: Intel Corporation 631xESB/632xESB/3100 Chipset SMBus Controller [8086:269b] (rev 09)
01:00.0 PCI bridge [0604]: Intel Corporation 6311ESB/6321ESB PCI Express Upstream Port [8086:3500] (rev 01)
01:00.3 PCI bridge [0604]: Intel Corporation 6311ESB/6321ESB PCI Express to PCI-X Bridge [8086:350c] (rev 01)
02:00.0 PCI bridge [0604]: Intel Corporation 6311ESB/6321ESB PCI Express Downstream Port E1 [8086:3510] (rev 01)
02:02.0 PCI bridge [0604]: Intel Corporation 6311ESB/6321ESB PCI Express Downstream Port E3 [8086:3518] (rev 01)
03:00.0 PCI bridge [0604]: Intel Corporation 6700PXH PCI Express-to-PCI Bridge A [8086:0329] (rev 09)
03:00.2 PCI bridge [0604]: Intel Corporation 6700PXH PCI Express-to-PCI Bridge B [8086:032a] (rev 09)
06:00.0 Ethernet controller [0200]: Intel Corporation 80003ES2LAN Gigabit Ethernet Controller (Copper) [8086:1096] (rev 01)
06:00.1 Ethernet controller [0200]: Intel Corporation 80003ES2LAN Gigabit Ethernet Controller (Copper) [8086:1096] (rev 01)
08:00.0 PCI bridge [0604]: Intel Corporation 80333 Segment-A PCI Express-to-PCI Express Bridge [8086:0370]
08:00.2 PCI bridge [0604]: Intel Corporation 80333 Segment-B PCI Express-to-PCI Express Bridge [8086:0372]
09:0e.0 RAID bus controller [0104]: Areca Technology Corp. ARC-1220 8-Port PCI-Express to SATA RAID Controller [17d3:1220]
0c:00.0 PCI bridge [0604]: Integrated Device Technology, Inc. PES12N3A PCI Express Switch [111d:8018] (rev 04)
0d:00.0 PCI bridge [0604]: Integrated Device Technology, Inc. PES12N3A PCI Express Switch [111d:8018] (rev 04)
0d:01.0 PCI bridge [0604]: Integrated Device Technology, Inc. PES12N3A PCI Express Switch [111d:8018] (rev 04)
0e:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06)
0e:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06)
0f:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06)
0f:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06)
10:01.0 VGA compatible controller [0300]: ATI Technologies Inc ES1000 [1002:515e] (rev 02)

root@XenDom:/boot/grub# dmesg | grep GSI | sort -u
[    0.174640] ACPI: PCI Interrupt 0000:0f:00.1[A] -> GSI 17 (level, low) -> IRQ 18
[    0.175082] ACPI: PCI Interrupt 0000:0f:00.0[B] -> GSI 18 (level, low) -> IRQ 17
[    0.175533] ACPI: PCI Interrupt 0000:0e:00.1[A] -> GSI 16 (level, low) -> IRQ 16
[    0.176047] ACPI: PCI Interrupt 0000:0e:00.0[B] -> GSI 17 (level, low) -> IRQ 18
[    0.176458] ACPI: PCI Interrupt 0000:06:00.1[B] -> GSI 19 (level, low) -> IRQ 19
[    0.841866] IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
[    0.841871] IOAPIC[1]: apic_id 9, version 32, address 0xfec80000, GSI 24-47
[    0.868206] ACPI: PCI Interrupt 0000:06:00.0[A] -> GSI 18 (level, low) -> IRQ 17
[    0.982087] ACPI: PCI Interrupt 0000:00:1d.0[A] -> GSI 17 (level, low) -> IRQ 18
[    1.087619] ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 19 (level, low) -> IRQ 19
[    1.191635] ACPI: PCI Interrupt 0000:00:1d.2[C] -> GSI 18 (level, low) -> IRQ 17
[    1.295767] ACPI: PCI Interrupt 0000:00:1d.7[A] -> GSI 17 (level, low) -> IRQ 18
[  139.692701] ACPI: PCI Interrupt 0000:06:00.1[B] -> GSI 19 (level, low) -> IRQ 19
[    2.029267] ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 16 (level, low) -> IRQ 16
[    2.029481] ACPI: PCI Interrupt 0000:01:00.0[A] -> GSI 16 (level, low) -> IRQ 16
[    2.029699] ACPI: PCI Interrupt 0000:02:00.0[A] -> GSI 16 (level, low) -> IRQ 16
[    2.029988] ACPI: PCI Interrupt 0000:02:02.0[A] -> GSI 18 (level, low) -> IRQ 17
[    2.030223] ACPI: PCI Interrupt 0000:00:04.0[A] -> GSI 16 (level, low) -> IRQ 16
[    2.030496] ACPI: PCI Interrupt 0000:00:06.0[A] -> GSI 16 (level, low) -> IRQ 16
[    2.030744] ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI 17 (level, low) -> IRQ 18
[    2.907697] ACPI: PCI Interrupt 0000:09:0e.0[A] -> GSI 18 (level, low) -> IRQ 17
[    3.408474] ACPI: PCI Interrupt 0000:00:1f.1[A] -> GSI 18 (level, low) -> IRQ 17
[    3.408868] ACPI: PCI Interrupt 0000:00:1f.2[B] -> GSI 19 (level, low) -> IRQ 19
[    3.411717] ACPI: PCI Interrupt 0000:00:1f.1[A] -> GSI 18 (level, low) -> IRQ 17
[   37.477182] ACPI: PCI Interrupt 0000:0e:00.0[B] -> GSI 17 (level, low) -> IRQ 18
[    4.071839] ACPI: PCI Interrupt 0000:00:1f.2[B] -> GSI 19 (level, low) -> IRQ 19
[   52.152719] ACPI: PCI Interrupt 0000:0f:00.0[B] -> GSI 18 (level, low) -> IRQ 17
[   60.809215] ACPI: PCI Interrupt 0000:0f:00.1[A] -> GSI 17 (level, low) -> IRQ 18
[   69.457842] ACPI: PCI Interrupt 0000:0e:00.1[A] -> GSI 16 (level, low) -> IRQ 16

When the interrupt is disabled messages.log shows:
...
Aug 14 10:47:42 XenDom kernel: [759772.192543] blkback: ring-ref 9, event-channel 16, protocol 1 (x86_32-abi)
Aug 14 10:47:42 XenDom kernel: [759772.200813] blkback: ring-ref 10, event-channel 17, protocol 1 (x86_32-abi)
Aug 14 10:47:42 XenDom kernel: [759772.212134] blkback: ring-ref 11, event-channel 18, protocol 1 (x86_32-abi)
Aug 14 10:47:43 XenDom kernel: [759773.617958] PCI: Enabling device 0000:06:00.1 (0000 -> 0003)
Aug 14 10:47:43 XenDom kernel: [759773.617983] ACPI: PCI Interrupt 0000:06:00.1[B] -> GSI 19 (level, low) -> IRQ 19
Aug 14 10:50:29 XenDom kernel: [759939.086937] ACPI: PCI interrupt for device 0000:06:00.1 disabled
...

Does anybody know how to prevent DomUs from shutting down shared interrupts?
I've tried the noirqdebug kernel option already with no luck.

Thanks for your help, Rado


Trete mit Deinem inneren Athleten in Verbindung. Beantworte die Fragen!
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.