Hello,
since an upgrade from Debian lenny, which was running on our
Dell-Server since 2009, we are now having problems with the
Passthrough of an PCI-USB-Controller which shares its IRQ 11 with 2
other PCI-Devices.
A PCI-Device whichs IRQ 35 is not shared can be attached to the
guest and works perfectly. This new behavior with the IRQ-shared
USB-Controller has begun after the update of DomO and DomU to Debian
squeeze/stable. The former configuration was:
DomO:
Kernel: 2.6.26-2-xen-amd64 (2.6.26-26lenny1)
Xen-Hypervisor: 3.2-1-amd64 (3.2.1-2)
DomU:
Kernel: linux-image-2.6.26-2-xen-686 (2.6.26-26lenny3)
Running DomU with the the old 2.6.26 Kernel doesn't solve the
problem. Running DomO with the old configuration would be a greater
downgrade because of incompatibilities between drbd-modules and
drbd-userlevel tools, which needs old lenny archives.
The actual Xen running now is from Debian's stable squeeze
release with DomO and DomU Kernel 2.6.32-5-xen:
DomO:
#uname -a
Linux arche1 2.6.32-5-xen-amd64 #1 SMP Tue Jun 14 12:46:30
UTC 2011 x86_64 GNU/Linux
#cat /proc/cmdline
placeholder root=UUID=6361b25e-fde8-4e33-9599-7fd807fefe4d
ro xen-pciback.hide=(00:03.0)
DomU:
#uname -a
Linux slspaddev 2.6.32-5-xen-686 #1 SMP Mon Jun 13 09:07:50
UTC 2011 i686 GNU/Linux
#cat /proc/cmdline
root=/dev/xvda1 ro console=tty0 iommu=soft swiotlb=force
DomO:
#xm info
host : arche1
release : 2.6.32-5-xen-amd64
version : #1 SMP Tue Jun 14 12:46:30 UTC 2011
machine : x86_64
nr_cpus : 8
nr_nodes : 2
cores_per_socket : 4
threads_per_core : 1
cpu_mhz : 2294
hw_caps :
178bf3ff:efd3fbff:00000000:00001310:00802001:00000000:000037ff:00000000
virt_caps : hvm
total_memory : 16378
free_memory : 1358
node_to_cpu : node0:0,2,4,6
node1:1,3,5,7
node_to_memory : node0:488
node1:869
node_to_dma32_mem : node0:3
node1:0
max_node_id : 1
xen_major : 4
xen_minor : 0
xen_extra : .1
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p
hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
xen_commandline : placeholder
cc_compiler : gcc version 4.4.5 (Debian 4.4.5-10)
cc_compile_by : waldi
cc_compile_domain : debian.org
cc_compile_date : Wed Jan 12 14:04:06 UTC 2011
xend_config_format : 4
#xm dmesg
(XEN) Xen version 4.0.1 (Debian 4.0.1-2) (waldi@xxxxxxxxxx) (gcc
version 4.4.5 (Debian 4.4.5-10) ) Wed Jan 12 14:04:06 UTC 2011
(XEN) Bootloader: GRUB 1.98+20100804-14
(XEN) Command line: placeholder
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: none; EDID transfer time: 2 seconds
(XEN) EDID info not retrieved because no DDC retrieval method
detected
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 00000000000a0000 (usable)
(XEN) 0000000000100000 - 00000000bfac0000 (usable)
(XEN) 00000000bfac0000 - 00000000bfad6000 (reserved)
(XEN) 00000000bfad6000 - 00000000bfaf5c00 (ACPI data)
(XEN) 00000000bfaf5c00 - 00000000c0000000 (reserved)
(XEN) 00000000f0000000 - 00000000f8000000 (reserved)
(XEN) 00000000fe000000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 0000000440000000 (usable)
(XEN) ACPI: RSDP 000F2210, 0024 (r2 DELL )
(XEN) ACPI: XSDT 000F228C, 008C (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: FACP BFAF1BD8, 00F4 (r3 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: DSDT BFAD6000, 476A (r1 DELL PE_SC3 1 INTL
20050624)
(XEN) ACPI: FACS BFAF4400, 0040
(XEN) ACPI: APIC BFAF1878, 00A0 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: SPCR BFAF191C, 0050 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: HPET BFAF1970, 0038 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: MCFG BFAF19AC, 003C (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: SLIC BFAF19EC, 0024 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: ERST BFADA8EC, 0210 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: HEST BFADAAFC, 027C (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: BERT BFADA76C, 0030 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: EINJ BFADA79C, 0150 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: SRAT 000FC144, 0150 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: SSDT BFAF4800, 10F4 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: TCPA BFAF1B70, 0064 (r1 DELL PE_SC3 1
DELL 1)
(XEN) System RAM: 16378MB (16771456kB)
(XEN) Domain heap initialised DMA width 31 bits
(XEN) Processor #0 0:4 APIC version 16
(XEN) Processor #4 0:4 APIC version 16
(XEN) Processor #1 0:4 APIC version 16
(XEN) Processor #5 0:4 APIC version 16
(XEN) Processor #2 0:4 APIC version 16
(XEN) Processor #6 0:4 APIC version 16
(XEN) Processor #3 0:4 APIC version 16
(XEN) Processor #7 0:4 APIC version 16
(XEN) IOAPIC[0]: apic_id 8, version 17, address 0xfec00000, GSI
0-15
(XEN) IOAPIC[1]: apic_id 9, version 17, address 0xfec01000, GSI
32-47
(XEN) IOAPIC[2]: apic_id 10, version 17, address 0xfec02000, GSI
64-79
(XEN) Enabling APIC mode: Flat. Using 3 I/O APICs
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2294.328 MHz processor.
(XEN) Initing memory sharing.
(XEN) HVM: ASIDs enabled.
(XEN) HVM: SVM enabled
(XEN) HVM: Hardware Assisted Paging detected.
(XEN) AMD-Vi: IOMMU not found!
(XEN) I/O virtualisation disabled
(XEN) Total of 8 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) TSC is reliable, synchronization unnecessary
(XEN) Platform timer appears to have unexpectedly wrapped 1 times.
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) Brought up 8 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 ->
0x16ba000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000230000000->0000000238000000
(4080652 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff816ba000
(XEN) Init. ramdisk: ffffffff816ba000->ffffffff832d7200
(XEN) Phys-Mach map: ffffffff832d8000->ffffffff8523a060
(XEN) Start info: ffffffff8523b000->ffffffff8523b4b4
(XEN) Page tables: ffffffff8523c000->ffffffff85269000
(XEN) Boot stack: ffffffff85269000->ffffffff8526a000
(XEN) TOTAL: ffffffff80000000->ffffffff85400000
(XEN) ENTRY ADDRESS: ffffffff81509200
(XEN) Dom0 has maximum 8 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to
switch input to Xen)
(XEN) Freed 176kB init memory.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010004 from
0000995d:bf92b427 to 00000000:00000000.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010000 from
00000306:851554ec to 00000000:00430076.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(the guest DomU is running at this moment, but the PCI-Device is not
attached yet. The upper output of "xm dmesg" does not change between
xen with no running guest
and starting a single guest)
As you can see from the DomO's Kernel-parameter "xen-pciback.hide=(00:03.0)"
the PCI Device to be passed through has the ID 00:03.0 .
An extract from # lspci -vv on DomO shows this Device with the
others sharing IRQ 11:
DomO:
# lspci -vv
...
00:03.0 USB Controller: Broadcom BCM5785 [HT1000] USB (rev 01)
(prog-if 10 [OHCI])
Subsystem: Dell PowerEdge 2970 HT1000 USB Controller
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV-
VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium
>TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 11
Region 0: Memory at ee4ed000 (32-bit, non-prefetchable)
[disabled] [size=4K]
Region 1: I/O ports at b000 [disabled] [size=256]
Capabilities: [dc] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0
PME-
Bridge: PM- B3+
Kernel driver in use: pciback
00:03.1 USB Controller: Broadcom BCM5785 [HT1000] USB (rev 01)
(prog-if 10 [OHCI])
Subsystem: Dell PowerEdge 2970 HT1000 USB Controller
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+
VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium
>TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 64, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 11
Region 0: Memory at ee4ee000 (32-bit, non-prefetchable)
[size=4K]
Region 1: I/O ports at b400 [size=256]
Capabilities: [dc] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0
PME-
Bridge: PM- B3+
Kernel driver in use: ohci_hcd
00:03.2 USB Controller: Broadcom BCM5785 [HT1000] USB (rev 01)
(prog-if 20 [EHCI])
Subsystem: Dell PowerEdge 2970 HT1000 USB Controller
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+
VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium
>TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 64, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 11
Region 0: Memory at ee4ef000 (32-bit, non-prefetchable)
[size=4K]
Region 1: I/O ports at b800 [size=256]
Capabilities: [dc] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA
PME(D0+,D1+,D2+,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0
PME-
Bridge: PM- B3+
Kernel driver in use: ehci_hcd
...
The PCI-Device with the unshared IRQ 35 works after attaching it to
a guest and looks like this in lspci -vv:
...
0c:00.0 SCSI storage controller: LSI Logic / Symbios Logic
SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
Subsystem: Hewlett-Packard Company SC44Ge Host Bus Adapter
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast
>TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 35
Region 0: I/O ports at cc00 [size=256]
Region 1: Memory at ee3ec000 (64-bit, non-prefetchable)
[size=16K]
Region 3: Memory at ee3f0000 (64-bit, non-prefetchable)
[size=64K]
Expansion ROM at ee000000 [disabled] [size=2M]
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [68] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 4096 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal+
Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 512 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq-
AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x8, ASPM L0s L1,
Latency L0 <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk-
DLActive- BWMgmt- ABWMgmt-
Capabilities: [98] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [b0] MSI-X: Enable- Count=1 Masked-
Vector table: BAR=1 offset=00002000
PBA: BAR=1 offset=00003000
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO+ CmpltAbrt+
UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UESvrt: DLP+ SDES- TLP+ FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr-
CEMsk: RxErr- BadTLP+ BadDLLP+ Rollover+ Timeout+
NonFatalErr-
AERCap: First Error Pointer: 00, GenCap+ CGenEn-
ChkCap+ ChkEn-
Kernel driver in use: pciback
...
I added this SAS-Controller manually after booting to the pciback
slots for demonstration. Its not part of the DomO's xen-pciback.hide
Kernel-parameter.
The interrupts on DomO are as follows. No PCI Device is attached to
the already running guest domU yet.
DomO:
# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
CPU4 CPU5 CPU6 CPU7
1: 2 0 0 0
0 0 0 0 xen-pirq-ioapic-edge i8042
8: 0 0 0 0
0 0 0 0 xen-pirq-ioapic-edge rtc0
9: 0 0 0 0
0 0 0 0 xen-pirq-ioapic-edge acpi
10: 94 0 0 0
0 0 0 0 xen-pirq-ioapic-edge sata_svw
11: 93 0 0 0
0 0 0 0 xen-pirq-ioapic-edge
ehci_hcd:usb1, ohci_hcd:usb2
12: 4 0 0 0
0 0 0 0 xen-pirq-ioapic-edge i8042
35: 66 0 0 0
0 0 0 0 xen-pirq-ioapic-level
37: 51283 0 0 0
0 0 0 0 xen-pirq-ioapic-level megasas
2237: 471 0 0 0
0 0 0 0 xen-dyn-event vif1.0
2238: 106 0 0 0
0 0 0 0 xen-dyn-event
blkif-backend
2239: 217 0 0 0
0 0 0 0 xen-dyn-event
blkif-backend
2240: 2749 0 0 0
0 0 0 0 xen-dyn-event
blkif-backend
2241: 16647 0 0 0
0 0 0 0 xen-dyn-event
blkif-backend
2242: 2408 0 0 0
0 0 0 0 xen-dyn-event
blkif-backend
2243: 1 0 0 0
0 0 0 0 xen-dyn-event
evtchn:qemu-dm
2244: 612700 0 0 0
0 0 0 0 xen-dyn-event
evtchn:qemu-dm
2245: 154 0 0 0
0 0 0 0 xen-dyn-event
evtchn:qemu-dm
2246: 593 0 0 0
0 0 0 0 xen-dyn-event
evtchn:xenstored
2247: 0 0 0 0
0 0 0 0 xen-dyn-event
evtchn:xenstored
2248: 1406 0 0 0
0 0 0 0 xen-dyn-event
evtchn:xenstored
2249: 10065 0 0 0
0 0 0 0 xen-pirq-msi peth0
2250: 90713 0 0 0
0 0 0 0 xen-pirq-msi eth1
2252: 0 0 0 0
0 0 0 0 xen-percpu-virq mce
2257: 0 0 0 0
0 0 0 0 xen-pirq-msi aerdrv
2258: 0 0 0 0
0 0 0 0 xen-pirq-msi aerdrv
2259: 0 0 0 0
0 0 0 0 xen-pirq-msi aerdrv
2260: 0 0 0 0
0 0 0 0 xen-pirq-msi aerdrv
2261: 0 0 0 0
0 0 0 0 xen-pirq-msi aerdrv
2262: 0 0 0 0
0 0 0 0 xen-percpu-virq pcpu
2263: 2605 0 0 0
0 0 0 0 xen-dyn-event xenbus
2264: 0 0 0 0
0 0 0 846 xen-percpu-ipi
callfuncsingle7
2265: 0 0 0 0
0 0 0 0 xen-percpu-virq debug7
2266: 0 0 0 0
0 0 0 74 xen-percpu-ipi callfunc7
2267: 0 0 0 0
0 0 0 17429 xen-percpu-ipi resched7
2268: 0 0 0 0
0 0 0 7268 xen-percpu-virq timer7
2269: 0 0 0 0
0 0 1250 0 xen-percpu-ipi
callfuncsingle6
2270: 0 0 0 0
0 0 0 0 xen-percpu-virq debug6
2271: 0 0 0 0
0 0 93 0 xen-percpu-ipi callfunc6
2272: 0 0 0 0
0 0 4901 0 xen-percpu-ipi resched6
2273: 0 0 0 0
0 0 35074 0 xen-percpu-virq timer6
2274: 0 0 0 0 0
2892 0 0 xen-percpu-ipi callfuncsingle5
2275: 0 0 0 0
0 0 0 0 xen-percpu-virq debug5
2276: 0 0 0 0
0 93 0 0 xen-percpu-ipi callfunc5
2277: 0 0 0 0 0
19826 0 0 xen-percpu-ipi resched5
2278: 0 0 0 0 0
6993 0 0 xen-percpu-virq timer5
2279: 0 0 0 0
3615 0 0 0 xen-percpu-ipi
callfuncsingle4
2280: 0 0 0 0
0 0 0 0 xen-percpu-virq debug4
2281: 0 0 0 0
104 0 0 0 xen-percpu-ipi
callfunc4
2282: 0 0 0 0
20926 0 0 0 xen-percpu-ipi
resched4
2283: 0 0 0 0
9741 0 0 0 xen-percpu-virq timer4
2284: 0 0 0 1327
0 0 0 0 xen-percpu-ipi
callfuncsingle3
2285: 0 0 0 0
0 0 0 0 xen-percpu-virq debug3
2286: 0 0 0 92
0 0 0 0 xen-percpu-ipi callfunc3
2287: 0 0 0 17738
0 0 0 0 xen-percpu-ipi resched3
2288: 0 0 0 12742
0 0 0 0 xen-percpu-virq timer3
2289: 0 0 3016 0
0 0 0 0 xen-percpu-ipi
callfuncsingle2
2290: 0 0 0 0
0 0 0 0 xen-percpu-virq debug2
2291: 0 0 95 0
0 0 0 0 xen-percpu-ipi callfunc2
2292: 0 0 20282 0
0 0 0 0 xen-percpu-ipi resched2
2293: 0 0 20647 0
0 0 0 0 xen-percpu-virq timer2
2294: 0 4890 0 0
0 0 0 0 xen-percpu-ipi
callfuncsingle1
2295: 0 0 0 0
0 0 0 0 xen-percpu-virq debug1
2296: 0 81 0 0
0 0 0 0 xen-percpu-ipi callfunc1
2297: 0 24716 0 0
0 0 0 0 xen-percpu-ipi resched1
2298: 0 21358 0 0
0 0 0 0 xen-percpu-virq timer1
2299: 257 0 0 0
0 0 0 0 xen-percpu-ipi
callfuncsingle0
2300: 0 0 0 0
0 0 0 0 xen-percpu-virq debug0
2301: 89 0 0 0
0 0 0 0 xen-percpu-ipi callfunc0
2302: 25100 0 0 0
0 0 0 0 xen-percpu-ipi resched0
2303: 291909 0 0 0
0 0 0 0 xen-percpu-virq timer0
NMI: 0 0 0 0
0 0 0 0 Non-maskable interrupts
LOC: 0 0 0 0
0 0 0 0 Local timer interrupts
SPU: 0 0 0 0
0 0 0 0 Spurious interrupts
PMI: 0 0 0 0
0 0 0 0 Performance monitoring
interrupts
PND: 0 0 0 0
0 0 0 0 Performance pending work
RES: 25100 24716 20282 17738 20927
19826 4901 17429 Rescheduling interrupts
CAL: 346 4971 3111 1419 3719
2985 1343 920 Function call interrupts
TLB: 0 0 0 0
0 0 0 0 TLB shootdowns
TRM: 0 0 0 0
0 0 0 0 Thermal event interrupts
THR: 0 0 0 0
0 0 0 0 Threshold APIC interrupts
MCE: 0 0 0 0
0 0 0 0 Machine check exceptions
MCP: 12 12 12 12
12 12 12 12 Machine check polls
ERR: 0
MIS: 0
Now I'm attaching the USB-Controller to the guest. The device is
available for xen:
DomO:
# xm pci-list-assignable
0000:00:03.0
0000:0c:00.0
# xm pci-attach slspaddev 0000:00:03.0
dmesg says now:
DomO:
[ 3859.997973] pciback: vpci: 0000:00:03.0: assign
to virtual slot 0
[ 3860.004621] alloc irq_desc for 2236 on node -1
[ 3860.004630] alloc kstat_irqs on node -1
[ 3860.180173] pciback 0000:00:03.0: enabling device (0000
-> 0003)
[ 3860.180240] xen: registering gsi 11 triggering 0 polarity 1
[ 3860.180249] xen_allocate_pirq: returning irq 11 for gsi 11
[ 3860.180296] xen: --> irq=11
[ 3860.180307] Already setup the GSI :11
[ 3860.180352] pciback 0000:00:03.0: PCI INT A ->
Link[LK01] -> GSI 11 (level, low) -> IRQ 11
[ 3860.375909] pciback[0000:00:03.0] IRQ line is not shared
with other domains. Turning ISR off
[ 3862.938422] irq 11: nobody cared (try booting with the
"irqpoll" option)
[ 3862.938464] Pid: 0, comm: swapper Tainted: G W
2.6.32-5-xen-amd64 #1
[ 3862.938503] Call Trace:
[ 3862.938532] <IRQ> [<ffffffff811fa808>] ?
pciback_guest_interrupt+0x14/0x6c
[ 3862.938583] [<ffffffff81096b3d>] ?
__report_bad_irq+0x30/0x7d
[ 3862.938620] [<ffffffff81096c8f>] ?
note_interrupt+0x105/0x16e
[ 3862.938658] [<ffffffff81097396>] ?
handle_level_irq+0x80/0xc3
[ 3862.938696] [<ffffffff811f1e0e>] ?
__xen_evtchn_do_upcall+0xe1/0x175
[ 3862.938735] [<ffffffff811f245c>] ?
xen_evtchn_do_upcall+0x2e/0x42
[ 3862.938774] [<ffffffff81012cfe>] ?
xen_do_hypervisor_callback+0x1e/0x30
[ 3862.938811] <EOI> [<ffffffff810093aa>] ?
hypercall_page+0x3aa/0x1001
[ 3862.938853] [<ffffffff810093aa>] ?
hypercall_page+0x3aa/0x1001
[ 3862.938891] [<ffffffff8100e6a7>] ?
xen_safe_halt+0xc/0x15
[ 3862.938927] [<ffffffff8100bfc7>] ?
xen_idle+0x37/0x40
[ 3862.938964] [<ffffffff81010eb1>] ?
cpu_idle+0xa2/0xda
[ 3862.939001] [<ffffffff81509cdd>] ?
start_kernel+0x3dc/0x3e8
[ 3862.939038] [<ffffffff8150bc93>] ?
xen_start_kernel+0x586/0x58a
[ 3862.939073] handlers:
[ 3862.939101] [<ffffffffa001e864>]
(usb_hcd_irq+0x0/0x7e [usbcore])
[ 3862.939162] [<ffffffffa001e864>]
(usb_hcd_irq+0x0/0x7e [usbcore])
[ 3862.939206] [<ffffffff811fa7f4>]
(pciback_guest_interrupt+0x0/0x6c)
[ 3862.939248] Disabling IRQ #11
# xm debug-keys i
# xm dmesg
(XEN) Xen version 4.0.1 (Debian 4.0.1-2) (waldi@xxxxxxxxxx) (gcc
version 4.4.5 (Debian 4.4.5-10) ) Wed Jan 12 14:04:06 UTC 2011
(XEN) Bootloader: GRUB 1.98+20100804-14
(XEN) Command line: placeholder
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: none; EDID transfer time: 2 seconds
(XEN) EDID info not retrieved because no DDC retrieval method
detected
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 00000000000a0000 (usable)
(XEN) 0000000000100000 - 00000000bfac0000 (usable)
(XEN) 00000000bfac0000 - 00000000bfad6000 (reserved)
(XEN) 00000000bfad6000 - 00000000bfaf5c00 (ACPI data)
(XEN) 00000000bfaf5c00 - 00000000c0000000 (reserved)
(XEN) 00000000f0000000 - 00000000f8000000 (reserved)
(XEN) 00000000fe000000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 0000000440000000 (usable)
(XEN) ACPI: RSDP 000F2210, 0024 (r2 DELL )
(XEN) ACPI: XSDT 000F228C, 008C (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: FACP BFAF1BD8, 00F4 (r3 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: DSDT BFAD6000, 476A (r1 DELL PE_SC3 1 INTL
20050624)
(XEN) ACPI: FACS BFAF4400, 0040
(XEN) ACPI: APIC BFAF1878, 00A0 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: SPCR BFAF191C, 0050 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: HPET BFAF1970, 0038 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: MCFG BFAF19AC, 003C (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: SLIC BFAF19EC, 0024 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: ERST BFADA8EC, 0210 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: HEST BFADAAFC, 027C (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: BERT BFADA76C, 0030 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: EINJ BFADA79C, 0150 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: SRAT 000FC144, 0150 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: SSDT BFAF4800, 10F4 (r1 DELL PE_SC3 1
DELL 1)
(XEN) ACPI: TCPA BFAF1B70, 0064 (r1 DELL PE_SC3 1
DELL 1)
(XEN) System RAM: 16378MB (16771456kB)
(XEN) Domain heap initialised DMA width 31 bits
(XEN) Processor #0 0:4 APIC version 16
(XEN) Processor #4 0:4 APIC version 16
(XEN) Processor #1 0:4 APIC version 16
(XEN) Processor #5 0:4 APIC version 16
(XEN) Processor #2 0:4 APIC version 16
(XEN) Processor #6 0:4 APIC version 16
(XEN) Processor #3 0:4 APIC version 16
(XEN) Processor #7 0:4 APIC version 16
(XEN) IOAPIC[0]: apic_id 8, version 17, address 0xfec00000, GSI
0-15
(XEN) IOAPIC[1]: apic_id 9, version 17, address 0xfec01000, GSI
32-47
(XEN) IOAPIC[2]: apic_id 10, version 17, address 0xfec02000, GSI
64-79
(XEN) Enabling APIC mode: Flat. Using 3 I/O APICs
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2294.317 MHz processor.
(XEN) Initing memory sharing.
(XEN) HVM: ASIDs enabled.
(XEN) HVM: SVM enabled
(XEN) HVM: Hardware Assisted Paging detected.
(XEN) AMD-Vi: IOMMU not found!
(XEN) I/O virtualisation disabled
(XEN) Total of 8 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) TSC is reliable, synchronization unnecessary
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) Brought up 8 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 ->
0x16ba000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000230000000->0000000238000000
(4080652 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff816ba000
(XEN) Init. ramdisk: ffffffff816ba000->ffffffff832d7200
(XEN) Phys-Mach map: ffffffff832d8000->ffffffff8523a060
(XEN) Start info: ffffffff8523b000->ffffffff8523b4b4
(XEN) Page tables: ffffffff8523c000->ffffffff85269000
(XEN) Boot stack: ffffffff85269000->ffffffff8526a000
(XEN) TOTAL: ffffffff80000000->ffffffff85400000
(XEN) ENTRY ADDRESS: ffffffff81509200
(XEN) Dom0 has maximum 8 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to
switch input to Xen)
(XEN) Freed 176kB init memory.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010004 from
0000995d:bf92b427 to 00000000:00000000.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010000 from
00000307:c50514ec to 00000000:00430076.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
(XEN) Guest interrupt information:
(XEN) IRQ: 0 affinity:ffffffff,ffffffff,ffffffff,ffffffff
vec:f0 type=IO-APIC-edge status=00000000 mapped, unbound
(XEN) IRQ: 1 affinity:00000000,00000000,00000000,00000001
vec:28 type=IO-APIC-edge status=00000014 in-flight=0
domain-list=0: 1(----),
(XEN) IRQ: 2 affinity:ffffffff,ffffffff,ffffffff,ffffffff
vec:e2 type=XT-PIC status=00000000 mapped, unbound
(XEN) IRQ: 3 affinity:00000000,00000000,00000000,000000ff
vec:30 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 4 affinity:00000000,00000000,00000000,000000ff
vec:38 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 5 affinity:00000000,00000000,00000000,00000001
vec:40 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 6 affinity:00000000,00000000,00000000,00000001
vec:48 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 7 affinity:00000000,00000000,00000000,00000001
vec:50 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 8 affinity:00000000,00000000,00000000,00000001
vec:58 type=IO-APIC-edge status=00000010 in-flight=0
domain-list=0: 8(----),
(XEN) IRQ: 9 affinity:00000000,00000000,00000000,00000001
vec:60 type=IO-APIC-edge status=00000010 in-flight=0
domain-list=0: 9(----),
(XEN) IRQ: 10 affinity:00000000,00000000,00000000,00000001
vec:68 type=IO-APIC-level status=00000010 in-flight=0
domain-list=0: 10(----),
(XEN) IRQ: 11 affinity:00000000,00000000,00000000,00000001
vec:70 type=IO-APIC-level status=00000010 in-flight=0
domain-list=0: 11(----),
(XEN) IRQ: 12 affinity:00000000,00000000,00000000,00000001
vec:78 type=IO-APIC-edge status=00000014 in-flight=0
domain-list=0: 12(----),
(XEN) IRQ: 13 affinity:00000000,00000000,00000000,000000ff
vec:88 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 14 affinity:00000000,00000000,00000000,00000001
vec:90 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 15 affinity:00000000,00000000,00000000,00000001
vec:98 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 32 affinity:00000000,00000000,00000000,000000ff
vec:a0 type=IO-APIC-level status=00000002 mapped, unbound
(XEN) IRQ: 33 affinity:00000000,00000000,00000000,000000ff
vec:a8 type=IO-APIC-level status=00000002 mapped, unbound
(XEN) IRQ: 35 affinity:00000000,00000000,00000000,00000001
vec:b8 type=IO-APIC-level status=00000002 mapped, unbound
(XEN) IRQ: 36 affinity:00000000,00000000,00000000,000000ff
vec:c0 type=IO-APIC-level status=00000002 mapped, unbound
(XEN) IRQ: 37 affinity:00000000,00000000,00000000,00000001
vec:b0 type=IO-APIC-level status=00000010 in-flight=0
domain-list=0: 37(----),
(XEN) IRQ: 39 affinity:00000000,00000000,00000000,000000ff
vec:51 type=IO-APIC-level status=00000002 mapped, unbound
(XEN) IRQ: 79 affinity:00000000,00000000,00000000,00000001
vec:c8 type=PCI-MSI status=00000010 in-flight=0
domain-list=0:334(----),
(XEN) IRQ: 80 affinity:00000000,00000000,00000000,00000001
vec:d0 type=PCI-MSI status=00000010 in-flight=0
domain-list=0:333(----),
(XEN) IRQ: 81 affinity:00000000,00000000,00000000,00000001
vec:d8 type=PCI-MSI status=00000010 in-flight=0
domain-list=0:332(----),
(XEN) IRQ: 82 affinity:00000000,00000000,00000000,00000001
vec:21 type=PCI-MSI status=00000010 in-flight=0
domain-list=0:331(----),
(XEN) IRQ: 83 affinity:00000000,00000000,00000000,00000001
vec:29 type=PCI-MSI status=00000010 in-flight=0
domain-list=0:330(----),
(XEN) IRQ: 84 affinity:ffffffff,ffffffff,ffffffff,ffffffff
vec:31 type=PCI-MSI status=00000002 mapped, unbound
(XEN) IRQ: 85 affinity:ffffffff,ffffffff,ffffffff,ffffffff
vec:39 type=PCI-MSI status=00000002 mapped, unbound
(XEN) IRQ: 86 affinity:ffffffff,ffffffff,ffffffff,ffffffff
vec:41 type=PCI-MSI status=00000002 mapped, unbound
(XEN) IRQ: 87 affinity:ffffffff,ffffffff,ffffffff,ffffffff
vec:49 type=PCI-MSI status=00000002 mapped, unbound
(XEN) IRQ: 88 affinity:00000000,00000000,00000000,00000001
vec:59 type=PCI-MSI status=00000010 in-flight=0
domain-list=0:325(----),
(XEN) IRQ: 89 affinity:00000000,00000000,00000000,00000001
vec:69 type=PCI-MSI status=00000010 in-flight=0
domain-list=0:324(----),
(XEN) IO-APIC interrupt information:
(XEN) IRQ 0 Vec240:
(XEN) Apic 0x00, Pin 2: vector=240, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=0, dest_id:255
(XEN) IRQ 1 Vec 40:
(XEN) Apic 0x00, Pin 1: vector=40, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=0, dest_id:1
(XEN) IRQ 3 Vec 48:
(XEN) Apic 0x00, Pin 3: vector=48, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=1, dest_id:255
(XEN) IRQ 4 Vec 56:
(XEN) Apic 0x00, Pin 4: vector=56, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=1, dest_id:255
(XEN) IRQ 5 Vec 64:
(XEN) Apic 0x00, Pin 5: vector=64, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=0, dest_id:1
(XEN) IRQ 6 Vec 72:
(XEN) Apic 0x00, Pin 6: vector=72, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=0, dest_id:1
(XEN) IRQ 7 Vec 80:
(XEN) Apic 0x00, Pin 7: vector=80, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=0, dest_id:1
(XEN) IRQ 8 Vec 88:
(XEN) Apic 0x00, Pin 8: vector=88, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=0, dest_id:1
(XEN) IRQ 9 Vec 96:
(XEN) Apic 0x00, Pin 9: vector=96, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=0, dest_id:1
(XEN) IRQ 10 Vec104:
(XEN) Apic 0x00, Pin 10: vector=104, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=1, irr=0,
trigger=level, mask=0, dest_id:1
(XEN) IRQ 11 Vec112:
(XEN) Apic 0x00, Pin 11: vector=112, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=1, irr=0,
trigger=level, mask=0, dest_id:1
(XEN) IRQ 12 Vec120:
(XEN) Apic 0x00, Pin 12: vector=120, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=0, dest_id:1
(XEN) IRQ 13 Vec136:
(XEN) Apic 0x00, Pin 13: vector=136, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=1, dest_id:255
(XEN) IRQ 14 Vec144:
(XEN) Apic 0x00, Pin 14: vector=144, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=0, dest_id:1
(XEN) IRQ 15 Vec152:
(XEN) Apic 0x00, Pin 15: vector=152, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=0, irr=0,
trigger=edge, mask=0, dest_id:1
(XEN) IRQ 32 Vec160:
(XEN) Apic 0x01, Pin 0: vector=160, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=1, irr=0,
trigger=level, mask=1, dest_id:255
(XEN) IRQ 33 Vec168:
(XEN) Apic 0x01, Pin 1: vector=168, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=1, irr=0,
trigger=level, mask=1, dest_id:255
(XEN) IRQ 35 Vec184:
(XEN) Apic 0x01, Pin 3: vector=184, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=1, irr=0,
trigger=level, mask=1, dest_id:1
(XEN) IRQ 36 Vec192:
(XEN) Apic 0x01, Pin 4: vector=192, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=1, irr=0,
trigger=level, mask=1, dest_id:255
(XEN) IRQ 37 Vec176:
(XEN) Apic 0x01, Pin 5: vector=176, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=1, irr=0,
trigger=level, mask=0, dest_id:1
(XEN) IRQ 39 Vec 81:
(XEN) Apic 0x01, Pin 7: vector=81, delivery_mode=1,
dest_mode=logical, delivery_status=0, polarity=1, irr=0,
trigger=level, mask=1, dest_id:255
(XEN) traps.c:2308:d1 Domain attempted WRMSR 00000000c0010048 from
00000000:00780000 to 00000000:00780400.
A bit strange is the difference between the triggers of IRQ 11
(level in xm dmesg, egde in /proc/interrupts):
in xm dmesg:
(XEN) Guest interrupt information:
(XEN) IRQ: 11 affinity:00000000,00000000,00000000,00000001
vec:70 type=IO-APIC-level status=00000010 in-flight=0
domain-list=0: 11(----),
an the line in /proc/interrupts
CPU0 CPU1 CPU2 CPU3
CPU4 CPU5 CPU6 CPU7
11: 200093 0 0 0
0 0 0 0 xen-pirq-ioapic-edge
ehci_hcd:usb1, ohci_hcd:usb2, pciback[0000:00:03.0]
while the triggers for IRQ 35 of the working SAS-Controllers look
like:
in xen dmesg
(XEN) Guest interrupt information:
(XEN) IRQ: 35 affinity:00000000,00000000,00000000,00000001
vec:b8 type=IO-APIC-level status=00000002 mapped, unbound
in /proc/interrupts:
CPU0 CPU1 CPU2 CPU3
CPU4 CPU5 CPU6 CPU7
35: 66 0 0 0
0 0 0 0 xen-pirq-ioapic-level
The complete output of /proc/interrupts on DomO follows..
DomU:
[ 2663.759919] pcifront pci-0: Installing PCI frontend
[ 2663.768458] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[ 2663.776434] pci 0000:00:00.0: reg 10 32bit mmio:
[0xee4ed000-0xee4edfff]
[ 2663.776506] pci 0000:00:00.0: reg 14 io port: [0xb000-0xb0ff]
[ 2663.780015] pcifront pci-0: claiming resource 0000:00:00.0/0
[ 2663.780015] pcifront pci-0: claiming resource 0000:00:00.0/1
[ 2663.905701] usbcore: registered new interface driver usbfs
[ 2663.909503] usbcore: registered new interface driver hub
[ 2663.911769] usbcore: registered new device driver usb
[ 2663.920062] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI)
Driver
[ 2663.927460] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI)
Driver
[ 2663.929648] ohci_hcd 0000:00:00.0: enabling device (0000 ->
0003)
[ 2663.931861] ohci_hcd 0000:00:00.0: Xen PCI enabling IRQ: 11
[ 2663.933399] xen_allocate_pirq: returning irq 11 for gsi 11
[ 2663.933399] ohci_hcd 0000:00:00.0: OHCI Host Controller
[ 2663.933399] ohci_hcd 0000:00:00.0: new USB bus registered,
assigned bus number 1
[ 2663.933399] ohci_hcd 0000:00:00.0: irq 11, io mem 0xee4ed000
[ 2663.988136] usb usb1: New USB device found, idVendor=1d6b,
idProduct=0001
[ 2663.992012] usb usb1: New USB device strings: Mfr=3, Product=2,
SerialNumber=1
[ 2663.992012] usb usb1: Product: OHCI Host Controller
[ 2663.992012] usb usb1: Manufacturer: Linux 2.6.32-5-xen-686
ohci_hcd
[ 2663.992012] usb usb1: SerialNumber: 0000:00:00.0
[ 2664.009170] usb usb1: configuration #1 chosen from 1 choice
[ 2664.010739] hub 1-0:1.0: USB hub found
[ 2664.012304] hub 1-0:1.0: 2 ports detected
[ 2664.520092] usb 1-2: new full speed USB device using ohci_hcd
and address 2
[ 2669.524083] ohci_hcd 0000:00:00.0: Unlink after no-IRQ?
Controller is probably using the wrong IRQ.
The interrupts are now as followed:
Dom0:
CPU0 CPU1 CPU2 CPU3
CPU4 CPU5 CPU6 CPU7
1: 2 0 0 0
0 0 0 0 xen-pirq-ioapic-edge i8042
8: 0 0 0 0
0 0 0 0 xen-pirq-ioapic-edge rtc0
9: 0 0 0 0
0 0 0 0 xen-pirq-ioapic-edge acpi
10: 94 0 0 0
0 0 0 0 xen-pirq-ioapic-edge sata_svw
11: 200093 0 0 0
0 0 0 0 xen-pirq-ioapic-edge
ehci_hcd:usb1, ohci_hcd:usb2, pciback[0000:00:03.0]
12: 4 0 0 0
0 0 0 0 xen-pirq-ioapic-edge i8042
35: 66 0 0 0
0 0 0 0 xen-pirq-ioapic-level
37: 57333 0 0 0
0 0 0 0 xen-pirq-ioapic-level megasas
2236: 392 0 0 0
0 0 0 0 xen-dyn-event pciback
2237: 555 0 0 0
0 0 0 0 xen-dyn-event vif1.0
2238: 106 0 0 0
0 0 0 0 xen-dyn-event
blkif-backend
2239: 217 0 0 0
0 0 0 0 xen-dyn-event
blkif-backend
2240: 2793 0 0 0
0 0 0 0 xen-dyn-event
blkif-backend
2241: 16658 0 0 0
0 0 0 0 xen-dyn-event
blkif-backend
2242: 2444 0 0 0
0 0 0 0 xen-dyn-event
blkif-backend
2243: 1 0 0 0
0 0 0 0 xen-dyn-event
evtchn:qemu-dm
2244: 622308 0 0 0
0 0 0 0 xen-dyn-event
evtchn:qemu-dm
2245: 290 0 0 0
0 0 0 0 xen-dyn-event
evtchn:qemu-dm
2246: 652 0 0 0
0 0 0 0 xen-dyn-event
evtchn:xenstored
2247: 0 0 0 0
0 0 0 0 xen-dyn-event
evtchn:xenstored
2248: 1548 0 0 0
0 0 0 0 xen-dyn-event
evtchn:xenstored
2249: 13089 0 0 0
0 0 0 0 xen-pirq-msi peth0
2250: 106066 0 0 0
0 0 0 0 xen-pirq-msi eth1
2252: 0 0 0 0
0 0 0 0 xen-percpu-virq mce
2257: 0 0 0 0
0 0 0 0 xen-pirq-msi aerdrv
2258: 0 0 0 0
0 0 0 0 xen-pirq-msi aerdrv
2259: 0 0 0 0
0 0 0 0 xen-pirq-msi aerdrv
2260: 0 0 0 0
0 0 0 0 xen-pirq-msi aerdrv
2261: 0 0 0 0
0 0 0 0 xen-pirq-msi aerdrv
2262: 0 0 0 0
0 0 0 0 xen-percpu-virq pcpu
2263: 2921 0 0 0
0 0 0 0 xen-dyn-event xenbus
2264: 0 0 0 0
0 0 0 1390 xen-percpu-ipi
callfuncsingle7
2265: 0 0 0 0
0 0 0 0 xen-percpu-virq debug7
2266: 0 0 0 0
0 0 0 74 xen-percpu-ipi callfunc7
2267: 0 0 0 0
0 0 0 22812 xen-percpu-ipi resched7
2268: 0 0 0 0
0 0 0 10217 xen-percpu-virq timer7
2269: 0 0 0 0
0 0 1445 0 xen-percpu-ipi
callfuncsingle6
2270: 0 0 0 0
0 0 0 0 xen-percpu-virq debug6
2271: 0 0 0 0
0 0 94 0 xen-percpu-ipi callfunc6
2272: 0 0 0 0
0 0 6297 0 xen-percpu-ipi resched6
2273: 0 0 0 0
0 0 45737 0 xen-percpu-virq timer6
2274: 0 0 0 0 0
3544 0 0 xen-percpu-ipi callfuncsingle5
2275: 0 0 0 0
0 0 0 0 xen-percpu-virq debug5
2276: 0 0 0 0
0 93 0 0 xen-percpu-ipi callfunc5
2277: 0 0 0 0 0
25613 0 0 xen-percpu-ipi resched5
2278: 0 0 0 0 0
21708 0 0 xen-percpu-virq timer5
2279: 0 0 0 0
4048 0 0 0 xen-percpu-ipi
callfuncsingle4
2280: 0 0 0 0
0 0 0 0 xen-percpu-virq debug4
2281: 0 0 0 0
105 0 0 0 xen-percpu-ipi
callfunc4
2282: 0 0 0 0
26843 0 0 0 xen-percpu-ipi
resched4
2283: 0 0 0 0
22882 0 0 0 xen-percpu-virq
timer4
2284: 0 0 0 1545
0 0 0 0 xen-percpu-ipi
callfuncsingle3
2285: 0 0 0 0
0 0 0 0 xen-percpu-virq debug3
2286: 0 0 0 92
0 0 0 0 xen-percpu-ipi callfunc3
2287: 0 0 0 23187
0 0 0 0 xen-percpu-ipi resched3
2288: 0 0 0 14847
0 0 0 0 xen-percpu-virq timer3
2289: 0 0 3351 0
0 0 0 0 xen-percpu-ipi
callfuncsingle2
2290: 0 0 0 0
0 0 0 0 xen-percpu-virq debug2
2291: 0 0 97 0
0 0 0 0 xen-percpu-ipi callfunc2
2292: 0 0 25053 0
0 0 0 0 xen-percpu-ipi resched2
2293: 0 0 29537 0
0 0 0 0 xen-percpu-virq timer2
2294: 0 6800 0 0
0 0 0 0 xen-percpu-ipi
callfuncsingle1
2295: 0 0 0 0
0 0 0 0 xen-percpu-virq debug1
2296: 0 83 0 0
0 0 0 0 xen-percpu-ipi callfunc1
2297: 0 31821 0 0
0 0 0 0 xen-percpu-ipi resched1
2298: 0 30837 0 0
0 0 0 0 xen-percpu-virq timer1
2299: 264 0 0 0
0 0 0 0 xen-percpu-ipi
callfuncsingle0
2300: 0 0 0 0
0 0 0 0 xen-percpu-virq debug0
2301: 89 0 0 0
0 0 0 0 xen-percpu-ipi callfunc0
2302: 32037 0 0 0
0 0 0 0 xen-percpu-ipi resched0
2303: 367301 0 0 0
0 0 0 0 xen-percpu-virq timer0
NMI: 0 0 0 0
0 0 0 0 Non-maskable interrupts
LOC: 0 0 0 0
0 0 0 0 Local timer interrupts
SPU: 0 0 0 0
0 0 0 0 Spurious interrupts
PMI: 0 0 0 0
0 0 0 0 Performance monitoring
interrupts
PND: 0 0 0 0
0 0 0 0 Performance pending work
RES: 32037 31821 25053 23187 26844
25613 6297 22812 Rescheduling interrupts
CAL: 353 6883 3448 1637 4153
3637 1539 1464 Function call interrupts
TLB: 0 0 0 0
0 0 0 0 TLB shootdowns
TRM: 0 0 0 0
0 0 0 0 Thermal event interrupts
THR: 0 0 0 0
0 0 0 0 Threshold APIC interrupts
MCE: 0 0 0 0
0 0 0 0 Machine check exceptions
MCP: 15 15 15 15
15 15 15 15 Machine check polls
ERR: 0
MIS: 0
DomU:
CPU0
11: 0 xen-pirq-xt-pic ohci_hcd:usb1
1520: 2 xen-dyn-event pcifront
1521: 5345 xen-dyn-event eth0
1522: 132 xen-dyn-event blkif
1523: 525 xen-dyn-event blkif
1524: 5766 xen-dyn-event blkif
1525: 17698 xen-dyn-event blkif
1526: 3013 xen-dyn-event blkif
1527: 0 xen-dyn-event vkbd
1528: 23 xen-dyn-event hvc_console
1529: 609264 xen-dyn-event vfb
1530: 1042 xen-dyn-event xenbus
1531: 0 xen-percpu-ipi callfuncsingle0
1532: 0 xen-percpu-virq debug0
1533: 0 xen-percpu-ipi callfunc0
1534: 0 xen-percpu-ipi resched0
1535: 32229 xen-percpu-virq timer0
NMI: 0 Non-maskable interrupts
LOC: 0 Local timer interrupts
SPU: 0 Spurious interrupts
PMI: 0 Performance monitoring interrupts
PND: 0 Performance pending work
RES: 0 Rescheduling interrupts
CAL: 0 Function call interrupts
TLB: 0 TLB shootdowns
TRM: 0 Thermal event interrupts
THR: 0 Threshold APIC interrupts
MCE: 0 Machine check exceptions
MCP: 0 Machine check polls
ERR: 0
MIS: 0
The USB-Controller is visible on DomU:
DomU:
# lspci -vv
00:00.0 USB Controller: Broadcom BCM5785 [HT1000] USB (rev 01)
(prog-if 10 [OHCI])
Subsystem: Dell PowerEdge 2970 HT1000 USB Controller
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium
>TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx+
Latency: 64, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 11
Region 0: Memory at ee4ed000 (32-bit, non-prefetchable)
[size=4K]
Region 1: I/O ports at b000 [size=256]
Capabilities: [dc] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Bridge: PM- B3+
Kernel driver in use: ohci_hcd
but lsusb stalls and produces messages:
[ 2880.040113] INFO: task khubd:1651 blocked for more than 120
seconds.
[ 2880.044042] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2880.044042] khubd D e0c53005 0 1651 2
0x00000000
[ 2880.044042] de8ba200 00000246 e0c4655d e0c53005 e0c92818
c145ee20 c145ee20 c145a354
[ 2880.044042] de8ba3bc c2f34e20 00000000 c21f02e8 00000000
e0c92a2c c21fa400 c233c100
[ 2880.044042] fffffffe de8ba3bc 00090af7 00000000 c233c100
00000000 df9c7e3c e0c563c4
[ 2880.044042] Call Trace:
[ 2880.044042] [<e0c4655d>] ?
usb_hcd_check_unlink_urb+0x65/0x81 [usbcore]
[ 2880.044042] [<e0c47f23>] ? usb_kill_urb+0x84/0xa0
[usbcore]
[ 2880.044042] [<c104b65a>] ?
autoremove_wake_function+0x0/0x2d
[ 2880.044042] [<e0c48f4c>] ? usb_start_wait_urb+0x62/0x92
[usbcore]
[ 2880.044042] [<e0c48389>] ? usb_init_urb+0x1a/0x26
[usbcore]
[ 2880.044042] [<e0c49187>] ? usb_control_msg+0xf7/0x111
[usbcore]
[ 2880.044042] [<c128dac3>] ? printk+0xe/0x13
[ 2880.044042] [<e0c43c46>] ? hub_port_init+0x2cd/0x650
[usbcore]
[ 2880.044042] [<c1006764>] ? check_events+0x8/0xc
[ 2880.044042] [<e0c45b1e>] ? hub_thread+0x825/0xe77
[usbcore]
[ 2880.044042] [<c10345f2>] ? try_to_wake_up+0x2a7/0x2b3
[ 2880.044042] [<c104b65a>] ?
autoremove_wake_function+0x0/0x2d
[ 2880.044042] [<e0c452f9>] ? hub_thread+0x0/0xe77
[usbcore]
[ 2880.044042] [<c104b428>] ? kthread+0x61/0x66
[ 2880.044042] [<c104b3c7>] ? kthread+0x0/0x66
[ 2880.044042] [<c1009ae7>] ? kernel_thread_helper+0x7/0x10
[ 3000.040096] INFO: task khubd:1651 blocked for more than 120
seconds.
[ 3000.044032] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3000.044032] khubd D e0c53005 0 1651 2
0x00000000
[ 3000.044032] de8ba200 00000246 e0c4655d e0c53005 e0c92818
c145ee20 c145ee20 c145a354
[ 3000.044032] de8ba3bc c2f34e20 00000000 c21f02e8 00000000
e0c92a2c c21fa400 c233c100
[ 3000.044032] fffffffe de8ba3bc 00090af7 00000000 c233c100
00000000 df9c7e3c e0c563c4
[ 3000.044032] Call Trace:
[ 3000.044032] [<e0c4655d>] ?
usb_hcd_check_unlink_urb+0x65/0x81 [usbcore]
[ 3000.044032] [<e0c47f23>] ? usb_kill_urb+0x84/0xa0
[usbcore]
[ 3000.044032] [<c104b65a>] ?
autoremove_wake_function+0x0/0x2d
[ 3000.044032] [<e0c48f4c>] ? usb_start_wait_urb+0x62/0x92
[usbcore]
[ 3000.044032] [<e0c48389>] ? usb_init_urb+0x1a/0x26
[usbcore]
[ 3000.044032] [<e0c49187>] ? usb_control_msg+0xf7/0x111
[usbcore]
[ 3000.044032] [<c128dac3>] ? printk+0xe/0x13
[ 3000.044032] [<e0c43c46>] ? hub_port_init+0x2cd/0x650
[usbcore]
[ 3000.044032] [<c1006764>] ? check_events+0x8/0xc
[ 3000.044032] [<e0c45b1e>] ? hub_thread+0x825/0xe77
[usbcore]
[ 3000.044032] [<c10345f2>] ? try_to_wake_up+0x2a7/0x2b3
[ 3000.044032] [<c104b65a>] ?
autoremove_wake_function+0x0/0x2d
[ 3000.044032] [<e0c452f9>] ? hub_thread+0x0/0xe77
[usbcore]
[ 3000.044032] [<c104b428>] ? kthread+0x61/0x66
[ 3000.044032] [<c104b3c7>] ? kthread+0x0/0x66
[ 3000.044032] [<c1009ae7>] ? kernel_thread_helper+0x7/0x10
[ 3120.040097] INFO: task khubd:1651 blocked for more than 120
seconds.
[ 3120.043975] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3120.044033] khubd D e0c53005 0 1651 2
0x00000000
[ 3120.044033] de8ba200 00000246 e0c4655d e0c53005 e0c92818
c145ee20 c145ee20 c145a354
[ 3120.044033] de8ba3bc c2f34e20 00000000 c21f02e8 00000000
e0c92a2c c21fa400 c233c100
[ 3120.044033] fffffffe de8ba3bc 00090af7 00000000 c233c100
00000000 df9c7e3c e0c563c4
[ 3120.044033] Call Trace:
[ 3120.044033] [<e0c4655d>] ?
usb_hcd_check_unlink_urb+0x65/0x81 [usbcore]
[ 3120.044033] [<e0c47f23>] ? usb_kill_urb+0x84/0xa0
[usbcore]
[ 3120.044033] [<c104b65a>] ?
autoremove_wake_function+0x0/0x2d
[ 3120.044033] [<e0c48f4c>] ? usb_start_wait_urb+0x62/0x92
[usbcore]
[ 3120.044033] [<e0c48389>] ? usb_init_urb+0x1a/0x26
[usbcore]
[ 3120.044033] [<e0c49187>] ? usb_control_msg+0xf7/0x111
[usbcore]
[ 3120.044033] [<c128dac3>] ? printk+0xe/0x13
[ 3120.044033] [<e0c43c46>] ? hub_port_init+0x2cd/0x650
[usbcore]
[ 3120.044033] [<c1006764>] ? check_events+0x8/0xc
[ 3120.044033] [<e0c45b1e>] ? hub_thread+0x825/0xe77
[usbcore]
[ 3120.044033] [<c10345f2>] ? try_to_wake_up+0x2a7/0x2b3
[ 3120.044033] [<c104b65a>] ?
autoremove_wake_function+0x0/0x2d
[ 3120.044033] [<e0c452f9>] ? hub_thread+0x0/0xe77
[usbcore]
[ 3120.044033] [<c104b428>] ? kthread+0x61/0x66
[ 3120.044033] [<c104b3c7>] ? kthread+0x0/0x66
[ 3120.044033] [<c1009ae7>] ? kernel_thread_helper+0x7/0x10
[ 3240.040101] INFO: task khubd:1651 blocked for more than 120
seconds.
[ 3240.043989] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3240.044035] khubd D e0c53005 0 1651 2
0x00000000
[ 3240.044035] de8ba200 00000246 e0c4655d e0c53005 e0c92818
c145ee20 c145ee20 c145a354
[ 3240.044035] de8ba3bc c2f34e20 00000000 c21f02e8 00000000
e0c92a2c c21fa400 c233c100
[ 3240.044035] fffffffe de8ba3bc 00090af7 00000000 c233c100
00000000 df9c7e3c e0c563c4
[ 3240.044035] Call Trace:
[ 3240.044035] [<e0c4655d>] ?
usb_hcd_check_unlink_urb+0x65/0x81 [usbcore]
[ 3240.044035] [<e0c47f23>] ? usb_kill_urb+0x84/0xa0
[usbcore]
[ 3240.044035] [<c104b65a>] ?
autoremove_wake_function+0x0/0x2d
[ 3240.044035] [<e0c48f4c>] ? usb_start_wait_urb+0x62/0x92
[usbcore]
[ 3240.044035] [<e0c48389>] ? usb_init_urb+0x1a/0x26
[usbcore]
[ 3240.044035] [<e0c49187>] ? usb_control_msg+0xf7/0x111
[usbcore]
[ 3240.044035] [<c128dac3>] ? printk+0xe/0x13
[ 3240.044035] [<e0c43c46>] ? hub_port_init+0x2cd/0x650
[usbcore]
[ 3240.044035] [<c1006764>] ? check_events+0x8/0xc
[ 3240.044035] [<e0c45b1e>] ? hub_thread+0x825/0xe77
[usbcore]
[ 3240.044035] [<c10345f2>] ? try_to_wake_up+0x2a7/0x2b3
[ 3240.044035] [<c104b65a>] ?
autoremove_wake_function+0x0/0x2d
[ 3240.044035] [<e0c452f9>] ? hub_thread+0x0/0xe77
[usbcore]
[ 3240.044035] [<c104b428>] ? kthread+0x61/0x66
[ 3240.044035] [<c104b3c7>] ? kthread+0x0/0x66
[ 3240.044035] [<c1009ae7>] ? kernel_thread_helper+0x7/0x10
[ 3360.040100] INFO: task khubd:1651 blocked for more than 120
seconds.
[ 3360.043981] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
...
After attaching the SAS-Controller, the domU's dmesg looks as
followed:
[ 92.562729] pcifront pci-0: Installing PCI frontend
[ 92.571091] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[ 92.579904] pci 0000:00:00.0: reg 10 io port: [0xcc00-0xccff]
[ 92.580028] pci 0000:00:00.0: reg 14 64bit mmio:
[0xee3ec000-0xee3effff]
[ 92.580149] pci 0000:00:00.0: reg 1c 64bit mmio:
[0xee3f0000-0xee3fffff]
[ 92.580268] pci 0000:00:00.0: reg 30 32bit mmio pref:
[0xee000000-0xee1fffff]
[ 92.580568] pci 0000:00:00.0: supports D1 D2
[ 92.582496] pcifront pci-0: claiming resource 0000:00:00.0/0
[ 92.582496] pcifront pci-0: claiming resource 0000:00:00.0/1
[ 92.582496] pcifront pci-0: claiming resource 0000:00:00.0/3
[ 92.582496] pcifront pci-0: claiming resource 0000:00:00.0/6
[ 92.711562] SCSI subsystem initialized
[ 92.761565] Fusion MPT base driver 3.04.12
[ 92.764008] Copyright (c) 1999-2008 LSI Corporation
[ 92.788484] Fusion MPT SAS Host driver 3.04.12
[ 92.790749] mptsas 0000:00:00.0: enabling device (0000 ->
0002)
[ 92.792004] mptsas 0000:00:00.0: Xen PCI enabling IRQ: 35
[ 92.795412] mptbase: ioc0: Initiating bringup
DomU's /proc/interrupts:
CPU0
11: 0 xen-pirq-xt-pic ohci_hcd:usb1
35: 66 xen-pirq-pcifront ioc0
1520: 1 xen-dyn-event pcifront
1521: 552 xen-dyn-event eth0
1522: 122 xen-dyn-event blkif
1523: 457 xen-dyn-event blkif
1524: 1235 xen-dyn-event blkif
1525: 1345 xen-dyn-event blkif
1526: 1104 xen-dyn-event blkif
1527: 0 xen-dyn-event vkbd
1528: 28 xen-dyn-event hvc_console
1529: 612765 xen-dyn-event vfb
1530: 1155 xen-dyn-event xenbus
1531: 0 xen-percpu-ipi callfuncsingle0
1532: 0 xen-percpu-virq debug0
1533: 0 xen-percpu-ipi callfunc0
1534: 0 xen-percpu-ipi resched0
1535: 8548 xen-percpu-virq timer0
NMI: 0 Non-maskable interrupts
LOC: 0 Local timer interrupts
SPU: 0 Spurious interrupts
PMI: 0 Performance monitoring interrupts
PND: 0 Performance pending work
RES: 0 Rescheduling interrupts
CAL: 0 Function call interrupts
TLB: 0 TLB shootdowns
TRM: 0 Thermal event interrupts
THR: 0 Threshold APIC interrupts
MCE: 0 Machine check exceptions
MCP: 0 Machine check polls
ERR: 0
MIS: 0
There is also a difference between the kinds of interrups between
IRQ 11 and IRQ 35.
Does someone have an idea how to fix this?
Kind regards,
Michael
|
