|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] xhci_hcd intterrupt affinity in Dom0/DomU limited to single interrupt
On Fri, 2015-09-04 at 01:41 -0600, Jan Beulich wrote:
> >>> On 03.09.15 at 18:52, <ackerj67@xxxxxxxxx> wrote:
> > On Thu, 2015-09-03 at 09:04 -0600, Jan Beulich wrote:
> >> >>> On 03.09.15 at 14:04, <ackerj67@xxxxxxxxx> wrote:
> >> > I am still confused as to whether any device, or in this case
> >> > xhci_hcd,
> >> > can use more than one cpu at any given time. My understanding based
> >> > on
> >> > David's response is that it cannot due to the event channel mapping. The
> >> > device interrupt can be pinned to a specific cpu by specifying the
> >> > affinity. I was hoping there was a way to allow the driver's interrupt
> >> > to be
> >> > scheduled to use more than 1 CPU at any given time.
> >>
> >> The problem is that you're mixing up two things: devices and
> >> interrupts. Any individual interrupt can only be serviced by a single
> >> CPU at a time, due to the way event channels get bound. Any
> >> individual device can have more than one interrupt (MSI or MSI-X),
> >> and then each of these interrupts can be serviced on different
> >> CPUs.
> >
> > Thanks for clarifying. To the original question, with respect to my
> > limited understanding of the event channels and interrupts, each
> > interrupt can be serviced on a different CPU using irqbalance or setting
> > the affinity manually, but the same interrupt cannot be serviced by more
> > than 1 CPU at a time? If so, is there a way around the 1:1 binding when
> > loading the Dom0 kernel - a flag or option to use the native interrupt
> > scheduling for some set of or all 8 CPUs that the device can schedule
> > interrupts on when not loading the Dom0? The xhci_hcd, as one example,
> > seems to perform better when it is able to have interrupts serviced by
> > multiple CPUs.
>
> I don't follow - we tell you this doesn't work (multiple times and
> different people), and you ask yet another time whether this can
> be made work? Just to make this very clear once again: Under Xen
> (and leaving aside pure HVM guests), interrupt load from a single
> device can be spread across CPUs only when the device uses
> multiple interrupts.
>
> Jan
>
I believe the driver does support use of multiple interrupts based on
the previous explanation of the lspci output where it was established
that the device could use up to 8 interrupts which is what I see on bare
metal. What would cause the interrupt usage to be limited to a single
CPU when the hypervisor is loaded? The device mentioned, xhci_hcd can
use multiple interrupts on baremetal and the other devices including the
network interfaces are also bound to a single interrupt when the
hypervisor is loaded but can use multiple interrupts running on
baremetal.
70: 0 0 0 0 0 0
0 133 xen-percpu-ipi callfuncsingle7
71: 0 0 0 0 0 0
0 0 xen-percpu-ipi irqwork7
72: 100 0 0 0 0 0
0 0 xen-dyn-event xenbus
73: 0 0 0 0 0 0
0 0 xen-dyn-virq xen-pcpu
74: 0 0 0 0 0 0
0 0 xen-dyn-virq hvc_console
75: 6809 0 0 0 0 0
0 0 xen-pirq-msi 0000:00:1f.2
76: 470660 0 0 0 0 0
0 0 xen-pirq-msi xhci_hcd
77: 608 0 0 0 0 0
0 0 xen-pirq-msi i915
78: 26 0 0 0 0 0
0 0 xen-pirq-msi mei_me
79: 243 0 0 0 0 0
0 0 xen-pirq-msi em1
80: 315 0 0 0 0 0
0 0 xen-pirq-msi-x p4p1-rx-0
81: 104 0 0 0 0 0
0 0 xen-pirq-msi-x p4p1-tx-0
82: 2 0 0 0 0 0
0 0 xen-pirq-msi-x p4p1
83: 59 0 0 0 0 0
0 0 xen-pirq-msi snd_hda_intel
84: 48 0 0 0 0 0
0 0 xen-dyn-event evtchn:xenstored
85: 0 0 0 0 0 0
0 0 xen-dyn-event evtchn:xenstored
NMI: 0 0 0 0 0 0
0 0 Non-maskable interrupts
LOC: 0 0 0 0 0 0
0 0 Local timer interrupts
SPU: 0 0 0 0 0 0
0 0 Spurious interrupts
PMI: 0 0 0 0 0 0
0 0 Performance monitoring interrupts
IWI: 1 0 0 0 0 0
0 0 IRQ work interrupts
RTR: 0 0 0 0 0 0
0 0 APIC ICR read retries
RES: 44788 33111 267650 5641 35515 2312
10730 2479 Rescheduling interrupts
CAL: 929 1202 1164 1314 1061 1296
1156 1097 Function call interrupts
TLB: 0 0 0 0 0 0
0 0 TLB shootdowns
TRM: 0 0 0 0 0 0
0 0 Thermal event interrupts
THR: 0 0 0 0 0 0
0 0 Threshold APIC interrupts
MCE: 0 0 0 0 0 0
0 0 Machine check exceptions
MCP: 1 1 1 1 1 1
1 1 Machine check polls
HYP: 538069 43568 273140 12112 44323 7871
18949 7924 Hypervisor callback interrupts
Example of another device driver output with Hypervisor loaded:
04:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
Connection
Subsystem: Intel Corporation Device 0000
Physical Slot: 1-3
Flags: bus master, fast devsel, latency 0, IRQ 19
Memory at f5600000 (32-bit, non-prefetchable) [size=128K]
I/O ports at b000 [size=32]
Memory at f5620000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [c8] Power Management version 2
Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [e0] Express Endpoint, MSI 00
Capabilities: [a0] MSI-X: Enable+ Count=5 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Device Serial Number
Kernel driver in use: e1000e
Kernel modules: e1000e
Example of the same device driver output on Bare metal:
04:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
Connection
Subsystem: Intel Corporation Device 0000
Physical Slot: 1-3
Flags: bus master, fast devsel, latency 0, IRQ 19
Memory at f5600000 (32-bit, non-prefetchable) [size=128K]
I/O ports at b000 [size=32]
Memory at f5620000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [c8] Power Management version 2
Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [e0] Express Endpoint, MSI 00
Capabilities: [a0] MSI-X: Enable+ Count=5 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Device Serial Number
Kernel driver in use: e1000e
Kernel modules: e1000e
Bare metal:
cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
CPU6 CPU7
0: 36 0 0 0 0 0
0 0 IR-IO-APIC-edge timer
8: 0 0 0 1 0 0
0 0 IR-IO-APIC-edge rtc0
9: 0 0 1 0 1 1
0 0 IR-IO-APIC-fasteoi acpi
16: 2 0 14 0 13 0
0 2 IR-IO-APIC 16-fasteoi ehci_hcd:usb3
18: 24420 225 39883 43 134488 1044
1085 46 IR-IO-APIC 18-fasteoi ath9k
23: 0 0 20 0 0 1
3 11 IR-IO-APIC 23-fasteoi ehci_hcd:usb4
24: 0 0 0 0 0 0
0 0 DMAR_MSI-edge dmar0
25: 0 0 0 0 0 0
0 0 DMAR_MSI-edge dmar1
26: 4119 1016 1074 519 4677 978
573 432 IR-PCI-MSI-edge 0000:00:1f.2
27: 337125 47893 708965 4049 53940667 263303
87847 4958 IR-PCI-MSI-edge xhci_hcd
28: 562 0 0 0 5 1
2 0 IR-PCI-MSI-edge i915
29: 7 0 0 0 4 8
0 6 IR-PCI-MSI-edge mei_me
30: 2222 23 2721 5 5810 181
149 5 IR-PCI-MSI-edge eth0
31: 1504 17 2196 6 6581 95
77 6 IR-PCI-MSI-edge eth1-rx-0
32: 2 0 1 0 3 0
0 0 IR-PCI-MSI-edge eth1-tx-0
33: 0 0 0 0 0 1
0 1 IR-PCI-MSI-edge eth1
34: 14 6 0 0 0 38
0 2 IR-PCI-MSI-edge snd_hda_intel
35: 4 3 0 0 0 178
0 3 IR-PCI-MSI-edge snd_hda_intel
NMI: 11 105 87 1 75 2
9 1 Non-maskable interrupts
LOC: 132282 1175126 905037 79133 428383 22748
118085 12513 Local timer interrupts
SPU: 0 0 0 0 0 0
0 0 Spurious interrupts
PMI: 11 105 87 1 75 2
9 1 Performance monitoring interrupts
IWI: 1 0 0 0 0 0
0 0 IRQ work interrupts
RTR: 0 0 0 0 0 0
0 0 APIC ICR read retries
RES: 36810 28341 99815 953 60355 1142
878 823 Rescheduling interrupts
CAL: 2134 1928 1793 1965 1917 1879
1877 1882 Function call interrupts
TLB: 8064 365 20958 28575 37592 113
28495 445 TLB shootdowns
TRM: 0 0 0 0 0 0
0 0 Thermal event interrupts
THR: 0 0 0 0 0 0
0 0 Threshold APIC interrupts
MCE: 0 0 0 0 0 0
0 0 Machine check exceptions
MCP: 9 9 9 9 9 9
9 9 Machine check polls
THR: 0 0 0 0 0 0
0 0 Hypervisor callback interrupts
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |