RE: [Xen-users] Missed interrupt. Increasing latency to 5 ms inorderto compensate.

["Jamie J. Begin" <begin@xxxxxxxxxxxxxxx>]

> -----Original Message-----
> ________________________________
> From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-users-
> bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Jamie J. Begin
> Sent: June 10, 2008 1:12 PM
> To: xen-users@xxxxxxxxxxxxxxxxxxx
> Subject: [Xen-users] Missed interrupt. Increasing latency to 5 ms in
> orderto compensate.
> 
> I am using pciback to pass a Digium PCIX analog telephony card to an
> Asterisk domU.  Seemingly at random times, the card wigs out with the
> following appended to my /var/log/messages:
> 
> Jun 10 12:17:19 asterisk01 kernel: wctdm24xxp0: Missed interrupt.
> Increasing latency to 4 ms in order to compensate.
> Jun 10 14:01:01 asterisk01 kernel: wctdm24xxp0: Missed interrupt.
> Increasing latency to 5 ms in order to compensate.
> 
> This is becoming a major problem because it will drop any active calls and
> refuse to accept new calls until I stop Asterisk, remove and then re-add
> the kernel module, and finally restart Asterisk.  I don't know whether the
> drivers, Xen, the motherboard/BIOS, or Asterisk is to blame.  However,
> this did not occur until after I virtualized the server.  Does anyone have
> any suggestions?  This is killing me here.


> From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-users-
> bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of [STD]Ein
> Sent: Tuesday, June 10, 2008 10:32 PM
> To: 'Jamie J. Begin'; xen-users@xxxxxxxxxxxxxxxxxxx
> Subject: RE: [Xen-users] Missed interrupt. Increasing latency to 5 ms
> inorderto compensate.
> 
> Is it possible the cpu(s) are being taxed?  If they are under heavy load
> (from another domain most likely) it may cause the domain to not respond
> to interrupts in a timely manner.  If this is the case, you might try
> changing the scheduling weights of your domains (search the list).
>

Thanks for the suggestion; unfortunately I wish it were that easy.  The
physical machine is a brand new Dell Poweredge 1650 III with a quad-core
Xeon and 2 GB of RAM.  The Asterisk domU is the only domU currently running.
It is allocated two of the cores and rarely gets above a 0.04 load average
throughout the day since it's only processing 2-3 voice channels at any
given time.

I've been in this business long enough to know that "random" problems are
rarely random but I'm having difficulty connecting the dots.  This problem
can occur during the weekend when the system is idle, or in the middle of a
phone call on a Monday afternoon.

I'm thinking it may somehow be an IRQ conflict. The card doesn't seem to
share an IRQ with anything; but it looks to have been assigned IRQ 18, which
seems rather odd.  I haven't played around with any of the APIC kernel
options because I'm not that familiar with them.  Here's some output that
may be useful:

"lspci" from the dom0:

0b:08.0 Ethernet controller: Digium, Inc. Unknown device 8002 (rev 11)
        Subsystem: Digium, Inc. Unknown device 8002
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
        Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32 (16000ns min, 32000ns max), Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 18
        Region 0: I/O ports at dc00 [size=256]
        Region 1: Memory at fc7dfc00 (32-bit, non-prefetchable) [size=1K]
        Expansion ROM at fc7e0000 [disabled] [size=128K]
        Capabilities: [c0] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA
PME(D0+,D1+,D2+,D3hot+,D3cold-)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME-


"cat /proc/interrupts" from dom0.
           CPU0              CPU1              CPU2              CPU3

  1:          2          0          0          0        Phys-irq  i8042
  8:          0          0          0          0        Phys-irq  rtc
  9:          0          0          0          0        Phys-irq  acpi
 12:          4          0          0          0        Phys-irq  i8042
 17:   31664532          0          0         94        Phys-irq  megasas,
peth0
 19:     446716       2870          5         36        Phys-irq  eth2
 20:   25613484          0       1305          0        Phys-irq
uhci_hcd:usb1, ehci_hcd:usb2
256:  102837808          0          0          0     Dynamic-irq  timer0
257:    2439233          0          0          0     Dynamic-irq  resched0
258:         43          0          0          0     Dynamic-irq  callfunc0
259:          0     634445          0          0     Dynamic-irq  resched1
260:          0        126          0          0     Dynamic-irq  callfunc1
261:          0   13329219          0          0     Dynamic-irq  timer1
262:          0          0     559915          0     Dynamic-irq  resched2
263:          0          0        120          0     Dynamic-irq  callfunc2
264:          0          0    6024225          0     Dynamic-irq  timer2
265:          0          0          0    1626891     Dynamic-irq  resched3
266:          0          0          0        101     Dynamic-irq  callfunc3
267:          0          0          0   31030283     Dynamic-irq  timer3
268:       4234          0          0          0     Dynamic-irq  xenbus
269:          0          0          0          0     Dynamic-irq  console
270:        378          0          0          0     Dynamic-irq  pciback
271:      89831       2843       2883         15     Dynamic-irq
blkif-backend
272:    4164194      24455      22095       4105     Dynamic-irq  vif1.0
273:      60758       7524       5516         50     Dynamic-irq
blkif-backend
274:     694799    3497036       2722          8     Dynamic-irq  vif6.0
NMI:          0          0          0          0 
LOC:          0          0          0          0

"cat /proc/interrupts" from the Asterisk domU:
           CPU0              CPU1              
 18:  577477891          0        Phys-irq  wctdm24xxp0
256:   78034763          0     Dynamic-irq  timer0
257:      59661          0     Dynamic-irq  resched0
258:         38          0     Dynamic-irq  callfunc0
259:        409          0     Dynamic-irq  xenbus
260:          0      73103     Dynamic-irq  resched1
261:          0         70     Dynamic-irq  callfunc1
262:          0    3942339     Dynamic-irq  timer1
263:        409          1     Dynamic-irq  xencons
264:     113336      10469     Dynamic-irq  blkif
265:    8317410          0     Dynamic-irq  eth0
NMI:          0          0 
LOC:          0          0 
ERR:          0
MIS:          0


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users