|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] RE: XenVif div by zero on Tx path after resume.
-----Original Message-----
From: win-pv-devel <win-pv-devel-bounces@xxxxxxxxxxxxxxxxxxxx> On Behalf Of
Durrant, Paul
Sent: 22 April 2022 13:32
To: paul@xxxxxxx; win-pv-devel@xxxxxxxxxxxxxxxxxxxx
Subject: Re: XenVif div by zero on Tx path after resume.
> Oh ffs... the call to KeLowerIrql(DISPATCH_LEVEL) is being done at the end of
> SyncEnableInterrupts() rather than after the late callbacks at the beginning
> of SyncRelease(). That will be the problem... and now I need to dig for why
> it is like that.
Yes, I thought something was a bit wrong.
On the running machine, I notice all the CPU's are at IRQL zero (small sample
here of a few CPU's of 32), and a dump of the running threads on the system,
one of which points fairly conclusively to SyncRelease:
31: kd> !pcr 0
KPCR for Processor 0 at fffff8061205a000:
Major 1 Minor 1
NtTib.ExceptionList: fffff80612169fb0
NtTib.StackBase: fffff80612168000
NtTib.StackLimit: 0000000000000000
NtTib.SubSystemTib: fffff8061205a000
NtTib.Version: 000000001205a180
NtTib.UserPointer: fffff8061205a870
NtTib.SelfTib: 000000cf28cf3000
SelfPcr: 0000000000000000
Prcb: fffff8061205a180
Irql: 0000000000000000
IRR: 0000000000000000
IDR: 0000000000000000
InterruptMode: 0000000000000000
IDT: 0000000000000000
GDT: 0000000000000000
TSS: 0000000000000000
CurrentThread: fffff8061753fb80
NextThread: ffffe084d02d9080
IdleThread: fffff8061753fb80
DpcQueue: Unable to read nt!_KDPC_DATA.DpcListHead.Flink @
fffff8061205d4c0
31: kd> !pcr 1
KPCR for Processor 1 at ffff9781f2beb000:
Major 1 Minor 1
NtTib.ExceptionList: ffff9781f2bfcfb0
NtTib.StackBase: ffff9781f2bfb000
NtTib.StackLimit: 0000000000000000
NtTib.SubSystemTib: ffff9781f2beb000
NtTib.Version: 00000000f2beb180
NtTib.UserPointer: ffff9781f2beb870
NtTib.SelfTib: 000000e761342000
SelfPcr: 0000000000000000
Prcb: ffff9781f2beb180
Irql: 0000000000000000
IRR: 0000000000000000
IDR: 0000000000000000
InterruptMode: 0000000000000000
IDT: 0000000000000000
GDT: 0000000000000000
TSS: 0000000000000000
CurrentThread: ffffe084d1164080
NextThread: 0000000000000000
IdleThread: ffff9781f2bf8040
DpcQueue: Unable to read nt!_KDPC_DATA.DpcListHead.Flink @
ffff9781f2bee4c0
31: kd> !pcr 2
KPCR for Processor 2 at ffff9781f2e87000:
Major 1 Minor 1
NtTib.ExceptionList: ffff9781f2e98fb0
NtTib.StackBase: ffff9781f2e97000
NtTib.StackLimit: 0000000000000000
NtTib.SubSystemTib: ffff9781f2e87000
NtTib.Version: 00000000f2e87180
NtTib.UserPointer: ffff9781f2e87870
NtTib.SelfTib: 000000855f4a4000
SelfPcr: 0000000000000000
Prcb: ffff9781f2e87180
Irql: 0000000000000000
IRR: 0000000000000000
IDR: 0000000000000000
InterruptMode: 0000000000000000
IDT: 0000000000000000
GDT: 0000000000000000
TSS: 0000000000000000
CurrentThread: ffff9781f2e94040
NextThread: 0000000000000000
IdleThread: ffff9781f2e94040
DpcQueue: Unable to read nt!_KDPC_DATA.DpcListHead.Flink @
ffff9781f2e8a4c0
31: kd> !thread ffffe084d1164080
THREAD ffffe084d1164080 Cid 0ef8.1bec Teb: 000000e761342000 Win32Thread:
0000000000000000 RUNNING on processor 1
Impersonation token: ffffc009fb7100f0 (Level Impersonation)
Owning Process ffffe084d00b0240 Image: svchost.exe
Attached Process N/A Image: N/A
Wait Start TickCount 6384 Ticks: 0
Context Switch Count 207 IdealProcessor: 0
UserTime 00:00:00.031
KernelTime 00:00:00.015
Win32 Start Address 0x00007ff9eb93bfc0
Stack Init ffff958818c67bf0 Current ffff958818c672f0
Base ffff958818c68000 Limit ffff958818c61000 Call 0000000000000000
Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5
Child-SP RetAddr : Args to Child
: Call Site
ffff9588`18c67520 fffff806`16b0b419 : ffffc009`fb710000 ffffe084`c96c2b01
ffff9588`00000001 ffffe084`d127bea0 : nt!SeCaptureSecurityDescriptor+0x4dd
ffff9588`18c67620 fffff806`16b0af33 : 00000000`00000002 ffffc009`fb876060
00000000`00000001 fffff806`16f851d9 :
nt!SeAccessCheckByTypeWithAdminlessChecks+0x4c9
ffff9588`18c678e0 fffff806`16b0d177 : 0000020b`6aef23d0 00000000`00000000
00000000`00000cb4 00000000`00000001 : nt!SeAccessCheckByType+0x113
ffff9588`18c67980 fffff806`16c28c35 : 00000000`00000000 00000000`00000000
0000020b`6aef23d0 00000000`00000000 : nt!NtAccessCheck+0x57
ffff9588`18c679f0 00007ff9`eb9ceef4 : 00000000`00000000 00000000`00000000
00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x25 (TrapFrame
@ ffff9588`18c67a60)
000000e7`61bfdd48 00000000`00000000 : 00000000`00000000 00000000`00000000
00000000`00000000 00000000`00000000 : 0x00007ff9`eb9ceef4
31: kd> !thread ffffe084cbb5e040
THREAD ffffe084cbb5e040 Cid 0004.02cc Teb: 0000000000000000 Win32Thread:
0000000000000000 RUNNING on processor 13
Not impersonating
DeviceMap ffffc009f5a366bf
Owning Process ffffe084c96b4040 Image: System
Attached Process N/A Image: N/A
Wait Start TickCount 5180 Ticks: 1204 (0:00:00:18.812)
Context Switch Count 3 IdealProcessor: 19
UserTime 00:00:00.000
KernelTime 00:00:18.812
Win32 Start Address xenbus!ThreadFunction (0xfffff806187fa850)
Stack Init ffff958815267bf0 Current ffff9588152676f0
Base ffff958815268000 Limit ffff958815261000 Call 0000000000000000
Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5
Child-SP RetAddr : Args to Child
: Call Site
(Inline Function) --------`-------- : --------`-------- --------`--------
--------`-------- --------`-------- : xenbus!SyncRelease+0xc4 (Inline Function
@ fffff806`187fa2e1)
[E:\jenkins\workspace\win-xenbus_master\local\src\xenbus\sync.c @ 460]
ffff9588`152679f0 fffff806`187eb206 : ffffe084`cbb09a00 ffffe084`001f0000
00000000`00989680 00000000`00989680 : xenbus!SuspendTrigger+0x5d1
[E:\jenkins\workspace\win-xenbus_master\local\src\xenbus\suspend.c @ 288]
ffff9588`15267aa0 fffff806`187fa86a : 5d158d48`00642464 48244489`48ffa778
ffffe084`cba38ce0 244c8900`00008824 : xenbus!FdoSuspend+0x1a6
[E:\jenkins\workspace\win-xenbus_master\local\src\xenbus\fdo.c @ 1658]
ffff9588`15267b40 fffff806`16ae83f5 : ffffe084`cbb5e040 fffff806`187fa850
ffffe084`cba38ce0 20246483`48002824 : xenbus!ThreadFunction+0x1a
[E:\jenkins\workspace\win-xenbus_master\local\src\xenbus\thread.c @ 109]
ffff9588`15267b70 fffff806`16c1a868 : ffff9781`f3dea180 ffffe084`cbb5e040
fffff806`16ae83a0 3d803745`8948c433 : nt!PspSystemThreadStartup+0x55
ffff9588`15267bc0 00000000`00000000 : ffff9588`15268000 ffff9588`15261000
00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x28
31: kd> !thread ffffe084d12c4080
THREAD ffffe084d12c4080 Cid 110c.087c Teb: 0000000000d64000 Win32Thread:
ffffe084d1a5fbd0 RUNNING on processor 15
Not impersonating
DeviceMap ffffc009f5a366bf
Owning Process ffffe084d01820c0 Image:
xenguestagent.exe
Attached Process N/A Image: N/A
Wait Start TickCount 6383 Ticks: 1 (0:00:00:00.015)
Context Switch Count 7 IdealProcessor: 27
UserTime 00:00:00.000
KernelTime 00:00:00.000
Win32 Start Address 0x00007ff9eb93bfc0
Stack Init ffff958816e2fbf0 Current ffff958816e2f350
Base ffff958816e30000 Limit ffff958816e29000 Call 0000000000000000
Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5
Child-SP RetAddr : Args to Child
: Call Site
ffff9588`16e2f670 fffff806`17242239 : 00000000`00000000 01000000`00100000
00000000`00000000 00000000`00000000 : nt!ExFreeHeapPool+0x1c8
ffff9588`16e2f770 fffff806`16f121b7 : 00000000`1e1b5180 ffffe084`c9818c60
00000000`1ed2c0e8 ffffc009`fb874be0 : nt!ExFreePool+0x9
ffff9588`16e2f7a0 fffff806`16f701e2 : ffffc009`00000000 00000000`00000000
ffffc009`fb874be0 00000000`1ed2c0e8 : nt!AlpcMessageDestroyProcedure+0x47
ffff9588`16e2f7d0 fffff806`16f6fdc2 : 00000000`1e1b5180 00000000`fe000000
ffffc009`fb874be0 00000000`fe000000 : nt!AlpcpUnlockBlob+0xc2
ffff9588`16e2f810 fffff806`16f6dc46 : ffffe084`c9818c60 ffff9588`00020000
00000000`1e1b5180 00000000`1c614c28 : nt!AlpcpProcessSynchronousRequest+0x562
ffff9588`16e2f930 fffff806`16c28c35 : ffffe084`d12c4080 ffff9588`16e2fae0
00000000`1ed2c0b8 ffff9588`16e2fa08 : nt!NtAlpcSendWaitReceivePort+0x1d6
ffff9588`16e2f9f0 00007ff9`eb9d0064 : 00000000`00000000 00000000`00000000
00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x25 (TrapFrame
@ ffff9588`16e2fa60)
00000000`1ed2c098 00000000`00000000 : 00000000`00000000 00000000`00000000
00000000`00000000 00000000`00000000 : 0x00007ff9`eb9d0064
31: kd> !thread ffffe084d0843040
THREAD ffffe084d0843040 Cid 15b8.15e8 Teb: 000000f9f5604000 Win32Thread:
0000000000000000 RUNNING on processor 16
Not impersonating
DeviceMap ffffc009f5a366bf
Owning Process ffffe084d083e280 Image: WmiPrvSE.exe
Attached Process N/A Image: N/A
Wait Start TickCount 6384 Ticks: 0
Context Switch Count 1836 IdealProcessor: 22
UserTime 00:00:00.093
KernelTime 00:00:00.031
Win32 Start Address 0x00007ff9eb93bfc0
Stack Init ffff9588182b7bf0 Current ffff9588182b7350
Base ffff9588182b8000 Limit ffff9588182b1000 Call 0000000000000000
Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5
Child-SP RetAddr : Args to Child
: Call Site
000000f9`f5a7ca38 00000000`00000000 : 00000000`00000000 00000000`00000000
00000000`00000000 00000000`00000000 : 0x00007ff9`eb9be2e2
31: kd> !thread ffffe084d02e1080
THREAD ffffe084d02e1080 Cid 0ef8.1d0c Teb: 000000e76134e000 Win32Thread:
0000000000000000 RUNNING on processor 19
Not impersonating
DeviceMap ffffc009f5a366bf
Owning Process ffffe084d00b0240 Image: svchost.exe
Attached Process N/A Image: N/A
Wait Start TickCount 6383 Ticks: 1 (0:00:00:00.015)
Context Switch Count 45 IdealProcessor: 17
UserTime 00:00:00.000
KernelTime 00:00:00.000
Win32 Start Address 0x00007ff9d7310e90
Stack Init ffff95881895fbf0 Current ffff95881895f350
Base ffff958818960000 Limit ffff958818959000 Call 0000000000000000
Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5
Child-SP RetAddr : Args to Child
: Call Site
000000e7`624ff778 00000000`00000000 : 00000000`00000000 00000000`00000000
00000000`00000000 00000000`00000000 : 0x00007ff9`eb9be2e2
31: kd> !thread ffffe084d0411080
THREAD ffffe084d0411080 Cid 1004.129c Teb: 000000855f4a6000 Win32Thread:
0000000000000000 RUNNING on processor 1f
Not impersonating
DeviceMap ffffc009f5a366bf
Owning Process ffffe084d01930c0 Image:
windowsclatsdaemon-x64.exe
Attached Process N/A Image: N/A
Wait Start TickCount 1246 Ticks: 5138 (0:00:01:20.281)
Context Switch Count 353 IdealProcessor: 31
UserTime 00:00:00.015
KernelTime 00:00:00.031
Win32 Start Address 0x0000000000c654a0
Stack Init ffff958817ca2bf0 Current ffff958817ca2400
Base ffff958817ca3000 Limit ffff958817c9c000 Call 0000000000000000
Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5
Child-SP RetAddr : Args to Child
: Call Site
ffff9588`17ca1748 fffff806`19f9084a : 00000000`0000dead fffff806`19fa7c50
fffff806`19fa7c70 00000000`0000144e : nt!KeBugCheckEx
(Inline Function) --------`-------- : --------`-------- --------`--------
--------`-------- --------`-------- : xenvif!__Bug+0x24 (Inline Function @
fffff806`19f9084a)
[E:\jenkins\workspace\nvif_private_martinhar_CA-355670\local\src\xenvif\assert.h
@ 49]
ffff9588`17ca1750 fffff806`19f9903a : ffffe084`cd6e19f0 00000000`c0000001
00000000`000000ca 00000000`00000000 : xenvif!TransmitterQueuePacket+0x14a
[E:\jenkins\workspace\nvif_private_martinhar_CA-355670\local\src\xenvif\transmitter.c
@ 5198]
ffff9588`17ca17c0 fffff806`1ab2702b : ffffffff`fffffffe 00000000`0000000c
00000000`00000000 00000000`00000000 : xenvif!VifTransmitterQueuePacket+0x3ea
[E:\jenkins\workspace\nvif_private_martinhar_CA-355670\local\src\xenvif\vif.c @
439]
(Inline Function) --------`-------- : --------`-------- --------`--------
--------`-------- --------`-------- :
xennet!__TransmitterSendNetBufferList+0x1d8 (Inline Function @
fffff806`1ab2702b)
[E:\jenkins\workspace\win-xennet_master\local\src\xennet\transmitter.c @ 329]
ffff9588`17ca1830 fffff806`185a5e7f : ffffe084`d0132010 ffffe084`d2297644
00000000`00000000 00000000`00000018 :
xennet!TransmitterSendNetBufferLists+0x27b
[E:\jenkins\workspace\win-xennet_master\local\src\xennet\transmitter.c @ 384]
ffff9588`17ca1900 fffff806`185a5d6e : 00000000`00000000 ffffe084`d22972c0
00000000`00000000 00000000`00000002 : NDIS!ndisMSendNBLToMiniportInternal+0xff
ffff9588`17ca19d0 fffff806`185a8062 : fffff806`1745ee00 00000000`0012a00f
00000000`00001080 fffff806`16b3f01c : NDIS!ndisMSendNBLToMiniport+0xe
ffff9588`17ca1a10 fffff806`185a2290 : 00000000`00000002 ffff9588`17ca1ae9
ffffe084`cd5ae800 fffffb80`00000000 : NDIS!ndisInvokeNextSendHandler+0x42
ffff9588`17ca1a60 fffff806`19106471 : ffffe084`d22972c0 ffffe084`d22972c0
00000014`00000000 ffff9588`00000001 : NDIS!NdisSendNetBufferLists+0x5f0
ffff9588`17ca1b50 fffff806`19105d4b : ffffe084`cd1620c0 00000000`00000014
ffffe084`cbf6d8a0 fffff806`00000006 : tcpip!IpNlpFastContinueSendDatagrams+0x551
ffff9588`17ca1ca0 fffff806`1910360a : ffffe084`d03a57e0 1a000001`00000014
00000000`00000000 ffffe084`cf886720 : tcpip!IpNlpFastSendDatagram+0x23b
ffff9588`17ca1d60 fffff806`19101bcb : 00000000`3b7599ec 00000000`00989680
ffffe084`cd13e168 00000000`00000000 : tcpip!TcpTcbSend+0x141a
ffff9588`17ca20b0 fffff806`19101865 : 00000000`00000000 00000000`00000000
00000000`00000000 ffff9588`17ca26d0 : tcpip!TcpEnqueueTcbSend+0x34b
ffff9588`17ca21d0 fffff806`16b135d8 : 00000000`00000000 00000000`00000000
00000000`00000000 ffffe084`cbd6b8f0 :
tcpip!TcpTlConnectionSendCalloutRoutine+0x25
ffff9588`17ca2210 fffff806`16b1354d : fffff806`19101840 ffff9588`17ca22f0
ffffe084`d069f878 00000000`00000000 :
nt!KeExpandKernelStackAndCalloutInternal+0x78
ffff9588`17ca2280 fffff806`1915de0d : 00000000`00000000 00000000`00000000
00000000`0000018b 00000000`00000000 : nt!KeExpandKernelStackAndCalloutEx+0x1d
ffff9588`17ca22c0 fffff806`19a2b266 : ffffe084`d069f878 ffffe084`d05645a0
00000000`00000018 ffffe084`d069f878 : tcpip!TcpTlConnectionSend+0x8d
ffff9588`17ca2330 fffff806`19a2ae7b : ffffe084`d05645a0 ffff9588`17ca2ae0
ffffe084`d069f670 ffffe084`d06336e0 : afd!AfdTLStartBufferedVcSend+0xb2
ffff9588`17ca23f0 fffff806`19a27d87 : ffffe084`d069f878 ffffe084`d05645a0
ffff9588`00000018 ffff9588`17ca28c8 : afd!AfdFastConnectionSend+0x293
ffff9588`17ca24c0 fffff806`16f936db : 00000000`00000284 00000000`00000000
00000000`00000000 ffffe084`d04c9d90 : afd!AfdFastIoDeviceControl+0xf27
ffff9588`17ca2850 fffff806`16f93ce6 : 00000000`00000000 00000000`00000000
00000000`00000000 000000c0`00134368 : nt!IopXxxControlFile+0x6fb
ffff9588`17ca2980 fffff806`16c28c35 : ffffe084`d0324f60 00000000`00000000
00000085`00000000 ffffe084`d04101c0 : nt!NtDeviceIoControlFile+0x56
ffff9588`17ca29f0 00007ff9`eb9cefd4 : 00000000`00000000 00000000`00000000
00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x25 (TrapFrame
@ ffff9588`17ca2a60)
00000085`601ffb68 00000000`00000000 : 00000000`00000000 00000000`00000000
00000000`00000000 00000000`00000000 : 0x00007ff9`eb9cefd4
31: kd>
|
![]() |
Lists.xenproject.org is hosted with RackSpace, monitoring our |