[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] RE: win-pv assertion failed while migrating



I've just pushed a fix which seems to fix the problem - no leaking at
all so far.

James

> -----Original Message-----
> From: alice wan [mailto:wanjia19870902@xxxxxxxxx]
> Sent: Wednesday, 16 February 2011 13:44
> To: James Harper
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: win-pv assertion failed while migrating
> 
> Assertion failure didn't appear, but vm still had problem after
migrating six
> hundred times.
> 
> XenPCI <-- XenPci_ShutdownHandler
> XenPCI     Adding watch for path = control/shutdown
> XenPCI --> XenPci_DeviceWatchHandler
> XenPCI <-- XenPci_DeviceWatchHandler
> XenPCI <-- XenBus_Resume
> XenPCI --> XenPci_BalloonHandler
> XenPCI     suspend event channel = 6
> XenPCI --> EvtChn_BindDpc
> XenPCI <-- EvtChn_BindDpc
> XenPCI     target memory value = 512 (524288)
> XenPCI     Resuming child
> XenPCI --> XenPci_Pdo_Resume
> XenPCI <-- XenPci_BalloonHandler
> XenPCI     path = device/vbd/768
> XenPCI --> XenPci_UpdateBackendState
> XenPCI --> XenPci_GetBackendAndAddWatch
> XenPCI     Got balloon event, current = 512, target = 512
> XenPCI     No change to memory
> XenPCI --> XenPci_IoWatch
> XenPCI     found pending read - MinorFunction = 0, length = 1024
> XenPCI <-- XenBus_ProcessReadRequest
> XenPCI <-- XenPci_IoWatch
> XenPCI --> XenPci_EvtIoDefault
> XenPCI --> XenPci_DeviceWatchHandler
> XenPCI --> XenBus_EvtIoWrite
> XenPCI     33 bytes of write buffer remaining
> XenPCI     Rescanning child list
> XenPCI --> XenPci_EvtChildListScanForChildren
> XenPCI     completing request with length 33
> XenPCI <-- XenBus_EvtIoWrite
> XenPCI <-- XenPci_EvtIoDefault
> XenPCI     Found path = device/vbd/768
> XenPCI --> XenPci_EvtIoDefault
> XenPCI --> XenBus_EvtIoRead
> XenPCI     Found path = device/vif/0
> XenPCI     found pending read
> XenPCI <-- XenPci_EvtChildListScanForChildren
> XenPCI <-- XenBus_ProcessReadRequest
> XenPCI <-- XenPci_DeviceWatchHandler
> XenPCI <-- XenBus_EvtIoRead
> XenPCI <-- XenPci_EvtIoDefault
> XenPCI --> XenPci_EvtIoDefault
> XenPCI <-- XenPci_GetBackendAndAddWatch
> XenPCI --> XenBus_EvtIoRead
> XenPCI --> XenPci_ChangeFrontendState
> XenPCI     no data to read
> XenPCI --> XenPci_UpdateBackendState
> XenPCI <-- XenBus_EvtIoRead
> XenPCI <-- XenPci_ChangeFrontendState
> XenPCI <-- XenPci_EvtIoDefault
> XenPCI --> XenPci_XenConfigDeviceSpecifyBuffers
> XenPCI     state unchanged
> XenPCI     XEN_INIT_TYPE_RING - ring-ref = FED06000
> XenPCI --> XenPci_DeviceWatchHandler
> XenPCI     No free grant refs
> XenPCI <-- XenPci_DeviceWatchHandler
> XenPCI     Error
> XenPCI --> XenPci_ChangeFrontendState
> XenPCI --> XenPci_DeviceWatchHandler
> XenPCI <-- XenPci_ChangeFrontendState
> XenPCI <-- XenPci_DeviceWatchHandler
> XenPCI <-- XenPci_XenConfigDeviceSpecifyBuffers, status = c0000001
> XenPCI --> XenPci_ChangeFrontendState
> XenPCI --> XenPci_DeviceWatchHandler
> XenPCI <-- XenPci_DeviceWatchHandler
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     Backend State Changed to Closing
> XenPCI     Requesting eject
> XenPCI <-- XenPci_UpdateBackendState
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> Log-dirty command enable
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI     Suspend detected via Dpc
> XenPCI --> XenPci_SuspendResume
> XenPCI <-- XenPci_SuspendResume
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Still waiting for 4 (currently 5)...
> XenPCI --> XenPci_UpdateBackendState
> XenPCI     state unchanged
> XenPCI     Timed out waiting for 4!
> XenPCI     Failed to change frontend state to connected
> XenPCI <-- XenPci_Pdo_Resume
> XenPCI     Resuming child
> XenPCI --> XenPci_Pdo_Resume
> XenPCI     path = device/vif/0
> XenPCI <-- XenPci_Pdo_Resume
> XenPCI <-- XenPci_SuspendResume
> 
> 
> xenstore-ls /local/domain/635/device
> vfb = ""
>  0 = ""
>   state = "1"
>   backend-id = "0"
>   backend = "/local/domain/0/backend/vfb/635/0"
> console = ""
>  0 = ""
>   state = "1"
>   backend-id = "0"
>   backend = "/local/domain/0/backend/console/635/0"
> vbd = ""
>  768 = ""
>   backend-id = "0"
>   virtual-device = "768"
>   device-type = "disk"
>   state = "4"
>   backend = "/local/domain/0/backend/vbd/635/768"
> vif = ""
>  0 = ""
>   backend-id = "0"
>   mac = "00:16:3e:e0:00:13"
>   handle = "0"
>   state = "1"
>   backend = "/local/domain/0/backend/vif/635/0"
> suspend = ""
>  event-channel = "6"
> 
> 
> 2011/2/15 James Harper <james.harper@xxxxxxxxxxxxxxxx>
> 
> 
>       I have just pushed a fix. It was a stupid mistake on my part and
I
>       didn't notice it because Windows 2008 doesn't seem to crash on
ASSERT
>       failures.
> 
>       Please try it now.
> 
>       Thanks
> 
> 
>       James
> 
>       > -----Original Message-----
>       > From: alice wan [mailto:wanjia19870902@xxxxxxxxx]
> 
>       > Sent: Tuesday, 15 February 2011 19:20
>       > To: James Harper
>       > Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
>       > Subject: Re: win-pv assertion failed while migrating
>       >
>       > new problem appeared in the fix and only after migrate once,
it came
>       out.
>       >
>       > XenNet --> XenNet_SuspendResume
>       > XenNet     New state SR_STATE_RESUMING
>       > XenNet <-- XenNet_SuspendResume
>       > XenNet --> XenNet_ResumeWorkItem
>       > XenPCI --> EvtChn_PdoEventChannelDpc
>       >
>       > *** Assertion failed: !xi->resume_work_item
>       > ***   Source File:
e:\win-pvdrivers\win-pvdrivers.hg\xennet\xennet.c,
>       line 248
>       >
>       > XenPCI <-- EvtChn_PdoEventChannelDpc
>       > XenVbd --> DriverEntry
>       > XenVbd     IRQL = 31
>       > XenVbd     DriverObject = 00000000, RegistryPath = 00000000
>       > XenVbd --> XenVbd_HwScsiFindAdapter
>       > XenVbd     IRQL = 31
>       > XenVbd     xvdd = 8196B300
>       > XenVbd     BusInterruptLevel = 28
>       > XenVbd     BusInterruptVector = 01c
>       > XenVbd     NumberOfAccessRanges = 1
>       > XenVbd     RangeStart = 1f9eb000, RangeLength = 00001000
>       > XenVbd     XEN_INIT_TYPE_VECTORS
>       > XenVbd     XEN_INIT_TYPE_DEVICE_STATE - 820E93CC
>       > XenVbd     XEN_INIT_TYPE_RING - ring-ref = 815FC000
>       > XenVbd     XEN_INIT_TYPE_EVENT_CHANNEL - event-channel = 7
>       > XenVbd     XEN_INIT_TYPE_READ_STRING - device-type = disk
>       > XenVbd     device-type = Disk
>       > XenVbd     XEN_INIT_TYPE_READ_STRING - mode = w
>       > XenVbd     mode = w
>       > XenVbd     XEN_INIT_TYPE_READ_STRING - sectors = 104857600
>       > XenVbd     XEN_INIT_TYPE_READ_STRING - sector-size = 512
>       > XenVbd     XEN_INIT_TYPE_GRANT_ENTRIES - entries = 11
>       > XenVbd     qemu_hide_flags_value = 3
>       > XenVbd     aligned_buffer_data = 8196D274
>       > XenVbd     aligned_buffer = 8196E000
>       > XenVbd     ConfigInfo->MaximumTransferLength = 4096
>       > XenVbd     ConfigInfo->NumberOfPhysicalBreaks = 0
>       > XenVbd     ConfigInfo->CachesData was initialised to 0
>       > XenVbd     Dma64BitAddresses not supported
>       > XenVbd <-- XenVbd_HwScsiFindAdapter
>       > XenVbd --> XenVbd_HwScsiInitialize
>       > XenVbd     IRQL = 31
>       > XenVbd     dump_mode = 1
>       > XenVbd <-- XenVbd_HwScsiInitialize
>       > XenVbd     Command = INQUIRY
>       > XenVbd --> XenVbd_HwScsiResetBus
>       > XenVbd     IRQL = 31
>       > XenVbd <-- XenVbd_HwScsiResetBus
>       > XenVbd     Command = SCSIOP_START_STOP_UNIT
>       > XenVbd     Command = READ_CAPACITY
>       > XenVbd <-- DriverEntry
>       > XenVbd     max_dump_mode_blocks = 8
>       > XenVbd     max_dump_mode_length = 4096
>       >
>       > *** Assertion failed: srb != NULL
>       > ***   Source File:
>       e:\win-pvdrivers\win-pvdrivers.hg\xenvbd\xenvbd_scsiport.c,
>       > line 988
>       >
>       > XenNet --> XenNet_Shutdown
>       > XenNet <-- XenNet_Shutdown
>       > XenPCI     Bug check 0x0000007E (0x80000003, 0x80878113,
0xF88A566C,
>       > 0xF88A5368)
>       >
>       >
>       > 2011/2/15 alice wan <wanjia19870902@xxxxxxxxx>
>       >
>       >
>       >       I build from source. today, i'll use latest version to
test, if
>       there's
>       > any results, i'll let u know
>       >
>       >
>       >       2011/2/14 James Harper <james.harper@xxxxxxxxxxxxxxxx>
>       >
>       >
>       >               I've found a race in xennet that could cause the
>       problems you are
>       > seeing
>       >               and have pushed a fix, but it's not well tested.
From
>       what you've
>       > said I
>       >               think you can build from source. If not, I
should be
>       able to
>       > upload some
>       >               binaries tomorrow. Please let me know how your
testing
>       goes.
>       >
>       >
>       >               James
>       >
>       >               > -----Original Message-----
>       >               > From: alice wan
[mailto:wanjia19870902@xxxxxxxxx]
>       >
>       >               > Sent: Monday, 14 February 2011 18:01
>       >
>       >               > To: James Harper
>       >               > Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
>       >
>       >               > Subject: Re: win-pv assertion failed while
migrating
>       >               >
>       >               > serveral hundred times, not regular and vm
blue screen
>       of death
>       >               >
>       >               >
>       >               >
>       >               >
>       >               > 2011/2/14 James Harper
<james.harper@xxxxxxxxxxxxxxxx>
>       >               >
>       >               >
>       >               >       >
>       >               >       > yes, windows 2003-32.
>       >               >       >
>       >               >       > my test is migrate vm between host A
and host
>       B
>       >               >       > continually(xen4.0.0.0+2.6.31.13)
>       >               >       >
>       >               >       > Almost after automatic migrate script
ran for
>       one day,
>       > this
>       >               problem
>       >               >       appeared
>       >               >       >
>       >               >       > maybe i can also do some test on
>       xen4.0.1+2.6.32.x to
>       > check
>       >               whether it
>       >               >       matters
>       >               >       > kernel or pv driver.
>       >               >
>       >               >
>       >               >       So you are saying that it doesn't happen
>       immediately,
>       > only after
>       >               some
>       >               >       time? Do you know how many times?
>       >               >
>       >               >       Maybe I'm leaking a resource somewhere
every
>       time migrate
>       >               happens and
>       >               >       eventually it runs out...
>       >               >
>       >               >       James
>       >               >
>       >               >
>       >
>       >
>       >
>       >
> 
> 
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.