Re: [win-pv-devel] Multiple save/restore failing with windows 8.1 64 bit domU

[Fabio Fantoni <fabio.fantoni@xxxxxxx>]

Il 24/04/2015 17:01, Paul Durrant ha scritto:
> > -----Original Message-----
> > From: Fabio Fantoni [mailto:fabio.fantoni@xxxxxxx]
> > Sent: 24 April 2015 15:46
> > To: Paul Durrant; win-pv-devel@xxxxxxxxxxxxxxxxxxxx
> > Subject: Re: Multiple save/restore failing with windows 8.1 64 bit domU
> >
> > Il 02/04/2015 16:41, Paul Durrant ha scritto:
> > > > -----Original Message-----
> > > > From: Fabio Fantoni [mailto:fabio.fantoni@xxxxxxx]
> > > > Sent: 02 April 2015 14:58
> > > > To: Paul Durrant; win-pv-devel@xxxxxxxxxxxxxxxxxxxx
> > > > Subject: Re: Multiple save/restore failing with windows 8.1 64 bit domU
> > > >
> > > > Il 01/04/2015 16:43, Fabio Fantoni ha scritto:
> > > > > Il 01/04/2015 15:21, Paul Durrant ha scritto:
> > > > > > > -----Original Message-----
> > > > > > > From: Fabio Fantoni [mailto:fabio.fantoni@xxxxxxx]
> > > > > > > Sent: 30 March 2015 12:00
> > > > > > > To: win-pv-devel@xxxxxxxxxxxxxxxxxxxx
> > > > > > > Cc: Paul Durrant
> > > > > > > Subject: Multiple save/restore failing with windows 8.1 64 bit domU
> > > > > > >
> > > > > > > I tried 2 times multiple save/restore (start domU, use it for some
> > > > > > > minutes, do a save restore, use it, do another save/restore) and in
> > > > > > > both
> > > > > > > case second save failed:
> > > > > > The patch I posted this morning should fix this one for you.
> > > > > >
> > > > > >      Paul
> > > > > Thanks.
> > > > >
> > > > > Today I tried windows 10 pro 64 bit technical preview, clean install,
> > > > > pv drivers (today's build) install ok but after restore display
> > > > > (connecting with spice) seems freezed, waited 4-5 minutes without
> > > > > changes and after I tried also xl shudown but do nothing.
> > > > > DomU have pv disk and network working, xl shutdown (before try
> > > > > save/restore) is working.
> > > > I updated pv drivers to today build and first save/restore now seems was
> > > > working also in Windows 10 but still not the multiple (seem only partial
> > > > solved), xl shutdown this time worked but it tooks some minutes.
> > > > In attachments domU's qemu logs with trace enabled.
> > > It looks like your 2nd save restore worked... certainly log.1 shows the VM
> > coming back from suspend and then going into suspend. The drivers basically
> > seem to be behaving as expected.
> > >     Paul
> > Thanks for fix the first problem, the second remained seems related to
> > save/restore with qxl.
> > With windows 7 freeze for some seconds instead and make me crazy trying
> > to found the cause for 1-2 years without result :(
> > With windows 7 temp. freeze after latest xenvif changes something is
> > changed... after restore freeze of some seconds in qemu log show pv
> > drivers lines from:
> > > xen_platform_log xen platform: XENVIF|PdoRequestEject:
> > > FFFFFA8003067540 (0)
> Hmm. That suggests that, when XENVIF is resuming, for some reason it has 
> decided the backend has gone offline. Not sure why that would happen unless 
> there's some sort of race with net attach.
>
> > And windows popup appair showing that Xen PV Network Device #0 was
> > removed (even if finished the second freeze network return working).
> > Is it normal?
> >
> > I have setted fixed mac address in domU's xl cfg, so should not saw
> > different netcard after save/restore FWIK
> >
> >
> > qemu log (with various trace and debug active) in attachment, first
> > freeze of some seconds was before this line:
> > > xen_platform_log xen platform: XENVIF|PdoRequestEject:
> > > FFFFFA8003067540 (0)
> > And second freeze near was before this line:
> > > xen_platform_log xen platform: XENNET|DriverEntry: XENNET 8.0.0 (18)
> > > (24.03.2015)
> > And third freeze after xl shutdown and before this line:
> > > xen_platform_log xen platform: XEN|SystemPowerStateCallback: <- S0
> > Is there a way to know if there are unexpected case in pv drivers on
> > restore that can cause the freeze or not?
>  From that, not really. Do you know if the guest vcpus are busy during the 
> freeze? Do you have any xentrace analysis suggesting what the guest is doing?
>
>    Paul
I already debugged a crash with gdb but I'm unable to do useful debug 
with temporary freeze :(
During domUs freeze I saw with xentop that domU's cpu sec increasing and 
cpu(%) at 200 (probably for both vcpus at 100%)
I don't have knowledge for good xen debug to find on what instructions 
freeze and/or what is the unexpected case that cause the problem.
Where I can find an howto for do more useful debugging for this case? 
(if exists)
Thanks for any reply and sorry for my bad english.

> > > > If you more informations/tests tell me and I'll post them.
> > > >
> > > > Thanks for any reply and sorry for my bad english.
> > > >
> > > > > About save/restore on W8 I tried now with new pv build and with first
> > > > > save/restore is all ok but with on second now xl save/restore
> > > > > completes correctly but after restore freeze same as the W10 one
> > > > > (since first save/restore).
> > > > >
> > > > > If you more informations/tests tell me and I'll post them.
> > > > >
> > > > > Thanks for any reply and sorry for my bad english.
> > > > >
> > > > >
> > > > >
> > > > > > > > xl -vvv save W8 /mnt/vm/save/W8
> > > > > > > > libxl: debug: libxl.c:6705:libxl_retrieve_domain_configuration: no
> > > > > > > > vtpm from xenstore for domain 40
> > > > > > > > libxl: debug: libxl.c:6707:libxl_retrieve_domain_configuration: no pci
> > > > > > > > from xenstore for domain 40
> > > > > > > > Saving to /mnt/vm/save/W8 new xl format (info 0x1/0x0/1941)
> > > > > > > > libxl: debug: libxl.c:953:libxl_domain_suspend: ao 0x68d910: create:
> > > > > > > > how=(nil) callback=(nil) poller=0x68d970
> > > > > > > > libxl: debug: libxl_dom.c:1573:libxl__toolstack_save: domain=40
> > > > > > > > toolstack data size=49
> > > > > > > > libxl: debug: libxl.c:975:libxl_domain_suspend: ao 0x68d910:
> > > > > > > > inprogress: poller=0x68d970, flags=i
> > > > > > > > libxl-save-helper: debug: starting save: Success
> > > > > > > > xc: detail: xc_domain_save: starting save of domid 40
> > > > > > > > libxl: debug: libxl_dom.c:1278:domain_suspend_callback_common:
> > > > issuing
> > > > > > > > PVHVM suspend request via XenBus control node
> > > > > > > > libxl: debug: libxl_event.c:577:libxl__ev_xswatch_register: watch
> > > > > > > > w=0x68dae0 wpath=/local/domain/40/control/shutdown
> > token=3/0:
> > > > > > > register
> > > > > > > > slotnum=3
> > > > > > > > libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x68dae0
> > > > > > > > wpath=/local/domain/40/control/shutdown token=3/0: event
> > > > > > > > epath=/local/domain/40/control/shutdown
> > > > > > > > libxl: debug: libxl_event.c:615:libxl__ev_xswatch_deregister: watch
> > > > > > > > w=0x68dae0 wpath=/local/domain/40/control/shutdown
> > token=3/0:
> > > > > > > > deregister slotnum=3
> > > > > > > > libxl: debug:
> > libxl_dom.c:1360:domain_suspend_common_pvcontrol_suspending:
> > > > guest
> > > > > > > > acknowledged suspend request
> > > > > > > > libxl: debug:
> > libxl_dom.c:1379:domain_suspend_common_wait_guest:
> > > > > > > wait
> > > > > > > > for the guest to suspend
> > > > > > > > libxl: debug: libxl_event.c:577:libxl__ev_xswatch_register: watch
> > > > > > > > w=0x68daf8 wpath=@releaseDomain token=3/1: register slotnum=3
> > > > > > > > libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x68daf8
> > > > > > > > wpath=@releaseDomain token=3/1: event epath=@releaseDomain
> > > > > > > > libxl: error:
> > libxl_dom.c:1451:suspend_common_wait_guest_timeout:
> > > > > > > > guest did not suspend, timed out
> > > > > > > > libxl: debug: libxl_event.c:615:libxl__ev_xswatch_deregister: watch
> > > > > > > > w=0x68daf8 wpath=@releaseDomain token=3/1: deregister
> > slotnum=3
> > > > > > > > xc: error: Suspend request failed: Internal error
> > > > > > > > xc: error: Domain appears not to have suspended: Internal error
> > > > > > > > libxl-save-helper: xc_domain_save.c:2077: xc_domain_save:
> > Assertion
> > > > > > > > `rc' failed.
> > > > > > > > libxl: error: libxl_utils.c:430:libxl_read_exactly: file/stream
> > > > > > > > truncated reading ipc msg header from domain 40 save/restore
> > helper
> > > > > > > > stdout pipe
> > > > > > > > libxl: error: libxl_exec.c:129:libxl_report_child_exitstatus: domain
> > > > > > > > 40 save/restore helper [-1] died due to fatal signal Aborted
> > > > > > > > libxl: error: libxl_dom.c:2035:remus_teardown_done: Remus: failed
> > to
> > > > > > > > teardown device for guest with domid 40, rc -3
> > > > > > > > libxl: debug: libxl_event.c:1766:libxl__ao_complete: ao 0x68d910:
> > > > > > > > complete, rc=-3
> > > > > > > > libxl: debug: libxl_event.c:1738:libxl__ao__destroy: ao 0x68d910:
> > > > > > > > destroy
> > > > > > > > Failed to save domain, resuming domain
> > > > > > > > libxl: debug: libxl.c:542:libxl_domain_resume: ao 0x68d910: create:
> > > > > > > > how=(nil) callback=(nil) poller=0x68d970
> > > > > > > > xc: error: Dom 40 not suspended: (shutdown 0, reason 255): Internal
> > > > > > > > error
> > > > > > > > libxl: error: libxl.c:511:libxl__domain_resume: xc_domain_resume
> > > > > > > > failed for domain 40: Invalid argument
> > > > > > > > libxl: debug: libxl_event.c:1766:libxl__ao_complete: ao 0x68d910:
> > > > > > > > complete, rc=-3
> > > > > > > > libxl: debug: libxl.c:545:libxl_domain_resume: ao 0x68d910:
> > > > > > > > inprogress: poller=0x68d970, flags=ic
> > > > > > > > libxl: debug: libxl_event.c:1738:libxl__ao__destroy: ao 0x68d910:
> > > > > > > > destroy
> > > > > > > > xc: debug: hypercall buffer: total allocations:29 total releases:29
> > > > > > > > xc: debug: hypercall buffer: current allocations:0 maximum
> > > > > > > > allocations:2
> > > > > > > > xc: debug: hypercall buffer: cache current size:2
> > > > > > > > xc: debug: hypercall buffer: cache hits:20 misses:2 toobig:7
> > > > > > > If you need more informations/tests tell me and I'll post them.
> > > > > > >
> > > > > > > Thanks for any reply and sorry for my bad english.

_______________________________________________
win-pv-devel mailing list
win-pv-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/win-pv-devel