Re: [Minios-devel] Mini-OS Support for Suspend

[Juergen Gross <lists@xxxxxxxxx>]

On 07/12/17 00:53, Bruno Alvisio wrote:
> Hi all,
> 
> I wanted to be able to save and suspend mini-os but found that mini-os
> doesn't handle "suspend" requests.
> I also found that the mini-os repository from NEC Labs Systems and
> Machine Learning (sysml) has some support to handle suspends: There is a
> kernel_suspend() function that does the suspend operations such as
> unmapping the shared info page and suspending Xenstore.
> I have tried to port the suspend functions from the sysml mini-os
> repository to the current xenbits mini-os repository but I was not able
> to make suspend work. Also looked at what the mirage OS does for suspend
> and replicate it without luck. The branch can be found in:
> 
> https://github.com/balvisio/mini-os/tree/feature/mini-os-suspend-support

I very much appreciate your efforts!

> Currently the "xl save" command fails with the following messages:
> 
> root@debianxen:/home/balvisio# xl -vvv save minios minios.chkpt
> 
> libxl: debug: libxl_domain.c:1687:libxl_retrieve_domain_configuration:
> Domain 58:No disk from xenstore
> 
> libxl: debug: libxl_domain.c:1687:libxl_retrieve_domain_configuration:
> Domain 58:No nic from xenstore
> 
> libxl: debug: libxl_domain.c:1687:libxl_retrieve_domain_configuration:
> Domain 58:No vtpm from xenstore
> 
> libxl: debug: libxl_domain.c:1687:libxl_retrieve_domain_configuration:
> Domain 58:No usbctrl from xenstore
> 
> libxl: debug: libxl_domain.c:1687:libxl_retrieve_domain_configuration:
> Domain 58:No usbdev from xenstore
> 
> libxl: debug: libxl_domain.c:1687:libxl_retrieve_domain_configuration:
> Domain 58:No pci from xenstore
> 
> libxl: debug: libxl_domain.c:1687:libxl_retrieve_domain_configuration:
> Domain 58:No vdispl from xenstore
> 
> Saving to minios.chkpt new xl format (info 0x3/0x0/705)
> 
> libxl: debug: libxl_domain.c:492:libxl_domain_suspend: Domain 58:ao
> 0x560af0252d40: create: how=(nil) callback=(nil) poller=0x560af024fb30
> 
> libxl: debug: libxl.c:719:libxl__fd_flags_modify_save: fnctl F_GETFL
> flags for fd 9 are 0x8001
> 
> libxl: debug: libxl.c:727:libxl__fd_flags_modify_save: fnctl F_SETFL of
> fd 9 to 0x8001
> 
> libxl: debug: libxl_domain.c:520:libxl_domain_suspend: Domain 58:ao
> 0x560af0252d40: inprogress: poller=0x560af024fb30, flags=i
> 
> libxl-save-helper: debug: starting save: Success
> 
> xc: detail: fd 9, dom 58, flags 0, hvm 0
> 
> xc: info: Saving domain 58, type x86 PV
> 
> xc: detail: 64 bits, 4 levels
> 
> xc: detail: max_mfn 0x440000
> 
> xc: detail: max_pfn 0x3fff, p2m_frames 32
> 
> libxl: debug: libxl_dom_suspend.c:191:domain_suspend_callback_common:
> Domain 58:issuing PV suspend request via XenBus control node
> 
> libxl: debug: libxl_event.c:636:libxl__ev_xswatch_register: watch
> w=0x560af02502d8 wpath=/local/domain/58/control/shutdown token=3/0:
> register slotnum=3
> 
> libxl: debug: libxl_event.c:573:watchfd_callback: watch w=0x560af02502d8
> wpath=/local/domain/58/control/shutdown token=3/0: event
> epath=/local/domain/58/control/shutdown
> 
> libxl: debug: libxl_event.c:673:libxl__ev_xswatch_deregister: watch
> w=0x560af02502d8 wpath=/local/domain/58/control/shutdown token=3/0:
> deregister slotnum=3
> 
> libxl: debug:
> libxl_dom_suspend.c:273:domain_suspend_common_pvcontrol_suspending:
> Domain 58:guest acknowledged suspend request
> 
> libxl: debug: libxl_dom_suspend.c:292:domain_suspend_common_wait_guest:
> Domain 58:wait for the guest to suspend
> 
> libxl: debug: libxl_event.c:636:libxl__ev_xswatch_register: watch
> w=0x560af02502f0 wpath=@releaseDomain token=3/1: register slotnum=3
> 
> libxl: debug: libxl_event.c:573:watchfd_callback: watch w=0x560af02502f0
> wpath=@releaseDomain token=3/1: event epath=@releaseDomain
> 
> libxl: debug: libxl_dom_suspend.c:346:suspend_common_wait_guest_check:
> Domain 58:guest we were suspending has shut down with unexpected reason
> code 0

So your domain just exited.

You should set all the shutdown actions in your domain config to
"preserve", e.g.:

on_poweroff="preserve"
on_reboot="preserve"
on_crash="preserve"

And then you can use

/usr/lib/xen/bin/xenctx <domid>

to get a stack backtrace of the exit path.

Some more notes on your work so far:

- In the end try to split up your modifications in multiple patches.
  This will make reviewing much easier.

- For full support you need to suspend _all_ devices - there might be
  multiple console devices, ...

- Maybe its easier to start with a pvh domain. You don't have to worry
  about all the p2m stuff. pv-domain support can be added after that.


Juergen

_______________________________________________
Minios-devel mailing list
Minios-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/minios-devel