[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] Re: XEN live migration: cannot console or ssh to the migrated guest VM (domU)



Wenda Ni <wonda.ni <at> gmail.com> writes:

> 
> 
> I am now testing XEN live migration on two physical hosts with XEN 4.0.1 
> pvops 
Ubuntu 10.10 (2.6.35-22). Host A also acts as a NFS server, and Host B acts as 
a 
NFS client. 
> When I migrate a guest domain from B to A. The ssh connection experiences a 
downtime of about 1 minute (the terminal does not react to the keyboard input 
till about 1 minute later). However, the "sudo xm console" does not work unless 
a reboot is triggered. 
> When I migrate the guest domain from A to back B, the situation is even worse 
-- neither ssh connection nor console works. 
> In both migration cases, you can notice that the guest domain is there by 
using "sudo xm list". Could anyone shed some light on the potential problems I 
am facing? I have been working on this for quite some time, but have no idea 
what goes wrong. The following is the log file copied from host B, when the 
guest domain is migrated from A back to B.[2011-02-06 23:35:51 5208] DEBUG 
(XendDomainInfo:235) XendDomainInfo.restore(['domain', ['domid', '8'], 
['cpu_weight', '256'], ['cpu_cap', '0'], ['bootloader', '/usr/bin/pygrub'], 
['on_crash', 'restart'], ['uuid', 'aaf5c3c7-a5a6-d306-dc77-483cea04bff5'], 
['bootloader_args', '-q'], ['vcpus', '1'], ['description'], ['name', 'Elite'], 
['cpus', [[]]], ['on_reboot', 'restart'], ['on_poweroff', 'destroy'], 
['maxmem', 
'128'], ['memory', '128'], ['shadow_memory', '0'], ['vcpu_avail', '1'], 
['features'], ['on_xend_start', 'ignore'], ['on_xend_stop', 'ignore'], 
['start_time', '1297053017.52'], ['cpu_time', '7.409878829'], ['online_vcpus', 
'1'], ['image', ['linux', ['kernel'], ['args', 'root=/dev/sda2 ro '], 
['superpages', '0'], ['videoram', '4'], ['pci', []], ['nomigrate', '0'], 
['tsc_mode', '0'], ['notes', ['HV_START_LOW', '4118806528'], ['FEATURES', 
'!writable_page_tables|pae_pgdir_above_4gb'], ['VIRT_BASE', '3221225472'], 
['GUEST_VERSION', '2.6'], ['PADDR_OFFSET', '0'], ['GUEST_OS', 'linux'], 
['HYPERCALL_PAGE', '3222282240'], ['LOADER', 'generic'], ['SUSPEND_CANCEL', 
'1'], ['PAE_MODE', 'yes'], ['ENTRY', '3229974528'], ['XEN_VERSION', 'xen-
3.0']]]], ['status', '2'], ['state', '-b----'], ['store_mfn', '472195'], 
['console_mfn', '472194'], ['device', ['vif', ['ip', '138.15.106.226'], ['mac', 
'00:16:3E:E9:17:69'], ['script', '/etc/xen/scripts/vif-bridge'], ['uuid', 
'b3761d83-4167-d29e-2833-98065be549c8'], ['backend', '0']]], ['device', 
['console', ['protocol', 'vt100'], ['location', '2'], ['uuid', '99ef56f7-bbd9-
0a3c-277e-bbcc8f5cebce']]], ['device', ['vbd', ['protocol', 'x86_32-abi'], 
['uuid', '638d020f-b8cd-4100-69a0-c3efc5334b08'], ['bootable', '1'], ['dev', 
'sda2:disk'], ['uname', 'file:/home/user1/vm-storage-
export/domains/Elite/disk.img'], ['mode', 'w'], ['backend', '0'], ['VDI']]], 
['device', ['vbd', ['protocol', 'x86_32-abi'], ['uuid', 'dac29d55-cda5-29c5-
b362-dee4b24e60d6'], ['bootable', '0'], ['dev', 'sda1:disk'], ['uname', 
'file:/home/user1/vm-storage-export/domains/Elite/swap.img'], ['mode', 'w'], 
['backend', '0'], ['VDI']]], ['change_home_server', 'False']])[2011-02-06 
23:35:51 5208] DEBUG (XendDomainInfo:2508) XendDomainInfo.constructDomain[2011-
02-06 23:35:51 5208] DEBUG (balloon:220) Balloon: 263812 KiB free; need 16384; 
done.[2011-02-06 23:35:51 5208] DEBUG (XendDomain:464) Adding Domain: 15[2011-
02-06 23:35:51 5208] DEBUG (XendDomainInfo:3400) Storing VM details: 
{'on_xend_stop': 'ignore', 'shadow_memory': '0', 'uuid': 'aaf5c3c7-a5a6-d306-
dc77-483cea04bff5', 'on_reboot': 'restart', 'start_time': '1297053017.52', 
'on_poweroff': 'destroy', 'bootloader_args': '-q', 'on_xend_start': 'ignore', 
'on_crash': 'restart', 'xend/restart_count': '0', 'vcpus': '1', 'vcpu_avail': 
'1', 'bootloader': '/usr/bin/pygrub', 'image': "(linux (kernel ) (args 
'root=/dev/sda2 ro ') (superpages 0) (videoram 4) (pci ()) (nomigrate 0) 
(tsc_mode 0) (notes (HV_START_LOW 4118806528) (FEATURES 
'!writable_page_tables|pae_pgdir_above_4gb') (VIRT_BASE 3221225472) 
(GUEST_VERSION 2.6) (PADDR_OFFSET 0) (GUEST_OS linux) (HYPERCALL_PAGE 
3222282240) (LOADER generic) (SUSPEND_CANCEL 1) (PAE_MODE yes) (ENTRY 
3229974528) (XEN_VERSION xen-3.0)))", 'name': 'Elite'}[2011-02-06 23:35:51 
5208] 
INFO (XendDomainInfo:2367) createDevice: console : {'protocol': 'vt100', 
'location': '2', 'uuid': '99ef56f7-bbd9-0a3c-277e-bbcc8f5cebce'}[2011-02-06 
23:35:51 5208] DEBUG (DevController:95) DevController: writing {'state': '1', 
'backend-id': '0', 'backend': '/local/domain/0/backend/console/15/0'} to 
/local/domain/15/device/console/0.[2011-02-06 23:35:51 5208] DEBUG 
(DevController:97) DevController: writing {'domain': 'Elite', 'frontend': 
'/local/domain/15/device/console/0', 'uuid': '99ef56f7-bbd9-0a3c-277e-
bbcc8f5cebce', 'frontend-id': '15', 'state': '1', 'location': '2', 'online': 
'1', 'protocol': 'vt100'} to /local/domain/0/backend/console/15/0.[2011-02-06 
23:35:51 5208] INFO (XendDomainInfo:2367) createDevice: vbd : {'protocol': 
'x86_32-abi', 'uuid': '638d020f-b8cd-4100-69a0-c3efc5334b08', 'bootable': 1, 
'driver': 'paravirtualised', 'dev': 'sda2:disk', 'uname': 'file:/home/user1/vm-
storage-export/domains/Elite/disk.img', 'mode': 'w', 'backend': '0'}[2011-02-06 
23:35:51 5208] DEBUG (DevController:95) DevController: writing {'virtual-
device': '2050', 'protocol': 'x86_32-abi', 'device-type': 'disk', 'backend-id': 
'0', 'state': '1', 'backend': '/local/domain/0/backend/vbd/15/2050'} to 
/local/domain/15/device/vbd/2050.[2011-02-06 23:35:51 5208] DEBUG 
(DevController:97) DevController: writing {'domain': 'Elite', 'frontend': 
'/local/domain/15/device/vbd/2050', 'uuid': '638d020f-b8cd-4100-69a0-
c3efc5334b08', 'bootable': '1', 'dev': 'sda2', 'state': '1', 'params': 
'/home/user1/vm-storage-export/domains/Elite/disk.img', 'mode': 'w', 'online': 
'1', 'frontend-id': '15', 'type': 'file'} to 
/local/domain/0/backend/vbd/15/2050.[2011-02-06 23:35:51 5208] INFO 
(XendDomainInfo:2367) createDevice: vbd : {'protocol': 'x86_32-abi', 'uuid': 
'dac29d55-cda5-29c5-b362-dee4b24e60d6', 'bootable': 0, 'driver': 
'paravirtualised', 'dev': 'sda1:disk', 'uname': 'file:/home/user1/vm-storage-
export/domains/Elite/swap.img', 'mode': 'w', 'backend': '0'}[2011-02-06 
23:35:51 
5208] DEBUG (DevController:95) DevController: writing {'virtual-device': 
'2049', 
'protocol': 'x86_32-abi', 'device-type': 'disk', 'backend-id': '0', 'state': 
'1', 'backend': '/local/domain/0/backend/vbd/15/2049'} to 
/local/domain/15/device/vbd/2049.[2011-02-06 23:35:51 5208] DEBUG 
(DevController:97) DevController: writing {'domain': 'Elite', 'frontend': 
'/local/domain/15/device/vbd/2049', 'uuid': 'dac29d55-cda5-29c5-b362-
dee4b24e60d6', 'bootable': '0', 'dev': 'sda1', 'state': '1', 'params': 
'/home/user1/vm-storage-export/domains/Elite/swap.img', 'mode': 'w', 'online': 
'1', 'frontend-id': '15', 'type': 'file'} to 
/local/domain/0/backend/vbd/15/2049.[2011-02-06 23:35:51 5208] INFO 
(XendDomainInfo:2367) createDevice: vif : {'ip': '138.15.106.226', 'mac': 
'00:16:3E:E9:17:69', 'script': '/etc/xen/scripts/vif-bridge', 'uuid': 'b3761d83-
4167-d29e-2833-98065be549c8', 'backend': '0'}[2011-02-06 23:35:51 5208] DEBUG 
(DevController:95) DevController: writing {'backend-id': '0', 'mac': 
'00:16:3E:E9:17:69', 'handle': '0', 'state': '1', 'backend': 
'/local/domain/0/backend/vif/15/0'} to 
/local/domain/15/device/vif/0.[2011-02-06 
23:35:51 5208] DEBUG (DevController:97) DevController: writing {'domain': 
'Elite', 'handle': '0', 'uuid': 'b3761d83-4167-d29e-2833-98065be549c8', 
'script': '/etc/xen/scripts/vif-bridge', 'ip': '138.15.106.226', 'mac': 
'00:16:3E:E9:17:69', 'frontend-id': '15', 'state': '1', 'online': '1', 
'frontend': '/local/domain/15/device/vif/0'} to 
/local/domain/0/backend/vif/15/0.[2011-02-06 23:35:51 5208] DEBUG 
(XendDomainInfo:1804) Storing domain details: {'image/entry': '3229974528', 
'console/port': '2', 'image/loader': 'generic', 'vm': '/vm/aaf5c3c7-a5a6-d306-
dc77-483cea04bff5', 'control/platform-feature-multiprocessor-suspend': '1', 
'image/hv-start-low': '4118806528', 'image/guest-os': 'linux', 'image/virt-
base': '3221225472', 'memory/target': '131072', 'image/guest-version': '2.6', 
'image/pae-mode': 'yes', 'description': '', 'console/limit': '1048576', 
'image/paddr-offset': '0', 'image/hypercall-page': '3222282240', 'image/suspend-
cancel': '1', 'cpu/0/availability': 'online', 'image/features/pae-pgdir-above-
4gb': '1', 'image/features/writable-page-tables': '0', 'console/type': 
'xenconsoled', 'name': 'Elite', 'domid': '15', 'image/xen-version': 'xen-3.0', 
'store/port': '1'}[2011-02-06 23:35:52 5208] DEBUG (XendCheckpoint:286) 
restore:shadow=0x0, _static_max=0x8000000, _static_min=0x0, [2011-02-06 
23:35:52 
5208] DEBUG (XendCheckpoint:305) [xc_restore]: /usr/lib/xen/bin/xc_restore 4 15 
1 2 0 0 0 0[2011-02-06 23:35:52 5208] INFO (XendCheckpoint:423) 
xc_domain_restore start: p2m_size = 8000[2011-02-06 23:35:52 5208] INFO 
(XendCheckpoint:423) Reloading memory pages:ÂÂ 0%[2011-02-06 23:36:03 5208] 
INFO 
(XendCheckpoint:423) ERROR Internal error: Error when reading batch size[2011-
02-06 23:36:03 5208] INFO (XendCheckpoint:423) ERROR Internal error: error when 
buffering batch, finishing[2011-02-06 23:36:03 5208] INFO (XendCheckpoint:423) 
[2011-02-06 23:36:03 5208] INFO (XendCheckpoint:423) 100%[2011-02-06 23:36:03 
5208] INFO (XendCheckpoint:423) Memory reloaded (9067 pages)[2011-02-06 
23:36:03 
5208] INFO (XendCheckpoint:423) read VCPU 0[2011-02-06 23:36:03 5208] INFO 
(XendCheckpoint:423) Completed checkpoint load[2011-02-06 23:36:03 5208] INFO 
(XendCheckpoint:423) Domain ready to be built.[2011-02-06 23:36:03 5208] INFO 
(XendCheckpoint:423) Restore exit with rc=0[2011-02-06 23:36:03 5208] DEBUG 
(XendCheckpoint:394) store-mfn 163723[2011-02-06 23:36:03 5208] DEBUG 
(XendCheckpoint:394) console-mfn 163722[2011-02-06 23:36:03 5208] DEBUG 
(XendDomainInfo:2992) XendDomainInfo.completeRestore[2011-02-06 23:36:03 5208] 
DEBUG (XendDomainInfo:1804) Storing domain details: {'console/ring-ref': 
'163722', 'image/entry': '3229974528', 'console/port': '2', 'store/ring> -ref': 
'163723', 'image/loader': 'generic', 'vm': '/vm/aaf5c3c7-a5a6-d306-dc77-
483cea04bff5', 'control/platform-feature-multiprocessor-suspend': '1', 
'image/hv-start-low': '4118806528', 'image/guest-os': 'linux', 'image/virt-
base': '3221225472', 'memory/target': '131072', 'image/guest-version': '2.6', 
'image/pae-mode': 'yes', 'description': '', 'console/limit': '1048576', 
'image/paddr-offset': '0', 'image/hypercall-page': '3222282240', 'image/suspend-
cancel': '1', 'cpu/0/availability': 'online', 'image/features/pae-pgdir-above-
4gb': '1', 'image/features/writable-page-tables': '0', 'console/type': 
'xenconsoled', 'name': 'Elite', 'domid': '15', 'image/xen-version': 'xen-3.0', 
'store/port': '1'}[2011-02-06 23:36:03 5208] DEBUG (XendDomainInfo:3005) 
XendDomainInfo.completeRestore done[2011-02-06 23:36:03 5208] DEBUG 
(DevController:139) Waiting for devices tap2.[2011-02-06 23:36:03 5208] DEBUG 
(XendDomainInfo:1891) XendDomainInfo.handleShutdownWatch[2011-02-06 23:36:03 
5208] DEBUG (DevController:139) Waiting for devices vif.[2011-02-06 23:36:03 
5208] DEBUG (DevController:144) Waiting for 0.[2011-02-06 23:36:03 5208] DEBUG 
(DevController:628) hotplugStatusCallback 
/local/domain/0/backend/vif/15/0/hotplug-status.[2011-02-06 23:36:03 5208] 
DEBUG 
(DevController:642) hotplugStatusCallback 1.[2011-02-06 23:36:03 5208] DEBUG 
(DevController:139) Waiting for devices vkbd.[2011-02-06 23:36:03 5208] DEBUG 
(DevController:139) Waiting for devices ioports.[2011-02-06 23:36:03 5208] 
DEBUG 
(DevController:139) Waiting for devices tap.[2011-02-06 23:36:03 5208] DEBUG 
(DevController:139) Waiting for devices vif2.[2011-02-06 23:36:03 5208] DEBUG 
(DevController:139) Waiting for devices console.[2011-02-06 23:36:03 5208] 
DEBUG 
(DevController:144) Waiting for 0.[2011-02-06 23:36:03 5208] DEBUG 
(DevController:139) Waiting for devices vscsi.[2011-02-06 23:36:03 5208] DEBUG 
(DevController:139) Waiting for devices vbd.[2011-02-06 23:36:03 5208] DEBUG 
(DevController:144) Waiting for 2050.[2011-02-06 23:36:03 5208] DEBUG 
(DevController:628) hotplugStatusCallback 
/local/domain/0/backend/vbd/15/2050/hotplug-status.[2011-02-06 23:36:03 5208] 
DEBUG (DevController:642) hotplugStatusCallback 1.[2011-02-06 23:36:03 5208] 
DEBUG (DevController:144) Waiting for 2049.[2011-02-06 23:36:03 5208] DEBUG 
(DevController:628) hotplugStatusCallback 
/local/domain/0/backend/vbd/15/2049/hotplug-status.[2011-02-06 23:36:03 5208] 
DEBUG (DevController:642) hotplugStatusCallback 1.[2011-02-06 23:36:03 5208] 
DEBUG (DevController:139) Waiting for devices irq.[2011-02-06 23:36:03 5208] 
DEBUG (DevController:139) Waiting for devices vfb.[2011-02-06 23:36:03 5208] 
DEBUG (DevController:139) Waiting for devices pci.[2011-02-06 23:36:03 5208] 
DEBUG (DevController:139) Waiting for devices vusb.[2011-02-06 23:36:03 5208] 
DEBUG (DevController:139) Waiting for devices vtpm.-- Cheers,
> Â
> Wenda Ni, Ph.D.
> Dept. of Computer Science and Engineering
> State University of New York (SUNY) at Buffalo
> T: 716 982 6746
> 
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users <at> lists.xensource.com
> http://lists.xensource.com/xen-users


Were you able to resolve the problem? I am also experiencing the same problem 
when using ubuntu 10.10 as DomU. Console as well as ssh connection both freezes 
for 10 mins in both the scenarios (A -> B or B -> A).


Hitesh Khandelwal | Graduate Student | Purdue University | 
http://www.cs.purdue.edu/~hkhande


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.