[Top] [All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] Live migration issues

I have been having a issue recently with the xm migrate feature. It seems to me that the migration begins to take place however somewhere in the middle of the migration things seem to fail. 
I run this command..
root@xen900:/xen-images# xm migrate --live feisty_base 192.168.0.1


and to check and see if the system has made the connection I did
root@xen901:/var/log/xen# netstat -an|grep 8002
tcp 0 0 0.0.0.0:8002 0.0.0.0:* LISTEN tcp 0 0 192.168.0.1:8002 192.168.0.2:60671 ESTABLISHED 
this is from the other xen host.
after a few moments the xm migrate command completes. Reviewing the logs it seems to be a issue on the receiving host. I am running the host machines with ubuntu 7.04. 
there are a few logs on both machines.. from xen900
/var/log/xen/xen-hotplug.log reads

Nothing to flush.
Nothing to flush.
Nothing to flush.
Nothing to flush.
SIOCGIFFLAGS: No such device
xenstore-read: couldn't read path backend/vbd/5/769/node
xenstore-read: couldn't read path backend/vbd/5/770/node
Nothing to flush.
Nothing to flush.
xenstore-read: couldn't read path backend/vbd/7/769/node
xenstore-read: couldn't read path backend/vbd/7/770/node
Nothing to flush.
Nothing to flush.
xenstore-read: couldn't read path backend/vbd/9/770/node
xenstore-read: couldn't read path backend/vbd/9/769/node
Nothing to flush.
Nothing to flush.

(xen900)/var/xen/xend.log reads
[2007-09-20 10:16:32 xend 15789] INFO (XendCheckpoint:239) 4: sent 81, skipped 0, delta 9ms, dom0 100%, target 0%, sent 294Mb/s, dirtied 294Mb/s $ [2007-09-20 10:16:32 xend 15789] INFO (XendCheckpoint:239) Total pages sent= 130884 (0.98x) [2007-09-20 10:16:32 xend 15789] INFO (XendCheckpoint:239) (of which 0 were fixups) [2007-09-20 10:16:32 xend 15789] INFO (XendCheckpoint:239) All memory is saved 
[2007-09-20 10:16:32 xend 15789] INFO (XendCheckpoint:239) Save exit rc=0
[2007-09-20 10:16:32 xend.XendDomainInfo 15789] DEBUG (XendDomainInfo:1466) XendDomainInfo.destroyDomain(11) [2007-09-20 10:16:32 xend.XendDomainInfo 15789] INFO (XendDomainInfo:1564) Dev 769 still active, looping... [2007-09-20 10:16:32 xend.XendDomainInfo 15789] INFO (XendDomainInfo:1564) Dev 770 still active, looping... [2007-09-20 10:16:32 xend.XendDomainInfo 15789] INFO (XendDomainInfo:1564) Dev 769 still active, looping... [2007-09-20 10:16:32 xend.XendDomainInfo 15789] INFO (XendDomainInfo:1564) Dev 770 still active, looping... [2007-09-20 10:16:32 xend.XendDomainInfo 15789] INFO (XendDomainInfo:1564) Dev 769 still active, looping... [2007-09-20 10:16:32 xend.XendDomainInfo 15789] INFO (XendDomainInfo:1564) Dev 770 still active, looping...
(xen900)/var/xen/xend-debug.log reads(these logs are not timestamped so I am not sure if they are from this run) 

Traceback (most recent call last):
 File "SocketServer.py", line 464, in process_request_thread
   self.finish_request(request, client_address)
 File "SocketServer.py", line 254, in finish_request
   self.RequestHandlerClass(request, client_address, self)
 File "SocketServer.py", line 522, in __init__
   self.handle()
 File "BaseHTTPServer.py", line 316, in handle
   self.handle_one_request()
 File "BaseHTTPServer.py", line 310, in handle_one_request
   method()
File "/usr/lib/python2.5/site-packages/xen/util/xmlrpclib2.py", line 66, in do_POST 
   self.send_response(200)
 File "BaseHTTPServer.py", line 368, in send_response
   (self.protocol_version, code, message))
 File "socket.py", line 262, in write
   self.flush()
 File "socket.py", line 249, in flush
   self._sock.sendall(buffer)
error: (32, 'Broken pipe')



on xen901 /var/xen/xend.log reads
[2007-09-20 10:15:45 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:265) XendDomainInfo.restore(['domain', ['domid', '11'], ['uuid', '9e934f3b-1$ [2007-09-20 10:15:45 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:296) parseConfig: config is ['domain', ['domid', '11'], ['uuid', '9e934f3b-1$ [2007-09-20 10:15:45 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:395) parseConfig: result is {'shadow_memory': 0, 'uuid': '9e934f3b-10b2-a40f$ [2007-09-20 10:15:45 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:1262) XendDomainInfo.construct: None [2007-09-20 10:15:45 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:713) Storing VM details: {'shadow_memory': '0', 'uuid': '9e934f3b-10b2-a40f-$ [2007-09-20 10:15:45 xend 6492] DEBUG (DevController:110) DevController: writing {'backend-id': '0', 'mac': '00:16:3e:5b:e6:66', 'handle': '0', '$ [2007-09-20 10:15:45 xend 6492] DEBUG (DevController:112) DevController: writing {'bridge': 'xenbr0', 'domain': 'feisty_base', 'handle': '0', 'sc$ [2007-09-20 10:15:45 xend 6492] DEBUG (blkif:24) exception looking up device number for hda1: [Errno 2] No such file or directory: '/dev/hda1' [2007-09-20 10:15:45 xend 6492] DEBUG (DevController:110) DevController: writing {'backend-id': '0', 'virtual-device': '769', 'device-type': 'dis$ [2007-09-20 10:15:45 xend 6492] DEBUG (DevController:112) DevController: writing {'domain': 'feisty_base', 'frontend': '/local/domain/6/device/vb$ [2007-09-20 10:15:45 xend 6492] DEBUG (blkif:24) exception looking up device number for hda2: [Errno 2] No such file or directory: '/dev/hda2' [2007-09-20 10:15:45 xend 6492] DEBUG (DevController:110) DevController: writing {'backend-id': '0', 'virtual-device': '770', 'device-type': 'dis$ [2007-09-20 10:15:45 xend 6492] DEBUG (DevController:112) DevController: writing {'domain': 'feisty_base', 'frontend': '/local/domain/6/device/vb$ [2007-09-20 10:15:45 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:748) Storing domain details: {'console/port': '2', 'name': 'feisty_base', 'c$ [2007-09-20 10:15:45 xend 6492] DEBUG (balloon:127) Balloon: 533004 KiB free; need 532480; done. [2007-09-20 10:15:45 xend 6492] DEBUG (XendCheckpoint:155) [xc_restore]: /usr/lib/xen/bin/xc_restore 18 6 133120 1 2 [2007-09-20 10:15:45 xend 6492] INFO (XendCheckpoint:239) xc_linux_restore start: max_pfn = 20800 [2007-09-20 10:15:45 xend 6492] INFO (XendCheckpoint:239) Increased domain reservation by 82000 KB [2007-09-20 10:15:45 xend 6492] INFO (XendCheckpoint:239) Reloading memory pages: 0% [2007-09-20 10:16:30 xend 6492] INFO (XendCheckpoint:239) Received all pages (0 races) 
[2007-09-20 10:16:30 xend 6492] INFO (XendCheckpoint:239) ^H^H^H^H100%
[2007-09-20 10:16:30 xend 6492] INFO (XendCheckpoint:239) Memory reloaded.
[2007-09-20 10:16:30 xend 6492] INFO (XendCheckpoint:239) Decreased reservation by 2409 pages [2007-09-20 10:16:30 xend 6492] INFO (XendCheckpoint:239) Domain ready to be built. [2007-09-20 10:16:30 xend 6492] INFO (XendCheckpoint:239) Restore exit with rc=0 
[2007-09-20 10:16:30 xend 6492] DEBUG (XendCheckpoint:209) store-mfn 565003
[2007-09-20 10:16:30 xend 6492] DEBUG (XendCheckpoint:209) console-mfn 561602 [2007-09-20 10:16:31 xend 6492] DEBUG (DevController:143) Waiting for devices vif. 
[2007-09-20 10:16:31 xend 6492] DEBUG (DevController:149) Waiting for 0.
[2007-09-20 10:16:31 xend 6492] DEBUG (DevController:464) hotplugStatusCallback /local/domain/0/backend/vif/6/0/hotplug-status. [2007-09-20 10:16:31 xend 6492] DEBUG (DevController:478) hotplugStatusCallback 1. [2007-09-20 10:16:31 xend 6492] DEBUG (DevController:143) Waiting for devices usb. [2007-09-20 10:16:31 xend 6492] DEBUG (DevController:143) Waiting for devices vbd. 
[2007-09-20 10:16:31 xend 6492] DEBUG (DevController:149) Waiting for 769.
[2007-09-20 10:16:31 xend 6492] DEBUG (DevController:464) hotplugStatusCallback /local/domain/0/backend/vbd/6/769/hotplug-status. [2007-09-20 10:16:31 xend 6492] DEBUG (DevController:478) hotplugStatusCallback 2. [2007-09-20 10:16:31 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:1458) XendDomainInfo.destroy: domid=6 [2007-09-20 10:16:31 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:1466) XendDomainInfo.destroyDomain(6) 
[2007-09-20 10:16:31 xend 6492] ERROR (__init__:1021) Restore failed
Traceback (most recent call last):
File "/usr/lib/python2.5/site-packages/xen/xend/XendDomain.py", line 263, in domain_restore_fd 
   return XendCheckpoint.restore(self, fd)
File "/usr/lib/python2.5/site-packages/xen/xend/XendCheckpoint.py", line 165, in restore 
   dominfo.waitForDevices() # Wait for backends to set up
File "/usr/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py", line 1605, in waitForDevices 
   self.waitForDevices_(c)
File "/usr/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py", line 1099, in waitForDevices_ 
   return self.getDeviceController(deviceClass).waitForDevices()
File "/usr/lib/python2.5/site-packages/xen/xend/server/DevController.py", line 145, in waitForDevices 
   return map(self.waitForDevice, self.deviceIDs())
File "/usr/lib/python2.5/site-packages/xen/xend/server/DevController.py", line 163, in waitForDevice 
   (devid, self.deviceClass))
VmError: Device 769 (vbd) could not be connected. Backend device not found.
there is nothing notable in the other logs on xen901. 
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

 

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.

Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.