[Top] [All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] DomUs blocked

Hi

I'm using Xen on my  AMD64 / Ubuntu 7.10 server which I recently upgraded, and 
my 2 VMs are decidedly flaky. They seem to be blocked for most of the time, 
and networking doesn't work properly. How can I debug this and get them 
online reliably? Any tips gratefully received!

* I can log in via console or ssh, can ping the server's NICs but can't ping 
the other VM or beyond the machine. From the server I can ping the VMs and 
outside.
* Firewall settings on the server are very relaxed at this point.
* The exceptions below about hda1 and hda2 don't make sense to me, since the 
devices are mounted via /etc/fstab as root and swap.
* /var/log/xen/xend-debug.log reports a broken pipe

Thanks!

M

uname string: 
Linux server.home.xx 2.6.19-4-generic-amd64 #2 SMP Sat Oct 6 01:35:53 UTC 2007 
x86_64 GNU/Linux

[11:57:03] $ sudo xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      782     1 r-----    487.2
slimserv                                   5       56     1 ------    183.3
work                                       7      128     1 ------      2.8
[11:57:06] $ sudo xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      782     1 r-----    488.3
slimserv                                   5       56     1 -b----    185.6
work                                       7      128     1 -b----      7.2
[11:57:20] $ sudo xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      782     1 r-----    521.0
slimserv                                   5       56     1 ------    267.3
work                                       7      128     1 -b----      7.3
[12:19:09] $  

Here is an excerpt from the xend-debug.log

Traceback (most recent call last):
  File "SocketServer.py", line 464, in process_request_thread
    self.finish_request(request, client_address)
  File "SocketServer.py", line 254, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "SocketServer.py", line 522, in __init__
    self.handle()
  File "BaseHTTPServer.py", line 316, in handle
    self.handle_one_request()
  File "BaseHTTPServer.py", line 310, in handle_one_request
    method()
  File "/usr/lib/python2.5/site-packages/xen/util/xmlrpclib2.py", line 66, in 
do_POST
    self.send_response(200)
  File "BaseHTTPServer.py", line 368, in send_response
    (self.protocol_version, code, message))
  File "socket.py", line 262, in write
    self.flush()
  File "socket.py", line 249, in flush
    self._sock.sendall(buffer)
error: (32, 'Broken pipe')


Here is a tail excerpt from /var/log/xen/xend.log:

[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:190) 
XendDomainInfo.create(['vm', ['name', 'work'], ['memory', '128'], 
['on_poweroff', 'destroy'], ['on_reboot', 'restart'], 
['on_crash', 'restart'], ['vcpus', 1], ['image', ['linux', 
['kernel', '/boot/vmlinuz-2.6.19-4-generic-amd64'], 
['ramdisk', '/boot/initrd.img-2.6.19-4-generic-amd64'], ['root', '/dev/hda1 
ro']]], ['device', ['vbd', ['uname', 'file:/var/xen/domains/work/disk.img'], 
['dev', 'hda1'], ['mode', 'w']]], ['device', ['vbd', 
['uname', 'file:/var/xen/domains/work/swap.img'], ['dev', 'hda2'], 
['mode', 'w']]], ['device', ['vif', ['ip', '192.168.2.6'], 
['mac', '00:16:3e:b1:d7:91'], ['vifname', 'vif2.0']]]])
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:296) 
parseConfig: config is ['vm', ['name', 'work'], ['memory', '128'], 
['on_poweroff', 'destroy'], ['on_reboot', 'restart'], 
['on_crash', 'restart'], ['vcpus', 1], ['image', ['linux', 
['kernel', '/boot/vmlinuz-2.6.19-4-generic-amd64'], 
['ramdisk', '/boot/initrd.img-2.6.19-4-generic-amd64'], ['root', '/dev/hda1 
ro']]], ['device', ['vbd', ['uname', 'file:/var/xen/domains/work/disk.img'], 
['dev', 'hda1'], ['mode', 'w']]], ['device', ['vbd', 
['uname', 'file:/var/xen/domains/work/swap.img'], ['dev', 'hda2'], 
['mode', 'w']]], ['device', ['vif', ['ip', '192.168.2.6'], 
['mac', '00:16:3e:b1:d7:91'], ['vifname', 'vif2.0']]]]
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:395) 
parseConfig: result is {'shadow_memory': None, 'uuid': 
None, 'on_crash': 'restart', 'on_reboot': 'restart', 'localtime': 
None, 'image': ['linux', ['kernel', '/boot/vmlinuz-2.6.19-4-generic-amd64'], 
['ramdisk', '/boot/initrd.img-2.6.19-4-generic-amd64'], ['root', '/dev/hda1 
ro']], 'on_poweroff': 'destroy', 'bootloader_args': None, 'cpus': 
None, 'name': 'work', 'backend': [], 'vcpus': 1, 'cpu_weight': 
None, 'features': None, 'vcpu_avail': None, 'memory': 128, 'device': [('vbd', 
['vbd', ['uname', 'file:/var/xen/domains/work/disk.img'], ['dev', 'hda1'], 
['mode', 'w']]), ('vbd', ['vbd', 
['uname', 'file:/var/xen/domains/work/swap.img'], ['dev', 'hda2'], 
['mode', 'w']]), ('vif', ['vif', ['ip', '192.168.2.6'], 
['mac', '00:16:3e:b1:d7:91'], ['vifname', 'vif2.0']])], 'bootloader': 
None, 'cpu': None, 'maxmem': None}
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:1262) 
XendDomainInfo.construct: None
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:1294) 
XendDomainInfo.initDomain: 7 1.0
[2007-10-30 11:57:01 xend 16192] DEBUG (balloon:127) Balloon: 131680 KiB free; 
need 131072; done.
[2007-10-30 11:57:01 xend 16192] INFO (image:136) buildDomain os=linux dom=7 
vcpus=1
[2007-10-30 11:57:01 xend 16192] DEBUG (image:199) dom            = 7
[2007-10-30 11:57:01 xend 16192] DEBUG (image:200) image          
= /boot/vmlinuz-2.6.19-4-generic-amd64
[2007-10-30 11:57:01 xend 16192] DEBUG (image:201) store_evtchn   = 1
[2007-10-30 11:57:01 xend 16192] DEBUG (image:202) console_evtchn = 2
[2007-10-30 11:57:01 xend 16192] DEBUG (image:203) cmdline        =  
root=/dev/hda1 ro
[2007-10-30 11:57:01 xend 16192] DEBUG (image:204) ramdisk        
= /boot/initrd.img-2.6.19-4-generic-amd64
[2007-10-30 11:57:01 xend 16192] DEBUG (image:205) vcpus          = 1
[2007-10-30 11:57:01 xend 16192] DEBUG (image:206) features       =
[2007-10-30 11:57:02 xend 16192] DEBUG (blkif:24) exception looking up device 
number for hda1: [Errno 2] No such file or directory: '/dev/hda1'
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:110) DevController: 
writing 
{'backend-id': '0', 'virtual-device': '769', 'device-type': 'disk', 'state': 
'1', 'backend': '/local/domain/0/backend/vbd/7/769'} 
to /local/domain/7/device/vbd/769.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:112) DevController: 
writing 
{'domain': 'work', 'frontend': '/local/domain/7/device/vbd/769', 'dev': 'hda1', 
'state': '1', 'params': '/var/xen/domains/work/disk.img', 'mode': 'w', 
'online': '1', 'frontend-id': '7', 'type': 'file'} 
to /local/domain/0/backend/vbd/7/769.
[2007-10-30 11:57:02 xend 16192] DEBUG (blkif:24) exception looking up device 
number for hda2: [Errno 2] No such file or directory: '/dev/hda2'
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:110) DevController: 
writing 
{'backend-id': '0', 'virtual-device': '770', 'device-type': 'disk', 'state': 
'1', 'backend': '/local/domain/0/backend/vbd/7/770'} 
to /local/domain/7/device/vbd/770.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:112) DevController: 
writing 
{'domain': 'work', 'frontend': '/local/domain/7/device/vbd/770', 'dev': 'hda2', 
'state': '1', 'params': '/var/xen/domains/work/swap.img', 'mode': 'w', 
'online': '1', 'frontend-id': '7', 'type': 'file'} 
to /local/domain/0/backend/vbd/7/770.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:110) DevController: 
writing 
{'backend-id': '0', 'mac': '00:16:3e:b1:d7:91', 'handle': '0', 'state': '1', 
'backend': '/local/domain/0/backend/vif/7/0'} 
to /local/domain/7/device/vif/0.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:112) DevController: 
writing 
{'domain': 'work', 'handle': '0', 'script': '/etc/xen/scripts/vif-route', 'ip': 
'192.168.2.6', 'state': '1', 'frontend': '/local/domain/7/device/vif/0', 'mac': 
'00:16:3e:b1:d7:91', 'vifname': 'vif2.0', 'online': '1', 'frontend-id': '7'} 
to /local/domain/0/backend/vif/7/0.
[2007-10-30 11:57:02 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:713) 
Storing VM details: 
{'shadow_memory': '0', 'uuid': '89708af1-f491-4aed-55e9-1be3acd352b7', 
'on_reboot': 'restart', 'start_time': '1193741822.28', 'on_poweroff': 
'destroy', 'name': 'work', 'xend/restart_count': '0', 'vcpus': '1', 
'vcpu_avail': '1', 'memory': '128', 'on_crash': 'restart', 'image': "(linux 
(kernel /boot/vmlinuz-2.6.19-4-generic-amd64) 
(ramdisk /boot/initrd.img-2.6.19-4-generic-amd64) (root '/dev/hda1 
ro'))", 'maxmem': '128'}
[2007-10-30 11:57:02 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:748) 
Storing domain details: 
{'console/ring-ref': '9890', 'console/port': '2', 'name': 'work', 
'console/limit': '1048576', 'vm': '/vm/89708af1-f491-4aed-55e9-1be3acd352b7', 
'domid': '7', 'cpu/0/availability': 'online', 'memory/target': '131072', 
'store/ring-ref': '9891', 'store/port': '1'}
[2007-10-30 11:57:02 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:938) 
XendDomainInfo.handleShutdownWatch
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:143) Waiting for devices 
vif.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:149) Waiting for 0.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:464) 
hotplugStatusCallback /local/domain/0/backend/vif/7/0/hotplug-status.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:464) 
hotplugStatusCallback /local/domain/0/backend/vif/7/0/hotplug-status.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:478) 
hotplugStatusCallback 1.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:143) Waiting for devices 
usb.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:143) Waiting for devices 
vbd.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:149) Waiting for 769.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:464) 
hotplugStatusCallback /local/domain/0/backend/vbd/7/769/hotplug-status.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:464) 
hotplugStatusCallback /local/domain/0/backend/vbd/7/769/hotplug-status.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:478) 
hotplugStatusCallback 1.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:149) Waiting for 770.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:464) 
hotplugStatusCallback /local/domain/0/backend/vbd/7/770/hotplug-status.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:478) 
hotplugStatusCallback 1.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices 
irq.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices 
pci.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices 
ioports.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices 
tap.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices 
vtpm.
[2007-10-30 11:57:03 xend 16192] INFO (XendDomain:370) Domain work (7) 
unpaused.
[12:26:51] $

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

 

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.

Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.