[Xen-users] Loss of hypervisor control - xm, xentop

[Steve Arntzen <xen@xxxxxxxxxx>]

I am running xen 3.2-1  (Debian 3.2.1-2) on Debian Lenny
(2.6.26-2-xen-amd64 #1 SMP Wed Mar 10) on two identical production
systems.

This morning on one of the systems, xentop stopped displaying.  Running
xentop results in:

unable to open xenstore
: Connection refused
Failed to initialize xenstat library


xm list:
Error: (9, 'Bad file descriptor')


xm dmseg:
Error: Subcommand dmseg not found!



The DomUs appear to be running fine, I just can't monitor or control
them.  Do I need to be concerned of the DomUs stability?

I have found numerous links relating to the same symptoms, but no
solution other than a reboot.  What causes this problem?

As they are running critical services on a production system, my last
resort would be rebooting the Dom0.  Is there anything else I can do?


Please help!

Steve



/var/log/xen/xend.log:

[2010-05-25 09:04:40 4256] ERROR (xmlrpclib2:178) Internal error
handling xend.domains_with_state
Traceback (most recent call last):
  File "/usr/lib/xen-3.2-1/lib/python/xen/util/xmlrpclib2.py", line 131,
in _marshaled_dispatch
    response = self._dispatch(method, params)
  File "/usr/lib/python2.5/SimpleXMLRPCServer.py", line 415, in
_dispatch
    return func(*params)
  File "/usr/lib/xen-3.2-1/lib/python/xen/xend/server/XMLRPCServer.py",
line 66, in domains_with_state
    domains = XendDomain.instance().list_sorted(state)
  File "/usr/lib/xen-3.2-1/lib/python/xen/xend/XendDomain.py", line 837,
in list_sorted
    doms = self.list(state)
  File "/usr/lib/xen-3.2-1/lib/python/xen/xend/XendDomain.py", line 807,
in list
    self._refresh(refresh_shutdown = False)
  File "/usr/lib/xen-3.2-1/lib/python/xen/xend/XendDomain.py", line 396,
in _refresh
    txn = xstransact()
  File "/usr/lib/xen-3.2-1/lib/python/xen/xend/xenstore/xstransact.py",
line 29, in __init__
    self.transaction = xshandle().transaction_start()
Error: (32, 'Broken pipe')



All the same (xen) processes are running on both machines:

ps waux|grep xen
root        63  0.0  0.0      0     0 ?        S<   May19   0:00
[xenwatch]
root        64  0.0  0.0      0     0 ?        S<   May19   0:00
[xenbus]
root      4251  0.0  0.4  95068  9312 ?        S    May19   0:00
python /usr/lib/xen-3.2-1/bin/xend start
root      4254  0.0  0.0  21624  1868 ?        S    May19
0:02 /usr/lib/xen-3.2-1/bin/xenconsoled
root      4256  0.0  1.3 301436 28804 ?        Sl   May19   4:37
python /usr/lib/xen-3.2-1/bin/xend start
root     24361  0.0  0.0   5164   796 pts/7    S+   12:13   0:00 grep
xen






_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users