[Xen-devel] Xend problems

[Kieran Mansley <kmansley@xxxxxxxxxxxxxx>]

There have been a few reports on the xen-devel mailing list (and bugs
filed/reopened on the Xen bugzilla) of problems getting xend to start
recently, so perhaps this is a duplicate, but I'm having problems
getting xend to start with the current xen-unstable tree.  In fact I've
been seeing problems for the last couple of weeks, but having updated
before investigating in detail in case it was already fixed I can't
guarantee that it has always been the same thing.

The problem is basically that after bringing up dom0 successfully
running /etc/init.d/xend seems to hang (i.e. the rest of the machine is
fine) and takes the machine off the network (there are no routes
configured).  The machine being off the network might explain the hang
as it has a number of nfs mounts, and trying to use or search one of
these would obviously cause delays.  There's nothing of interest
in /var/log/xen/xend-debug.log, but the /var/log/xen/xend.log says:

[2007-03-08 14:32:26 3775] INFO (SrvDaemon:322) Xend Daemon started
[2007-03-08 14:32:26 3775] INFO (SrvDaemon:326) Xend changeset:
unavailable.
[2007-03-08 14:32:26 3775] INFO (SrvDaemon:333) Xend version: Unknown.

[2007-03-08 14:32:27 3775] ERROR (XendNode:98) {u'modelname':
'Intel(R) Xeon(R) CPU 5160 @ 3.00GHz', u'vendor': 'GenuineIntel',
u'uuid': '9f594ee5-3f47-f8b5-1d0b-d75f2c4d1b24', u'number': 0,
u'host': 'b9c23064-dba6-03fa-a2bb-5ff53d9c46b5', u'flags': 'fpu tsc
msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr
sse sse2 ss ht tm pbe nx lm constant_tsc pni monitor ds_cpl vmx est
tm2 cx16 xtpr lahf_lm', u'stepping': '6', u'speed': 2992, u'features':
'bfebfbff:20100000:00000000:00000140:0004e3bd:00000000:00000001'}

[2007-03-08 14:32:27 3775] ERROR (XendNode:105) 0

[2007-03-08 14:32:27 3775] ERROR (XendNode:106) {0: {'fpu': 'yes',
'f00f_bug': 'no', 'cpu family': '6', 'model name': 'Intel(R) Xeon(R)
CPU 5160 @ 3.00GHz', 'vendor_id': 'GenuineIntel', 'flags': 'fpu tsc
msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr
sse sse2 ss ht tm pbe nx lm constant_tsc pni monitor ds_cpl vmx est
tm2 cx16 xtpr lahf_lm', 'bogomips': '5987.46', 'hlt_bug': 'no',
'fpu_exception': 'yes', 'stepping': '6', 'wp': 'yes', 'cpuid level':
'10', 'model': '15', 'coma_bug': 'no', 'fdiv_bug': 'no', 'cpu MHz':
'2992.612', 'cache size': '4096 KB'}, 1: {'fpu': 'yes', 'f00f_bug':
'no', 'cpu family': '6', 'model name': 'Intel(R) Xeon(R) CPU 5160 @
3.00GHz', 'vendor_id': 'GenuineIntel', 'flags': 'fpu tsc msr pae mce
cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss
ht tm pbe nx lm constant_tsc up pni monitor ds_cpl vmx est tm2 cx16
xtpr lahf_lm', 'bogomips': '5987.46', 'hlt_bug': 'no',
'fpu_exception': 'yes', 'stepping': '6', 'wp': 'yes', 'cpuid level':
'10', 'model': '15', 'coma_bug': 'no', 'fdiv_bug': 'no', 'cpu MHz':
'2992.612', 'cache size': '4096 KB'}}

[2007-03-08 14:32:27 3775] ERROR (XendNode:98) {u'modelname':
'Intel(R) Xeon(R) CPU 5160 @ 3.00GHz', u'vendor': 'GenuineIntel',
u'uuid': '7fc33166-365a-e7e5-ae7a-f7f2802c1481', u'number': 1,
u'host': 'b9c23064-dba6-03fa-a2bb-5ff53d9c46b5', u'flags': 'fpu tsc
msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr
sse sse2 ss ht tm pbe nx lm constant_tsc up pni monitor ds_cpl vmx est
tm2 cx16 xtpr lahf_lm', u'stepping': '6', u'speed': 2992, u'features':
'bfebfbff:20100000:00000000:00000140:0004e3bd:00000000:00000001'}

[2007-03-08 14:32:27 3775] ERROR (XendNode:105) 1

[2007-03-08 14:32:27 3775] ERROR (XendNode:106) {0: {'fpu': 'yes',
'f00f_bug': 'no', 'cpu family': '6', 'model name': 'Intel(R) Xeon(R)
CPU 5160 @ 3.00GHz', 'vendor_id': 'GenuineIntel', 'flags': 'fpu tsc
msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr
sse sse2 ss ht tm pbe nx lm constant_tsc pni monitor ds_cpl vmx est
tm2 cx16 xtpr lahf_lm', 'bogomips': '5987.46', 'hlt_bug': 'no',
'fpu_exception': 'yes', 'stepping': '6', 'wp': 'yes', 'cpuid level':
'10', 'model': '15', 'coma_bug': 'no', 'fdiv_bug': 'no', 'cpu MHz':
'2992.612', 'cache size': '4096 KB'}, 1: {'fpu': 'yes', 'f00f_bug':
'no', 'cpu family': '6', 'model name': 'Intel(R) Xeon(R) CPU 5160 @
3.00GHz', 'vendor_id': 'GenuineIntel', 'flags': 'fpu tsc msr pae mce
cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss
ht tm pbe nx lm constant_tsc up pni monitor ds_cpl vmx est tm2 cx16
xtpr lahf_lm', 'bogomips': '5987.46', 'hlt_bug': 'no',
'fpu_exception': 'yes', 'stepping': '6', 'wp': 'yes', 'cpuid level':
'10', 'model': '15', 'coma_bug': 'no', 'fdiv_bug': 'no', 'cpu MHz':
'2992.612', 'cache size': '4096 KB'}}

[2007-03-08 14:32:27 3775] DEBUG (XendDomainInfo:123)
XendDomainInfo.recreate({'max_vcpu_id': 1, 'cpu_time': 12708959672L,
'ssidref': 0, 'hvm': 0, 'shutdown_reason': 0, 'dying': 0,
'online_vcpus': 2, 'domid': 0, 'paused': 0, 'crashed': 0, 'running':
1, 'maxmem_kb': -4, 'shutdown': 0, 'mem_kb': 131072, 'handle': [0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'blocked': 0, 'name':
'Domain-0'})

[2007-03-08 14:32:27 3775] DEBUG (XendConfig:324) XendConfig.init:
{'vcpus_params': {}, 'PV_args': '', 'features': '', 'image': {},
'cpus': [], 'paused': 0, 'domid': 0, 'shutdown': 0, 'blocked': 0,
'actions_after_crash': 'restart', 'vbd_refs': [], 'PV_ramdisk': '',
'name_label': 'Domain-0', 'HVM_boot_params': {}, 'cpu_weight': 256,
'console_refs': [], 'online_vcpus': 2, 'cpu_cap': 0, 'on_xend_stop':
'ignore', 'memory_static_min': 0, 'HVM_boot_policy': '',
'shutdown_reason': 0, 'memory_actual': 0, 'memory_static_max': 0,
'actions_after_shutdown': 'destroy', 'on_xend_start': 'ignore',
'crashed': 0, 'memory_dynamic_max': 0, 'vcpus_number': 2,
'actions_after_suspend': '', 'memory_dynamic_min': 128, 'uuid':
'00000000-0000-0000-0000-000000000000', 'PV_kernel': '', 'cpu_time':
12.708959672000001, 'shadow_memory': 0, 'dying': 0,
'PV_bootloader_args': '', 'other_config': {}, 'running': 1,
'actions_after_reboot': 'restart', 'PV_bootloader': '', 'vif_refs':
[], 'vtpm_refs': [], 'security': None, 'devices': {}}

[2007-03-08 14:32:27 3775] INFO (XendDomainInfo:137) Recreating domain
0, UUID 00000000-0000-0000-0000-000000000000. at /local/domain/0

[2007-03-08 14:32:27 3775] WARNING (XendDomainInfo:151)
/local/domain/0/vm is missing. recreate is confused, trying our best
to recover

[2007-03-08 14:32:27 3775] DEBUG (XendDomainInfo:1937) Storing VM
details: {'on_xend_stop': 'ignore', 'shadow_memory': '0', 'uuid':
'00000000-0000-0000-0000-000000000000', 'name': 'Domain-0',
'on_reboot': 'restart', 'on_poweroff': 'destroy', 'on_xend_start':
'ignore', 'on_crash': 'restart', 'xend/restart_count': '0', 'vcpus':
'2', 'vcpu_avail': '3', 'memory': '128', 'maxmem': '128'}

[2007-03-08 14:32:27 3775] DEBUG (XendDomainInfo:814) Storing domain
details: {'name': 'Domain-0', 'console/limit': '1048576',
'memory/target': '131072', 'vm':
'/vm/00000000-0000-0000-0000-000000000000', 'domid': '0',
'cpu/0/availability': 'online', 'cpu/1/availability': 'online',
'control/platform-feature-multiprocessor-suspend': '1'}

[2007-03-08 14:32:27 3775] DEBUG (XendDomain:427) Adding Domain: 0
[2007-03-08 14:32:27 3775] DEBUG (XendDomain:372) number of vcpus to use
is 0
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
VIF_metrics.destroy not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
VIF_metrics.get_all not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call: task.destroy
not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
host_metrics.destroy not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call: VBD.set_VM
not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call: VBD.set_VDI
not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
VBD.set_device not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call: VBD.set_mode
not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call: VBD.set_type
not found
[2007-03-08 14:32:27 3775] DEBUG (XendDomainInfo:881)
XendDomainInfo.handleShutdownWatch
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
VM.get_auto_power_on not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
VM.set_auto_power_on not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
PIF.set_network not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call: PIF.set_host
not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call: PIF.set_VLAN
not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
console.get_other_config not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
console.set_other_config not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
console.destroy not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
VIF.get_network not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
VIF.set_device not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
VIF.set_network not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call: VIF.set_VM
not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call: VIF.set_MAC
not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call: VIF.set_MTU
not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call: host.destroy
not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
session.destroy not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
session.get_all not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
PIF_metrics.destroy not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
VBD_metrics.destroy not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
VBD_metrics.get_all not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call:
host_cpu.destroy not found
[2007-03-08 14:32:27 3775] WARNING (XendAPI:462) API call: debug.get_all
not found


Any help with this would be greatly appreciated as this is causing me
some inconvenience.  I'm happy to help debug it further, but wasn't sure
what other information would be of use.

Kieran


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel