[Xen-devel] Configuration parameters get "sometimes" omitted

[Christian Ehrhardt <ehrhardt@xxxxxxxxxxxxxxxxxx>]

Hi,
we have the problem that our xen configuration settings are not correctly 
evaluated when creating a xen guests.

When creating domains with "xm create domainname" the system sometimes has all 
configuration parameters out of the config files, but sometimes not.
We did some tests and saw that the parameters are only certainly used if we specify them 
via the commandline like "xm create domainname parameter=value,...".

If the parameters are only specified in the configuration files in /etc/xen/ they are 
parsed and used "sometimes".
Another fact we found is that if the "xm create" is called by a script the error occurs 
very often ~70% of the testcases, while calling the command from a ssh shell manually only fails in 
~10% of the testcases. We stripped down the script and it does nothing but calling "xm 
create" and the normal shell where it works mostly has no .profile/.bashrc/... that change the 
environment.
Additionally there is some persistent behavior - if it failed once it will fail 
until the host is rebooted (maybe some config parts are stored and not 
re-evaluated). I checked xenstore with the script from 
http://wiki.xensource.com/xenwiki/XenStore and saw that xenstore only has the 
parameters if a good-case guest is running.

An interesting fact is that it does not fail initializing memory or pci 
passthrough in host or guest - somehow it fails the initial config parsing. You 
can see below that already the output of XendDomainInfo.create in the log is 
different in good/bad case.

Because of the unstable behavior I expect some kind of race condition while 
parsing from the files.



### See below for more verbose details on what we found in the log's ,system 
setup and configuration files ###



The command "xm create lnudb1 pci=04:01.0 memory=2048" works always successful.
A "xm create lnudb1" creates only *sometimes* a system with a pci adapter and 2 
GB memory.
Looking in the /var/log/xend.log reveals a different XendDomainInfo.create statement 
for good and bad case.
(I added the >>> <<< signs to point to the important statement)

XendDomainInfo.create(['vm', ['name', 'lnudb1'], ['memory', 2048], ['vcpus', 1], 
['on_xend_start', 'ignore'], ['on_xend_stop', 'ignore'], ['image', ['linux', 
['kernel', '/etc/xen/vmlinuz-xen'], ['ramdisk', '/etc/xen/initrd-xen'], ['root', '/dev/xvda1'], 
['args', 'TERM=xterm selinux=off xencons=tty']]], ['device', ['vbd', ['uname', 'phy:sde5'], 
['dev', 'xvda1'], ['mode', 'w']]], ['device', ['vbd', ['uname', 'phy:/dev/hda'], ['dev', 'hdc:cdrom'], 
['mode', 'r']]], 
> > > ['device', ['pci', ['dev', ['domain', '0x0'], ['bus', '0x04'], ['slot', '0x01'], 
> > > ['func', '0x0']]]], <<<
['device', ['vif', ['bridge', 'xenbr0'], ['mac', '00:16:3e:00:00:10']]], 
['device', ['vif', ['bridge', 'xenbr1'],
['mac', '00:16:3e:00:00:31']]], ['device', ['vkbd']], ['device', ['vfb', 
['vncunused', '1'], ['type', 'vnc'],
['display', 'localhost:10.0'], ['xauthority', '/root/.Xauthority']]]])


The xm create with the profile described above produces following statement 
XendDomainInfo.create(['vm', ['name', 'lnudb1'], ['memory', 1024], ['vcpus', 1], 
['on_xend_start', 'ignore'], ['on_xend_stop', 'ignore'], ['image', ['linux', 
['kernel', '/etc/xen/vmlinuz-xen'], ['ramdisk', '/etc/xen/initrd-xen'], ['root', '/dev/xvda1'], 
['args', 'TERM=xterm selinux=off xencons=tty']]], ['device', ['vbd', ['uname', 'phy:sde5'], 
['dev', 'xvda1'], ['mode', 'w']]], ['device', ['vbd', ['uname', 'phy:/dev/hda'], ['dev', 'hdc:cdrom'], 
['mode', 'r']]], 
> > >  <<<<
['device', ['vif', ['bridge', 'xenbr0'], ['mac', '00:16:3e:00:00:10']]], 
['device', ['vif', ['bridge', 'xenbr1'],
['mac', '00:16:3e:00:00:31']]], ['device', ['vkbd']], ['device', ['vfb', 
['vncunused', '1'], ['type', 'vnc'],
['display', 'localhost:10.0'], ['xauthority', '/root/.Xauthority']]]])

Also note the different values for the memory statement

System Description
Hardware IBM x3950, 4 Intel Xeon CPUs (dual core) 3.5GHz, 23Gb memory.
The operating system is Linux SLES10 SP1, with the following rpms:
o xen-kmp-smp-3.1.0_2.6.16.46_0.12-0.1
o xen-libs-32bit-3.0.4_13138-0.40
o xen-doc-pdf-3.1.0-0.1
o kernel-xen-2.6.16.46-0.12
o xen-3.1.0-0.1
o xen-tools-3.1.0-0.1
o xen-doc-html-3.1.0-0.1
o xen-libs-3.1.0-0.1
o xen-tools-ioemu-3.1.0-0.1

the kernel is booted with the parameters 
pciback.hide=(04:01.0)(04:01.1)(06:01.0)(06:01.1)(08:01.0)(08:01.1) 
and /etc/modprobe.conf constains the statements 
install lpfc /sbin/modprobe pciback ; /sbin/modprobe --first-time --ignore-install lpfc
options  pciback hide=(04:01.0)(04:01.1)(06:01.0)(06:01.1)(08:01.0)(08:01.1)
to hide the pci adapters for the Dom0.

The xen profile used is:
builder='linux'
memory = 2048
name = "lnudb1"
vcpus = 1
vif = [ 'mac=00:16:3e:00:00:10, bridge=xenbr0', 'mac=00:16:3e:00:00:31, 
bridge=xenbr1' ]
disk = [ 'phy:sde5,xvda1,w', 'phy:/dev/hda,hdc:cdrom,r' ]
root = "/dev/xvda1"
extra = "TERM=xterm selinux=off xencons=tty"
vfb=[ "type=vnc,vncunused=1" ]
pci= [ '0000:04:01.0' ]

The critical parameters are memory and pci, we never saw the issue with e.g. 
vif's.

P.S. It was hard to decide between xen-devel and xen-users by topic for that 
mail.
Eventually I coose xen-devel becasue I'm already member of this list and 
thereby get also responses that may lack a cc to me. Please redirect if needed.

--

Grüsse / regards, 
Christian Ehrhardt
IBM Linux Technology Center, Open Virtualization
+49 7031/16-3385
Ehrhardt@xxxxxxxxxxxxxxxxxx
Ehrhardt@xxxxxxxxxx

IBM Deutschland Entwicklung GmbH
Vorsitzender des Aufsichtsrats: Johann Weihen 
Geschäftsführung: Herbert Kircher 
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel