Re: [Xen-users] 2.6.18.8 xen.org dom0 kernel w/RHEL 5 PV on HVM guests

[Andrew Evans <andrewe@xxxxxxxxxxxxx>]

On 03/22/10 11:22, Pasi Kärkkäinen wrote:
> On Mon, Mar 22, 2010 at 11:13:43AM -0700, Andrew Evans wrote:
>   
>> On 03/18/10 15:32, Pasi Kärkkäinen wrote:
>>     
>>> Hmm.. can you try with the RHEL5 default hypervisor (3.1.2) ?
>>> (and the 2.6.18-xen dom0 kernel) 
>>>
>>> Would be good to know if it makes any difference.
>>>   
>>>       
>> I installed the xen-3.0.3 RPM (xm dmesg/xm info says 3.1.2 -- what the
>> heck is up with Red Hat's version numbering?)
>>
>>     
> Xen hypervisor in RHEL5 comes in the kernel rpm together with the dom0 kernel.
>
> RHEL5 currently has Xen 3.1.2 (+patches) hypervisor, but the userland
> tools (xen rpm) are based on 3.0.3. 
>
> That is because RHEL5 is an enterprise distro, and they need to keep
> the ABI stable.. RHEL 5.0 first shipped with Xen 3.0.3 so they cannot
> change versions and break things just like that.
>   

Ah, that makes sense. I realize RHEL makes ABI backward compatibility
guarantees, but I hadn't guessed that was the issue here.

>> but am getting the dreaded
>> "Hotplug scripts not working" errors. Looks like a TUN/TAP problem:
>>
>> $ cat qemu-dm.4910.log
>> domid: 2
>> qemu: the number of cpus is 7
>> Change xvda to look like hda
>> Change xvdb to look like hdb
>> Change xvdc to look like hdc
>> Watching /local/domain/2/logdirty/next-active
>> Watching /local/domain/0/device-model/2/command
>> /etc/xen/qemu-ifup: could not launch network script
>> Could not initialize device 'tap'
>>
>> $ cat xend-debug.log.1
>> Xend started at Thu Mar 18 23:25:38 2010.
>> Nothing to flush.
>> Nothing to flush.
>> can't add tap0 to bridge eth0: Operation not supported
>> can't add tap0 to bridge eth0: Operation not supported
>>
>> $ cat /etc/xen/qemu-ifup
>> #!/bin/sh
>>
>> #. /etc/rc.d/init.d/functions
>> #ulimit -c unlimited
>>
>> echo 'config qemu network with xen bridge for ' $*
>>
>> ifconfig $1 0.0.0.0 up
>> brctl addif $2 $1
>>
>> strace of `brctl addif eth0 tap0` shows:
>>
>> ioctl(4, 0x89a2, 0x7fff724c1e50)        = -1 EOPNOTSUPP (Operation not
>> supported)
>> ioctl(4, SIOCDEVPRIVATE, 0x7fff724c1e50) = -1 EOPNOTSUPP (Operation not
>> supported)
>> write(2, "can't add tap0 to bridge eth0: O"..., 55can't add tap0 to
>> bridge eth0: Operation not supported
>> ) = 55
>> exit_group(1)                           = ?
>>
>> I think the RHEL 5 2.6.18 dom0 kernel must support this ioctl, while the
>> xen.org 2.6.18.8 dom0 kernel must not. So RHEL 5's Xen 3.1.2 package on
>> xen.org 2.6.18.8 dom0 kernel looks like a dead end.
>>     
> Do you have a bridge called 'eth0' for real? Try "brctl show" to verify.
> RHEL5 as a default creates 'xenbr0', not 'eth0'.
>   

Well spotted! I don't have such a bridge, and I can't create one because
eth0 is already an existing network interface name. I've changed my
libvirt domU config to use xenbr0 instead of eth0. Not sure why this
works with Xen 3.4.2 though.

My domU *still* isn't starting, though. xend-debug.log shows this traceback:

Traceback (most recent call last):
  File "/usr/lib64/python2.4/SocketServer.py", line 463, in
process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib64/python2.4/SocketServer.py", line 254, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib64/python2.4/SocketServer.py", line 521, in __init__
    self.handle()
  File "/usr/lib64/python2.4/BaseHTTPServer.py", line 316, in handle
    self.handle_one_request()
  File "/usr/lib64/python2.4/BaseHTTPServer.py", line 310, in
handle_one_request
    method()
  File "/usr/lib64/python2.4/site-packages/xen/util/xmlrpclib2.py", line
66, in do_POST
    self.send_response(200)
  File "/usr/lib64/python2.4/BaseHTTPServer.py", line 367, in send_response
    self.wfile.write("%s %d %s\r\n" %
  File "/usr/lib64/python2.4/socket.py", line 256, in write
    self.flush()
  File "/usr/lib64/python2.4/socket.py", line 243, in flush
    self._sock.sendall(buffer)
error: (32, 'Broken pipe')

No idea if it's relevant or not. Certainly doesn't look good though.

xend.log contains everyone's favorite, "Hotplug scripts not working":

[2010-03-22 19:28:38 xend 4863] ERROR (SrvBase:88) Request
wait_for_devices failed.
Traceback (most recent call last):
  File "/usr/lib64/python2.4/site-packages/xen/web/SrvBase.py", line 85,
in perform
    return op_method(op, req)
  File
"/usr/lib64/python2.4/site-packages/xen/xend/server/SrvDomain.py", line
72, in op_wait_for_devices
    return self.dom.waitForDevices()
  File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py",
line 2330, in waitForDevices
    self.waitForDevices_(c)
  File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py",
line 1422, in waitForDevices_
    return self.getDeviceController(deviceClass).waitForDevices()
  File
"/usr/lib64/python2.4/site-packages/xen/xend/server/DevController.py",
line 160, in waitForDevices
    return map(self.waitForDevice, self.deviceIDs())
  File
"/usr/lib64/python2.4/site-packages/xen/xend/server/DevController.py",
line 170, in waitForDevice
    raise VmError("Device %s (%s) could not be connected. "
VmError: Device 0 (vif) could not be connected. Hotplug scripts not working.

Device model log:

$ cat qemu-dm.16544.log
domid: 4
qemu: the number of cpus is 7
Change xvda to look like hda
Change xvdb to look like hdb
Change xvdc to look like hdc
Watching /local/domain/4/logdirty/next-active
Watching /local/domain/0/device-model/4/command
xs_read(): vncpasswd get error.
/vm/51711672-9512-4bd8-91ea-a1d542bbda92/vncpasswd.
char device redirected to /dev/pts/1
qemu_map_cache_init nr_buckets = 10000
shared page at pfn effff
buffered io page at pfn efffd
xs_read(/vm/51711672-9512-4bd8-91ea-a1d542bbda92/rtc/timeoffset): read error
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
Triggered log-dirty buffer switch
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0

Current domU config (libvirt XML):

<domain type='xen'>
  <name>foo</name>
  <memory>7340032</memory>
  <currentMemory>7340032</currentMemory>
  <vcpu>7</vcpu>
  <os>
    <type arch='x86_64' machine='xenfv'>hvm</type>
    <loader>/usr/lib/xen/boot/hvmloader</loader>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/lib64/xen/bin/qemu-dm</emulator>
    <disk type='block' device='disk'>
      <driver name='phy'/>
      <source dev='/dev/sys/lv0'/>
      <target dev='xvda' bus='xen'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='phy'/>
      <source dev='/dev/sys/lv1'/>
      <target dev='xvdb' bus='xen'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='phy'/>
      <source dev='/dev/sys/lv2'/>
      <target dev='xvdc' bus='xen'/>
    </disk>
    <interface type='bridge'>
      <mac address='00:50:56:0f:dd:02'/>
      <source bridge='xenbr0'/>
      <script path='vif-bridge'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
  </devices>
</domain>

thanks,

-Andrew


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users