[Xen-users] Issue with booting domU

[Geoffrey Tran <gtran@xxxxxxx>]

Hello,

I have been trying to use Xen 4.5 in an openstack distribution,
using libvirt.  It works fine with booting the cirrosOS sample
images, but does not boot my custom image.  

The custom image is an Ubuntu 12.04 image, built according to
the instructions at:
http://docs.openstack.org/image-guide/content/ubuntu-image.html

This image was build on a separate server that uses KVM as it's
hypervisor.  Right now, the instance fails and I have pulled
the following log from libvirt:

libxl: debug: libxl_create.c:1501:do_domain_create: ao 0x7f1f10001ed0: create: 
how=(nil) callback=(nil) poller=0x7f1f10000c80
libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk vdev=xvda 
spec.backend=qdisk
libxl: debug: libxl_create.c:907:initiate_domain_create: running bootloader
libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk 
vdev=(null) spec.backend=qdisk
libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk vdev=xvdb 
spec.backend=qdisk
libxl: debug: libxl_event.c:629:libxl__ev_xswatch_deregister: watch 
w=0x7f1f100024e0: deregister unregistered
libxl: debug: libxl.c:3088:local_device_attach_cb: locally attaching qdisk 
/dev/xvdb
libxl: debug: libxl_bootloader.c:411:bootloader_disk_attached_cb: Config 
bootloader value: pygrub
libxl: debug: libxl_bootloader.c:427:bootloader_disk_attached_cb: Checking for 
bootloader in libexec path: /usr/lib/xen-4.5/bin/pygrub
libxl: debug: libxl_create.c:1517:do_domain_create: ao 0x7f1f10001ed0: 
inprogress: poller=0x7f1f10000c80, flags=i
libxl: debug: libxl_event.c:577:libxl__ev_xswatch_register: watch 
w=0x7f1f100026a0 wpath=/local/domain/6 token=3/0: register slotnum=3
libxl: debug: libxl_event.c:1945:libxl__ao_progress_report: ao 0x7f1f10001ed0: 
progress report: callback queued aop=0x7f1f10005950
libxl: debug: libxl_bootloader.c:537:bootloader_gotptys: executing bootloader: 
/usr/lib/xen-4.5/bin/pygrub
libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
/usr/lib/xen-4.5/bin/pygrub
libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
--output=/var/run/xen/bootloader.6.out
libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
--output-format=simple0
libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
--output-directory=/var/run/xen/bootloader.6.d
libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
/dev/xvdb
libxl: debug: libxl_event.c:1334:egc_run_callbacks: ao 0x7f1f10001ed0: progress 
report: callback aop=0x7f1f10005950
libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x7f1f100026a0 
wpath=/local/domain/6 token=3/0: event epath=/local/domain/6
libxl: debug: libxl_bootloader.c:636:bootloader_finished: bootloader completed
libxl: debug: libxl_bootloader.c:138:bootloader_result_command: bootloader 
output contained kernel /var/run/xen/bootloader.6.d/boot_kernel.azlwht
libxl: debug: libxl_bootloader.c:138:bootloader_result_command: bootloader 
output contained ramdisk /var/run/xen/bootloader.6.d/boot_ramdisk.Z_gkwV
libxl: debug: libxl_bootloader.c:138:bootloader_result_command: bootloader 
output contained args root=/dev/mapper/ubuntu--vg-root ro   splash quiet 
rtsched=gsn_edf $vt_handoff
libxl: debug: libxl_bootloader.c:649:bootloader_finished: bootloader execution 
successful
libxl: debug: libxl_event.c:615:libxl__ev_xswatch_deregister: watch 
w=0x7f1f100026a0 wpath=/local/domain/6 token=3/0: deregister slotnum=3
libxl: debug: libxl_event.c:577:libxl__ev_xswatch_register: watch 
w=0x7f1f10002458 wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/1: 
register slotnum=3
libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x7f1f10002458 
wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/1: event 
epath=/local/domain/0/backend/qdisk/0/51728/state
libxl: debug: libxl_event.c:834:devstate_watch_callback: backend 
/local/domain/0/backend/qdisk/0/51728/state wanted state 6 still waiting state 5
libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x7f1f10002458 
wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/1: event 
epath=/local/domain/0/backend/qdisk/0/51728/state
libxl: debug: libxl_event.c:834:devstate_watch_callback: backend 
/local/domain/0/backend/qdisk/0/51728/state wanted state 6 still waiting state 5
libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x7f1f10002458 
wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/1: event 
epath=/local/domain/0/backend/qdisk/0/51728/state
libxl: debug: libxl_event.c:834:devstate_watch_callback: backend 
/local/domain/0/backend/qdisk/0/51728/state wanted state 6 still waiting state 5
libxl: debug: libxl_event.c:848:devstate_timeout: backend 
/local/domain/0/backend/qdisk/0/51728/state wanted state 6  timed out
libxl: debug: libxl_event.c:615:libxl__ev_xswatch_deregister: watch 
w=0x7f1f10002458 wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/1: 
deregister slotnum=3
libxl: debug: libxl_event.c:629:libxl__ev_xswatch_deregister: watch 
w=0x7f1f10002458: deregister unregistered
libxl: debug: libxl_event.c:577:libxl__ev_xswatch_register: watch 
w=0x7f1f10002458 wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/2: 
register slotnum=3
libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x7f1f10002458 
wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/2: event 
epath=/local/domain/0/backend/qdisk/0/51728/state
libxl: debug: libxl_event.c:834:devstate_watch_callback: backend 
/local/domain/0/backend/qdisk/0/51728/state wanted state 6 still waiting state 5
libxl: debug: libxl_event.c:848:devstate_timeout: backend 
/local/domain/0/backend/qdisk/0/51728/state wanted state 6  timed out
libxl: debug: libxl_event.c:615:libxl__ev_xswatch_deregister: watch 
w=0x7f1f10002458 wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/2: 
deregister slotnum=3
libxl: debug: libxl_event.c:629:libxl__ev_xswatch_deregister: watch 
w=0x7f1f10002458: deregister unregistered
libxl: error: libxl_device.c:950:device_backend_callback: unable to remove 
device with path /local/domain/0/backend/qdisk/0/51728
libxl: debug: libxl_event.c:629:libxl__ev_xswatch_deregister: watch 
w=0x7f1f100024e0: deregister unregistered
libxl: error: libxl.c:3168:local_device_detach_cb: unable to remove vbd with id 
51728: No such file or directory
libxl: error: libxl_bootloader.c:278:bootloader_local_detached_cb: unable to 
detach locally attached disk
libxl: error: libxl_create.c:1121:domcreate_rebuild_done: cannot (re-)build 
domain: -9
libxl: debug: libxl_event.c:1765:libxl__ao_complete: ao 0x7f1f10001ed0: 
complete, rc=-3
libxl: debug: libxl_event.c:1737:libxl__ao__destroy: ao 0x7f1f10001ed0: destroy
xc: debug: hypercall buffer: total allocations:66 total releases:66
xc: debug: hypercall buffer: current allocations:0 maximum allocations:2
xc: debug: hypercall buffer: cache current size:2
xc: debug: hypercall buffer: cache hits:52 misses:2 toobig:12


It looks like there is a timeout with waiting for "state 6"?  I also pulled
the following log from the bootloader log:
Using <class 'grub.GrubConf.Grub2ConfigFile'> to parse /grub/grub.cfg
WARNING:root:Unknown directive load_video
WARNING:root:Unknown directive terminal_output
WARNING:root:Unknown directive else
WARNING:root:Unknown directive elif
WARNING:root:Unknown directive else
WARNING:root:Unknown directive else
WARNING:root:Unknown directive else
WARNING:root:Unknown directive else
WARNING:root:Unknown directive export
WARNING:root:Unknown image directive recordfail
WARNING:root:Unknown image directive gfxmode
WARNING:root:Unknown image directive recordfail
WARNING:root:Unknown directive source
^[(B^[)0^[[1;24r^[[m^O^[[?7h^[[?1h^[=^[[H^[[J^[[?1h^[=
    pyGRUB  version 0.6^M
 
^[[0m^Nlqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk^O^M
 ^[[0m^Nx^O ^[[0;7m^OUbuntu, with Linux 4.1.3                                   
           ^[[m^O ^[[0m^Nx^O^M
 ^[[0m^Nx^O Ubuntu, with Linux 4.1.3 (recovery mode)^[[31C^[[0m^Nx^O^M
 ^[[0m^Nx^O Memory test (memtest86+)^[[47C^[[0m^Nx^O^M
 ^[[0m^Nx^O Memory test (memtest86+, serial console 115200)^[[24C^[[0m^Nx^O^M
 ^[[0m^Nx^O^[[72C^[[0m^Nx^O^M
 ^[[0m^Nx^O^[[72C^[[0m^Nx^O^M
 ^[[0m^Nx^O^[[72C^[[0m^Nx^O^M
 ^[[0m^Nx^O^[[72C^[[0m^Nx^O^M
 
^[[0m^Nmqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj^O
^[[70D^[[0m^N^OUse the ^[[0m^N^^O and ^[[0m^Nv^O keys to select which entry is 
highlighted.
^[[58DPress enter to boot the selected OS, 'e' to edit the
^[[52Dcommands before booting, 'a' to modify the kernel arguments
^[[59Dbefore booting, or 'c' for a command 
line.^[[9A^[[26C^[[15B^[[68D^[[?1l^[>^[[24;1H^M^[[?1l^[>


I am not sure where to go to debug/get this fixed.  The image kernel does
have the pertinent xen options enabled.

Thanks for any help!
Geoffrey

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users