[Xen-devel] libxl: locally attaching disk

[Thanos Makatos <thanos.makatos@xxxxxxxxxx>]

I want to implement booting from VHD images using pygrub for blktap3. I'm 
trying to use the existing code for qemu: in 
libxl__device_disk_local_initiate_attach I tell libxl to do exactly the same 
thing it does for the LIBXL_DISK_BACKEND_QDISK case in the switch statement 
(similar for libxl__device_disk_local_initiate_detach).

I run into the following problem: blkfront goes directly from state 1 to state 
3, the back-end follows by jumping to state 4, and finally the front-end goes 
to state 4 and everything works fine (this is what is done for domU guests 
using blktap3 without pygrub). However, it seems that libxl expects the backend 
to step through each state (specifically it times out waiting for the back-end 
to go to state 2 but the back-end has already gone to state 4). If I correctly 
understand the protocol specification in xen/include/public/io/blkif.h, libxl 
shouldn't be doing that. Here's the output of libxl:

libxl_create.c:1244:do_domain_create: ao 0xe79160: create: how=(nil) 
callback=(nil) poller=0xe77090
libxl_device.c:246:libxl__device_disk_set_backend: Disk vdev=xvda 
spec.backend=tap3
libxl_create.c:691:initiate_domain_create: running bootloader
libxl_device.c:246:libxl__device_disk_set_backend: Disk vdev=(null) 
spec.backend=tap3
libxl_device.c:246:libxl__device_disk_set_backend: Disk vdev=xvda 
spec.backend=tap3
libxl_event.c:559:libxl__ev_xswatch_register: watch w=0xe78130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: register slotnum=3
libxl_create.c:1257:do_domain_create: ao 0xe79160: inprogress: poller=0xe77090, 
flags=i
libxl_event.c:503:watchfd_callback: watch w=0xe78130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: event 
epath=/local/domain/0/backend/vbd3/0/51712/state
libxl_event.c:647:devstate_watch_callback: backend 
/local/domain/0/backend/vbd3/0/51712/state wanted state 2 still waiting state 1
libxl_event.c:503:watchfd_callback: watch w=0xe78130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: event 
epath=/local/domain/0/backend/vbd3/0/51712/state
libxl_event.c:647:devstate_watch_callback: backend 
/local/domain/0/backend/vbd3/0/51712/state wanted state 2 still waiting state 4
libxl_event.c:503:watchfd_callback: watch w=0xe78130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: event 
epath=/local/domain/0/backend/vbd3/0/51712/state
libxl_event.c:647:devstate_watch_callback: backend 
/local/domain/0/backend/vbd3/0/51712/state wanted state 2 still waiting state 4
libxl_event.c:503:watchfd_callback: watch w=0xe78130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: event 
epath=/local/domain/0/backend/vbd3/0/51712/state
libxl_event.c:647:devstate_watch_callback: backend 
/local/domain/0/backend/vbd3/0/51712/state wanted state 2 still waiting state 4
libxl_event.c:661:devstate_timeout: backend 
/local/domain/0/backend/vbd3/0/51712/state wanted state 2  timed out
libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0xe78130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: deregister slotnum=3
libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0xe78130: deregister 
unregistered
libxl_device.c:882:device_backend_callback: unable to add device with path 
/local/domain/0/backend/vbd3/0/51712
libxl.c:2687:local_device_attach_cb: unable to add vbd with id 51712: Resource 
temporarily unavailable
libxl_bootloader.c:405:bootloader_disk_attached_cb: failed to attach local disk 
for bootloader execution

If I modify the back-end to go to the same state as the front-end one but 
making sure that this is done by writing all the intermediate state transitions 
in XenStore, libxl again complains because it cannot see the back-end state 
passing from state 2. Here's the output of libxl:

libxl_create.c:1244:do_domain_create: ao 0x1394160: create: how=(nil) 
callback=(nil) poller=0x1392090
libxl_device.c:246:libxl__device_disk_set_backend: Disk vdev=xvda 
spec.backend=tap3
libxl_create.c:691:initiate_domain_create: running bootloader
libxl_device.c:246:libxl__device_disk_set_backend: Disk vdev=(null) 
spec.backend=tap3
libxl_device.c:246:libxl__device_disk_set_backend: Disk vdev=xvda 
spec.backend=tap3
libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x1393130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: register slotnum=3
libxl_create.c:1257:do_domain_create: ao 0x1394160: inprogress: 
poller=0x1392090, flags=i
libxl_event.c:503:watchfd_callback: watch w=0x1393130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: event 
epath=/local/domain/0/backend/vbd3/0/51712/state
libxl_event.c:647:devstate_watch_callback: backend 
/local/domain/0/backend/vbd3/0/51712/state wanted state 2 still waiting state 1
libxl_event.c:503:watchfd_callback: watch w=0x1393130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: event 
epath=/local/domain/0/backend/vbd3/0/51712/state
libxl_event.c:647:devstate_watch_callback: backend 
/local/domain/0/backend/vbd3/0/51712/state wanted state 2 still waiting state 3
libxl_event.c:503:watchfd_callback: watch w=0x1393130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: event 
epath=/local/domain/0/backend/vbd3/0/51712/state
libxl_event.c:647:devstate_watch_callback: backend 
/local/domain/0/backend/vbd3/0/51712/state wanted state 2 still waiting state 3
libxl_event.c:661:devstate_timeout: backend 
/local/domain/0/backend/vbd3/0/51712/state wanted state 2  timed out
libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x1393130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: deregister slotnum=3
libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x1393130: deregister 
unregistered
libxl_device.c:882:device_backend_callback: unable to add device with path 
/local/domain/0/backend/vbd3/0/51712
libxl.c:2687:local_device_attach_cb: unable to add vbd with id 51712: Resource 
temporarily unavailable
libxl_bootloader.c:405:bootloader_disk_attached_cb: failed to attach local disk 
for bootloader execution

If I modify the back-end to only step into the next state from its current one 
in trying to reach the front-end's state (e.g. if the back-end state is 1 and 
the front-end state is 3, the back-end will go to state 2), the back-end gets 
stuck in state 2, which does make libxl happy but the Xenbus protocol isn't 
advancing because there isn't any other event in XenStore such that the 
back-end can continue increasing its state and utterly match the front-end's 
one. Here's the output of libxl:

libxl_create.c:1244:do_domain_create: ao 0x194e160: create: how=(nil) 
callback=(nil) poller=0x194c090
libxl_device.c:246:libxl__device_disk_set_backend: Disk vdev=xvda 
spec.backend=tap3
libxl_create.c:691:initiate_domain_create: running bootloader
libxl_device.c:246:libxl__device_disk_set_backend: Disk vdev=(null) 
spec.backend=tap3
libxl_device.c:246:libxl__device_disk_set_backend: Disk vdev=xvda 
spec.backend=tap3
libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x194d130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: register slotnum=3
libxl_create.c:1257:do_domain_create: ao 0x194e160: inprogress: 
poller=0x194c090, flags=i
libxl_event.c:503:watchfd_callback: watch w=0x194d130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: event 
epath=/local/domain/0/backend/vbd3/0/51712/state
libxl_event.c:647:devstate_watch_callback: backend 
/local/domain/0/backend/vbd3/0/51712/state wanted state 2 still waiting state 1
libxl_event.c:503:watchfd_callback: watch w=0x194d130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: event 
epath=/local/domain/0/backend/vbd3/0/51712/state
libxl_event.c:643:devstate_watch_callback: backend 
/local/domain/0/backend/vbd3/0/51712/state wanted state 2 ok
libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x194d130 
wpath=/local/domain/0/backend/vbd3/0/51712/state token=3/0: deregister slotnum=3
libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x194d130: deregister 
unregistered
libxl.c:2692:local_device_attach_cb: locally attaching qdisk /dev/xvda
libxl_device.c:1108:libxl__wait_for_backend: Backend 
/local/domain/0/backend/vbd3/0/51712 not ready
libxl_bootloader.c:405:bootloader_disk_attached_cb: failed to attach local disk 
for bootloader execution

Any help appreciated.

--
Thanos Makatos


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel