Re: [Xen-devel] Question about unable to add disk device after VM is destroyed and recreated on NVIDIA Jetson board

[Meng Xu <xumengpanda@xxxxxxxxx>]

On Mon, Jun 20, 2016 at 11:43 AM, Meng Xu <xumengpanda@xxxxxxxxx> wrote:
>
> Hi all,
>
> I'm running Xen on NVIDIA Jetson TK1 board. The Xen code is from Ian's
> repo.: git://xenbits.xen.org/people/ianc/xen.git with the commit point
> c78d51660446d33dac4bb07c3c17e1d14d62ebc2
>
> Right now, I can boot dom0 on Xen on the Jetson board. After the
> system boots up, I can boot up a VM1 using the vm1.config (attached
> below). However, after I use "xl destroy vm1" to destroy the vm1 and
> try to boot up vm1 again with the exact same configuration file, it
> starts to reports the following error:
>
> --- Start of the output of the  "xl create vm1.xl"  ---
> libxl: error: libxl_device.c:952:device_backend_callback: unable to
> add device with path /local/domain/0/backend/vbd/5/51712
>
> libxl: error: libxl_create.c:1161:domcreate_launch_dm: unable to add
> disk devices
>
> libxl: error: libxl_device.c:952:device_backend_callback: unable to
> remove device with path /local/domain/0/backend/vbd/5/51712
>
> libxl: error: libxl.c:1650:devices_destroy_cb: libxl__devices_destroy
> failed for 5
>
> ---End of of the output of the  "xl create vm1.xl"  ---
>
> I found that this issue was raised before in 2013 at
> http://lists.xen.org/archives/html/xen-devel/2013-02/msg00704.html
> However, I didn't see the solution on that thread.
>
> I'm wondering if someone has encountered this issue before and know
> how to fix it?
>
> Thank you very much for your help and time! Any advice is really appreciated!
>
> I attached the configurations and more log messages as below:
>
> ---The vm1's configuration file vm1.config---
>
> kernel = "/boot/zImage-domU" # zImage is kernel domU will uses. zImage
> is inside dom0 and it’s dom0’s path.
>
> memory = 512
>
> name = "vm1"
>
> vcpus = 1
>
> disk = [ 'file:/home/ubuntu/sdcard/vm1.disk,xvda,w' ]
>
> vif = ['bridge=xenbr0']
>
> extra = 'console=hvc0 xencons=tty root=/dev/xvda'
>
>
> --- The log message when I create vm1---
>
> # xl -vvv create -c vm1.xl
>
> Parsing config from vm1.xl
>
> libxl: debug: libxl_create.c:1512:do_domain_create: ao 0x3cdd8:
> create: how=(nil) callback=(nil) poller=0x3ce20
>
> libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk
> vdev=xvda spec.backend=unknown
>
> libxl: debug: libxl_device.c:298:libxl__device_disk_set_backend: Disk
> vdev=xvda, using backend phy
>
> libxl: debug: libxl_create.c:915:initiate_domain_create: running bootloader
>
> libxl: debug: libxl_bootloader.c:329:libxl__bootloader_run: no
> bootloader configured, using user supplied kernel
>
> libxl: debug: libxl_event.c:629:libxl__ev_xswatch_deregister: watch
> w=0x3d2dc: deregister unregistered
>
> domainbuilder: detail: xc_dom_allocate: cmdline="console=hvc0
> xencons=tty root=/dev/xvda", features="(null)"
>
> libxl: debug: libxl_dom.c:536:libxl__build_pv: pv kernel mapped 0 path
> /boot/zImage-domU
>
> domainbuilder: detail: xc_dom_kernel_file: filename="/boot/zImage-domU"
>
> domainbuilder: detail: xc_dom_malloc_filemap    : 5091 kB
>
> domainbuilder: detail: xc_dom_boot_xen_init: ver 4.6, caps xen-3.0-armv7l
>
> domainbuilder: detail: xc_dom_rambase_init: RAM starts at 40000
>
> domainbuilder: detail: xc_dom_parse_image: called
>
> domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ...
>
> domainbuilder: detail: loader probe failed
>
> domainbuilder: detail: xc_dom_find_loader: trying Linux zImage (ARM64)
> loader ...
>
> domainbuilder: detail: xc_dom_probe_zimage64_kernel: kernel is not an
> arm64 Image
>
> domainbuilder: detail: loader probe failed
>
> domainbuilder: detail: xc_dom_find_loader: trying Linux zImage (ARM32)
> loader ...
>
> domainbuilder: detail: loader probe OK
>
> domainbuilder: detail: xc_dom_parse_zimage32_kernel: called
>
> domainbuilder: detail: xc_dom_parse_zimage32_kernel: xen-3.0-armv7l:
> 0x40008000 -> 0x40500c20
>
> libxl: debug: libxl_arm.c:537:libxl__arch_domain_init_hw_description:
> configure the domain
>
> libxl: debug: libxl_arm.c:545:libxl__arch_domain_init_hw_description:
> constructing DTB for Xen version 4.6 guest
>
> libxl: debug: libxl_arm.c:546:libxl__arch_domain_init_hw_description:
>  - vGIC version: V2
>
> libxl: debug: libxl_arm.c:303:make_memory_nodes: Creating placeholder
> node /memory@40000000
>
> libxl: debug: libxl_arm.c:303:make_memory_nodes: Creating placeholder
> node /memory@200000000
>
> libxl: debug: libxl_arm.c:620:libxl__arch_domain_init_hw_description:
> fdt total size 1266
>
> domainbuilder: detail: xc_dom_devicetree_mem: called
>
> domainbuilder: detail: xc_dom_mem_init: mem 512 MB, pages 0x20000 pages, 4k 
> each
>
> domainbuilder: detail: xc_dom_mem_init: 0x20000 pages
>
> domainbuilder: detail: xc_dom_boot_mem_init: called
>
> domainbuilder: detail: set_mode: guest xen-3.0-armv7l, address size 32
>
> domainbuilder: detail: xc_dom_malloc            : 1024 kB
>
> domainbuilder: detail: populate_guest_memory: populating RAM @
> 0000000040000000-0000000060000000 (512MB)
>
> domainbuilder: detail: populate_one_size: populated 0x100/0x100
> entries with shift 9
>
> domainbuilder: detail: arch_setup_meminit: placing boot modules at 0x48000000
>
> domainbuilder: detail: arch_setup_meminit: devicetree: 0x48000000 -> 
> 0x48001000
>
> libxl: debug: libxl_arm.c:651:finalise_one_memory_node: Populating
> placeholder node /memory@40000000
>
> libxl: debug: libxl_arm.c:645:finalise_one_memory_node: Nopping out
> placeholder node /memory@200000000
>
> domainbuilder: detail: xc_dom_build_image: called
>
> domainbuilder: detail: xc_dom_alloc_segment:   kernel       :
> 0x40008000 -> 0x40501000  (pfn 0x40008 + 0x4f9 pages)
>
> domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn
> 0x40008+0x4f9 at 0xb614c000
>
> domainbuilder: detail: xc_dom_load_zimage_kernel: called
>
> domainbuilder: detail: xc_dom_load_zimage_kernel: kernel seg
> 0x40008000-0x40501000
>
> domainbuilder: detail: xc_dom_load_zimage_kernel: copy 5213216 bytes
> from blob 0xb6746000 to dst 0xb614c000
>
> domainbuilder: detail: xc_dom_alloc_segment:   devicetree   :
> 0x48000000 -> 0x48001000  (pfn 0x48000 + 0x1 pages)
>
> domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn
> 0x48000+0x1 at 0xb6edc000
>
> domainbuilder: detail: alloc_magic_pages: called
>
> domainbuilder: detail: count_pgtables_arm: called
>
> domainbuilder: detail: xc_dom_build_image  : virt_alloc_end : 0x48001000
>
> domainbuilder: detail: xc_dom_build_image  : virt_pgtab_end : 0x0
>
> domainbuilder: detail: xc_dom_boot_image: called
>
> domainbuilder: detail: arch_setup_bootearly: doing nothing
>
> domainbuilder: detail: xc_dom_compat_check: supported guest type:
> xen-3.0-armv7l <= matches
>
> domainbuilder: detail: setup_pgtables_arm: called
>
> domainbuilder: detail: clear_page: pfn 0x39000, mfn 0x39000
>
> domainbuilder: detail: clear_page: pfn 0x39001, mfn 0x39001
>
> domainbuilder: detail: start_info_arm: called
>
> domainbuilder: detail: domain builder memory footprint
>
> domainbuilder: detail:    allocated
>
> domainbuilder: detail:       malloc             : 1055 kB
>
> domainbuilder: detail:       anon mmap          : 0 bytes
>
> domainbuilder: detail:    mapped
>
> domainbuilder: detail:       file mmap          : 5091 kB
>
> domainbuilder: detail:       domU mmap          : 5096 kB
>
> domainbuilder: detail: vcpu_arm32: called
>
> domainbuilder: detail: Initial state CPSR 0x1d3 PC 0x40008000
>
> domainbuilder: detail: launch_vm: called, ctxt=0xb6ef7004
>
> domainbuilder: detail: xc_dom_gnttab_hvm_seed: called, pfn=0x38000
>
> domainbuilder: detail: xc_dom_release: called
>
> libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk
> vdev=xvda spec.backend=phy
>
> libxl: debug: libxl_event.c:577:libxl__ev_xswatch_register: watch
> w=0x3e230 wpath=/local/domain/0/backend/vbd/4/51712/state token=3/0:
> register slotnum=3
>
> libxl: debug: libxl_create.c:1528:do_domain_create: ao 0x3cdd8:
> inprogress: poller=0x3ce20, flags=i
>
> libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x3e230
> wpath=/local/domain/0/backend/vbd/4/51712/state token=3/0: event
> epath=/local/domain/0/backend/vbd/4/51712/state
>
> libxl: debug: libxl_event.c:834:devstate_watch_callback: backend
> /local/domain/0/backend/vbd/4/51712/state wanted state 2 still waiting
> state 1
>
> libxl: debug: libxl_event.c:848:devstate_timeout: backend
> /local/domain/0/backend/vbd/4/51712/state wanted state 2  timed out


OK. Never mind. It's due to my fault. :-(
I thought the kernel I used has built-in block device backend driver,
but actually it did not. After recompiling the kernel and use the
block backend driver in dom0, it works now.

Meng

-- 
-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel