All,
I am attempting to get a MirageOS Unikernel working under Xen on an ODROID-XU3 (ARMv7) board. I am able to start a Ubuntu DomU successfully following the instructions from HardKernel (https://wiki.odroid.com/odroid-xu4/application_note/software/xen_virtualization)
but I have not been able to start any MirageOS Unikernel under Xen yet.
The noop tutorial from mirage-skeleton works fine when built for UNIX on the ARM board but the boot process appears to hang under Xen (see the log below). I have to use Ctrl + ] to detach from the frozen console. The noop domU then appears
in the results of “xl list” and I cannot shut it down with “xl shutdown noop” I must kill it with “xl destroy noop”.
I’ve also tried building mirage-www and it appears to have the same problem under Xen.
Xen is version 4.6 (https://github.com/bkrepo/xen -b odroid-xu4)
OCaml is version 4.04.2 (mirage-xen-ocaml requires OCaml < 4.05
https://github.com/ocaml/opam-repository/blob/master/packages/mirage-xen-ocaml/mirage-xen-ocaml.3.0.4/opam)
mirage is version 3.0.5
base is pinned to v0.9.3 (newer versions of base require OCaml > 4.04., older versions have mpopcnt error
https://github.com/janestreet/base/issues/15)
uri is pinned to 1.9.2 (Work around the assembler error described here:
https://github.com/mirage/ocaml-uri/issues/106)
Here is the output of running xl -vv create -c noop.xl:
Parsing config from noop.xl
libxl: debug: libxl_create.c:1557:do_domain_create: ao 0x3c1c8: create: how=(nil) callback=(nil) poller=0x35b50
libxl: debug: libxl_arm.c:59:libxl__arch_domain_prepare_config: Configure the domain
libxl: debug: libxl_arm.c:62:libxl__arch_domain_prepare_config: - Allocate 0 SPIs
libxl: debug: libxl_create.c:945:initiate_domain_create: running bootloader
libxl: debug: libxl_bootloader.c:330:libxl__bootloader_run: no bootloader configured, using user supplied kernel
libxl: debug: libxl_event.c:691:libxl__ev_xswatch_deregister: watch w=0x363a0: deregister unregistered
domainbuilder: detail: xc_dom_allocate: cmdline="(null)", features="(null)"
libxl: debug: libxl_dom.c:624:libxl__build_pv: pv kernel mapped 0 path /root/mirage-skeleton/tutorial/noop/./noop.xen
domainbuilder: detail: xc_dom_kernel_file: filename="/root/mirage-skeleton/tutorial/noop/./noop.xen"
domainbuilder: detail: xc_dom_malloc_filemap : 1470 kB
domainbuilder: detail: xc_dom_boot_xen_init: ver 4.6, caps xen-3.0-armv7l
domainbuilder: detail: xc_dom_rambase_init: RAM starts at 40000
domainbuilder: detail: xc_dom_parse_image: called
domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ...
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying Linux zImage (ARM64) loader ...
domainbuilder: detail: xc_dom_probe_zimage64_kernel: kernel is not an arm64 Image
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying Linux zImage (ARM32) loader ...
domainbuilder: detail: loader probe OK
domainbuilder: detail: xc_dom_parse_zimage32_kernel: called
domainbuilder: detail: xc_dom_parse_zimage32_kernel: xen-3.0-armv7l: 0x40008000 -> 0x40177ac0
libxl: debug: libxl_arm.c:776:libxl__arch_domain_init_hw_description: constructing DTB for Xen version 4.6 guest
libxl: debug: libxl_arm.c:777:libxl__arch_domain_init_hw_description: - vGIC version: V2
libxl: debug: libxl_arm.c:380:make_memory_nodes: Creating placeholder node /memory@40000000
libxl: debug: libxl_arm.c:380:make_memory_nodes: Creating placeholder node /memory@200000000
libxl: debug: libxl_arm.c:871:libxl__arch_domain_init_hw_description: fdt total size 1237
domainbuilder: detail: xc_dom_devicetree_mem: called
domainbuilder: detail: xc_dom_mem_init: mem 256 MB, pages 0x10000 pages, 4k each
domainbuilder: detail: xc_dom_mem_init: 0x10000 pages
domainbuilder: detail: xc_dom_boot_mem_init: called
domainbuilder: detail: set_mode: guest xen-3.0-armv7l, address size 32
domainbuilder: detail: xc_dom_malloc : 512 kB
domainbuilder: detail: populate_guest_memory: populating RAM @ 0000000040000000-0000000050000000 (256MB)
domainbuilder: detail: populate_one_size: populated 0x80/0x80 entries with shift 9
domainbuilder: detail: arch_setup_meminit: placing boot modules at 0x48000000
domainbuilder: detail: arch_setup_meminit: devicetree: 0x48000000 -> 0x48001000
libxl: debug: libxl_arm.c:902:finalise_one_memory_node: Populating placeholder node /memory@40000000
libxl: debug: libxl_arm.c:896:finalise_one_memory_node: Nopping out placeholder node /memory@200000000
domainbuilder: detail: xc_dom_build_image: called
domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x40008000 -> 0x40178000 (pfn 0x40008 + 0x170 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x40008+0x170 at 0xb6991000
domainbuilder: detail: xc_dom_load_zimage_kernel: called
domainbuilder: detail: xc_dom_load_zimage_kernel: kernel seg 0x40008000-0x40178000
domainbuilder: detail: xc_dom_load_zimage_kernel: copy 1505984 bytes from blob 0xb6b82000 to dst 0xb6991000
domainbuilder: detail: xc_dom_alloc_segment: devicetree : 0x48000000 -> 0x48001000 (pfn 0x48000 + 0x1 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x48000+0x1 at 0xb6f4d000
domainbuilder: detail: alloc_magic_pages: called
domainbuilder: detail: count_pgtables_arm: called
domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0x48001000
domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0x0
domainbuilder: detail: xc_dom_boot_image: called
domainbuilder: detail: arch_setup_bootearly: doing nothing
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-armv7l <= matches
domainbuilder: detail: setup_pgtables_arm: called
domainbuilder: detail: clear_page: pfn 0x39000, mfn 0x39000
domainbuilder: detail: clear_page: pfn 0x39001, mfn 0x39001
domainbuilder: detail: start_info_arm: called
domainbuilder: detail: domain builder memory footprint
domainbuilder: detail: allocated
domainbuilder: detail: malloc : 522 kB
domainbuilder: detail: anon mmap : 0 bytes
domainbuilder: detail: mapped
domainbuilder: detail: file mmap : 1470 kB
domainbuilder: detail: domU mmap : 1476 kB
domainbuilder: detail: vcpu_arm32: called
domainbuilder: detail: Initial state CPSR 0x1d3 PC 0x40008000
domainbuilder: detail: launch_vm: called, ctxt=0xb6f4f004
domainbuilder: detail: xc_dom_gnttab_hvm_seed: called, pfn=0x38000
domainbuilder: detail: xc_dom_release: called
libxl: debug: libxl_event.c:2193:libxl__ao_progress_report: ao 0x3c1c8: progress report: callback queued aop=0x3adb8
libxl: debug: libxl_event.c:1874:libxl__ao_complete: ao 0x3c1c8: complete, rc=0
libxl: debug: libxl_event.c:1410:egc_run_callbacks: ao 0x3c1c8: progress report: callback aop=0x3adb8
libxl: debug: libxl_create.c:1580:do_domain_create: ao 0x3c1c8: inprogress: poller=0x35b50, flags=ic
libxl: debug: libxl_event.c:1843:libxl__ao__destroy: ao 0x3c1c8: destroy
xc: debug: hypercall buffer: total allocations:95 total releases:95
xc: debug: hypercall buffer: current allocations:0 maximum allocations:3
xc: debug: hypercall buffer: cache current size:3
xc: debug: hypercall buffer: cache hits:84 misses:3 toobig:8
Thanks in advance!
Regards,
David
