Revisit: HVM on storage driver domain

["G.R." <firemeteor@xxxxxxxxxxxxxxxxxxxxx>]

Revisiting a failed attempt 5 years ago but still don't have the luck :-(

>> BTW, an irrelevant question:
>> What's the current status of HVM domU on top of storage driver domain?
>> About 7 years ago, one user on the list was able to get this setup up
>> and running with your help (patch).[1]
>> When I attempted to reproduce a similar setup two years later, I
>> discovered that the patch was not submitted.
>> And even with that patch the setup cannot be reproduced successfully.
>> We spent some time debugging on the problem together[2], but didn't
>> bottom out the root cause at that time.
>> In case it's still broken and you still have the interest and time, I
>> can launch a separate thread on this topic and provide required
>> testing environment.
>
>  Yes, better as a new thread please.
>
> FWIW, I haven't looked at this since a long time, but I recall some
> fixes in order to be able to use driver domains with HVM guests, which
> require attaching the disk to dom0 in order for the device model
> (QEMU) to access it.
>
> I would give it a try without using stubdomains and see what you get.
> You will need to run `xl devd` inside of the driver domain, so you
> will need to install xen-tools on the domU. There's an init script to
> launch `xl devd` at boot, it's called 'xendriverdomain'.

>

For this testing purpose, I did the following:

1. downloaded an official FreeBSD 12.2 VM image as a domU

2. Install the xen-tools package and enable the devd daemon

3. reboot the domU with driver_domain=1, mount the guest image from NAS

4. instance a new domU with the following config:

     device_model_stubdomain_override=0
     disk = ['backend=freebsd12,/mnt/vmfs/Windows/ruibox/ruibox.img,raw,xvda,w']

5. When boot the new domU, see the following warning and the firmware cannot boot the disk:

       libxl: warning: libxl_dm.c:1888:libxl__build_device_model_args_new: Domain 17:No way to get local access disk to image: xvda
       Disk will be available via PV drivers but not as anemulated disk.

6. When boot with stubdomain_override=1, it fails differently:

    libxl: error: libxl_dm.c:2332:libxl__spawn_stub_dm: Domain 79:could not access stubdomain kernel /usr/lib/xen-4.14/boot/ioemu-stubdom.gz: No such file or directory
    libxl: error: libxl_dm.c:2724:stubdom_pvqemu_cb: Domain 79:error connecting nics devices: No such file or directory

Looks like the XEN 4.14 package from Debian does not have the stubdom feature available.

I wonder if it's just a single file that I can grab from somewhere or if I need to build xen from the source to get this stubdom work?

Anything obviously wrong in my setup, or suggestion on diagnosis, Roger?

Thanks,

G.R.

> Thanks, Roger.
>
>> [1] https://lists.xenproject.org/archives/html/xen-users/2014-08/msg00003.html
>> [2] https://xen-users.narkive.com/9ihP0QG4/hvm-domu-on-storage-driver-domain