|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-users] Storage driver domain for HVM guest
On Mon, Feb 22, 2016 at 6:15 AM, Roger Pau Monnà <roger.pau@xxxxxxxxxx> wrote: > El 19/2/16 a les 17:24, Alex Velazquez ha escrit: >> Hello, >> >> I am running Xen 4.6.0, with Ubuntu 14.04 as my Domain-0. >> >> I am trying to set up a storage driver domain to serve disks to a >> variety of DomU's (for now, all guest OSes are also Ubuntu 14.04). I >> followed the instructions on the following Wiki page to set up the >> storage driver domain: >> >>> http://wiki.xenproject.org/wiki/Storage_driver_domains >> >> This works fine for a PV guest. After adding "backend=StorageDom" in >> the disk spec, the guest is able to boot from a disk provided by the >> storage driver domain. >> >> I am wondering if it is also possible for a storage driver domain to >> provide disks to HVM (or PVHVM) guests. At the bottom of the Wiki >> page, there is a note indicating that it isn't possible (yet?): >> >>> It is not possible to use driver domains with pygrub or HVM guests yet, so >>> it will only work with PV guests that have the kernel in Dom0. >> >> Some time has passed since that page was last modified, so I decided >> to try it. Here is the config file of my PVHVM guest: >> >>> name = "ClientDom" >>> memory = 1024 >>> maxmem = 1024 >>> vcpus = 2 >>> maxvcpus = 2 >>> vnc = 1 >>> vnclisten = "0.0.0.0" >>> vncdisplay = 1 >>> builder = "hvm" >>> xen_platform_pci = 1 >>> device_model_version = "qemu-xen" >>> disk = [ >>> "format=raw,vdev=xvda,access=rw,backend=StorageDom,target=/dev/sdb" ] >>> boot = "c" >> >> The disk target /dev/sdb is relative to the storage driver domain, on >> a passthrough PCI device. That disk has three partitions: /boot >> (bootable flag set), / (root), and swap. However, after creating the >> guest, the BIOS (SeaBIOS for qemu-xen, but I also tried with >> qemu-xen-traditional with similar results) complains that no bootable >> device is found: >> >>> Booting from Hard Disk... >>> Boot failed: not a bootable disk >>> [ .... ] >>> No bootable device. Retrying in 60 seconds. >> >> In xenstore, the disk backend/frontend appear to be setup correctly: >> >>> user@ubuntu ~> $ sudo xenstore-ls /local/domain/1/backend >>> vbd = "" >>> 2 = "" >>> 51712 = "" >>> frontend = "/local/domain/2/device/vbd/51712" >>> params = "/dev/sdb" >>> script = "/etc/xen/scripts/block" >>> frontend-id = "2" >>> online = "1" >>> removable = "0" >>> bootable = "1" >>> state = "2" >>> dev = "xvda" >>> type = "phy" >>> mode = "w" >>> device-type = "disk" >>> discard-enable = "1" >>> physical-device = "8:10" >>> hotplug-status = "connected" >> >>> user@ubuntu ~> $ sudo xenstore-ls /local/domain/2/device >>> suspend = "" >>> event-channel = "" >>> vbd = "" >>> 51712 = "" >>> backend = "/local/domain/1/backend/vbd/2/51712" >>> backend-id = "1" >>> state = "1" >>> virtual-device = "51712" >>> device-type = "disk" >>> vkbd = "" >>> 0 = "" >>> backend = "/local/domain/0/backend/vkbd/2/0" >>> backend-id = "0" >>> state = "1" >> >> Here is the verbose output of the "xl create" command: >> >>> user@ubuntu ~> $ sudo xl -vvv create /etc/xen/ClientDom.cfg >>> Parsing config from /etc/xen/ClientDom.cfg >>> libxl: debug: libxl_create.c:1568:do_domain_create: ao 0x1df8230: create: >>> how=(nil) callback=(nil) poller=0x1dee440 >>> libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk >>> vdev=xvda spec.backend=unknown >>> libxl: debug: libxl_device.c:201:disk_try_backend: Disk vdev=xvda, is using >>> a storage driver domain, skipping physical device check >>> libxl: debug: libxl_device.c:298:libxl__device_disk_set_backend: Disk >>> vdev=xvda, using backend phy >>> libxl: debug: libxl_create.c:954:initiate_domain_create: running bootloader >>> libxl: debug: libxl_bootloader.c:324:libxl__bootloader_run: not a PV >>> domain, skipping bootloader >>> libxl: debug: libxl_event.c:691:libxl__ev_xswatch_deregister: watch >>> w=0x1deede8: deregister unregistered >>> libxl: debug: libxl_numa.c:483:libxl__get_numa_candidate: New best NUMA >>> placement candidate found: nr_nodes=1, nr_cpus=24, nr_vcpus=4, >>> free_memkb=255993 >>> libxl: debug: libxl_numa.c:483:libxl__get_numa_candidate: New best NUMA >>> placement candidate found: nr_nodes=1, nr_cpus=24, nr_vcpus=4, >>> free_memkb=256242 >>> libxl: detail: libxl_dom.c:181:numa_place_domain: NUMA placement candidate >>> with 1 nodes, 24 cpus and 256242 KB free selected >>> xc: detail: elf_parse_binary: phdr: paddr=0x100000 memsz=0xbad04 >>> xc: detail: elf_parse_binary: memory: 0x100000 -> 0x1bad04 >>> xc: detail: VIRTUAL MEMORY ARRANGEMENT: >>> xc: detail: Loader: 0000000000100000->00000000001bad04 >>> xc: detail: Modules: 0000000000000000->0000000000000000 >>> xc: detail: TOTAL: 0000000000000000->000000003f800000 >>> xc: detail: ENTRY: 0000000000100610 >>> xc: detail: PHYSICAL MEMORY ALLOCATION: >>> xc: detail: 4KB PAGES: 0x0000000000000200 >>> xc: detail: 2MB PAGES: 0x00000000000001fb >>> xc: detail: 1GB PAGES: 0x0000000000000000 >>> xc: detail: elf_load_binary: phdr 0 at 0x7fc11e76e000 -> 0x7fc11e81f151 >>> domainbuilder: detail: xc_dom_gnttab_hvm_seed: called, pfn=0xff000 >>> libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk >>> vdev=xvda spec.backend=phy >>> libxl: debug: libxl_device.c:201:disk_try_backend: Disk vdev=xvda, is using >>> a storage driver domain, skipping physical device check >>> libxl: debug: libxl_event.c:639:libxl__ev_xswatch_register: watch >>> w=0x1df0880 wpath=/local/domain/1/backend/vbd/2/51712/state token=3/0: >>> register slotnum=3 >>> libxl: debug: libxl_create.c:1591:do_domain_create: ao 0x1df8230: >>> inprogress: poller=0x1dee440, flags=i >>> libxl: debug: libxl_event.c:576:watchfd_callback: watch w=0x1df0880 >>> wpath=/local/domain/1/backend/vbd/2/51712/state token=3/0: event >>> epath=/local/domain/1/backend/ vbd/2/51712/state >>> libxl: debug: libxl_event.c:884:devstate_callback: backend >>> /local/domain/1/backend/vbd/2/51712/state wanted state 2 still waiting >>> state 1 >>> libxl: debug: libxl_event.c:576:watchfd_callback: watch w=0x1df0880 >>> wpath=/local/domain/1/backend/vbd/2/51712/state token=3/0: event >>> epath=/local/domain/1/backend/ vbd/2/51712/state >>> libxl: debug: libxl_event.c:880:devstate_callback: backend >>> /local/domain/1/backend/vbd/2/51712/state wanted state 2 ok >>> libxl: debug: libxl_event.c:677:libxl__ev_xswatch_deregister: watch >>> w=0x1df0880 wpath=/local/domain/1/backend/vbd/2/51712/state token=3/0: >>> deregister slotnum=3 >>> libxl: debug: libxl_device.c:937:device_backend_callback: calling >>> device_backend_cleanup >>> libxl: debug: libxl_event.c:691:libxl__ev_xswatch_deregister: watch >>> w=0x1df0880: deregister unregistered >>> libxl: debug: libxl_device.c:992:device_hotplug: Backend domid 1, domid 0, >>> assuming driver domains >>> libxl: debug: libxl_device.c:995:device_hotplug: Not a remove, not >>> executing hotplug scripts >>> libxl: debug: libxl_event.c:691:libxl__ev_xswatch_deregister: watch >>> w=0x1df0980: deregister unregistered >>> libxl: debug: libxl_dm.c:1778:libxl__spawn_local_dm: Spawning device-model >>> /usr/local/lib/xen/bin/qemu-system-i386 with arguments: >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: >>> /usr/local/lib/xen/bin/qemu-system-i386 >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -xen-domid >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: 2 >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -chardev >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: >>> socket,id=libxl-cmd,path=/var/run/xen/qmp-libxl-2,server,nowait >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -no-shutdown >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -mon >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: >>> chardev=libxl-cmd,mode=control >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -chardev >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: >>> socket,id=libxenstat-cmd,path=/var/run/xen/qmp-libxenstat-2,server,nowait >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -mon >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: >>> chardev=libxenstat-cmd,mode=control >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -nodefaults >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -name >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: ClientDom >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -vnc >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: 0.0.0.0:1,to=99 >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -display >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: none >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -device >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: >>> cirrus-vga,vgamem_mb=8 >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -boot >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: order=c >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -smp >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: 2,maxcpus=2 >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -net >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: none >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -machine >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: xenfv >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -m >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: 1016 >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: -drive >>> libxl: debug: libxl_dm.c:1780:libxl__spawn_local_dm: >>> file=/dev/sdb,if=ide,index=0,media=disk,format=raw,cache=writeback >>> [ .... ] >> >> I'm not sure what qemu's role here is, but that last line with >> "file=/dev/sdb..." makes no mention of a backend, so I thought it >> might be picking up /dev/sdb relative to Domain-0, instead of the >> storage driver domain. To test this theory, I created a partition >> table on Domain-0's /dev/sdb with a bootable partition at /dev/sdb1. >> Indeed, after creating the guest, I get to the GRUB2 menu that was >> installed on Domain-0's /dev/sdb (despite still having StorageDom >> configured as the backend). Upon selecting a boot entry, the guest OS >> begins to boot but drops into BusyBox because it can't find the >> boot/root/swap partitions (because Domain-0's /dev/sdb partitions and >> StorageDom's /dev/sdb partitions have different UUIDs). >> >> So, in short, it seems like the BIOS sees a disk that exists on >> Domain-0, and then once the guest kernel and guest OS take over, they >> see the disk provided by the storage driver domain. >> >> If anyone has experience getting a storage driver domain to serve >> disks that HVM/PVHVM guests can use to boot, or can spot any mistakes >> I've made, (or can tell me it's not possible yet), I'd be very >> grateful for advice. > > Sadly the wiki page is still right, the only way to use disk driver > domains with HVM guests is to use a stubdomain (by adding > device_model_stubdomain_override=1 to your domain config file). > > To sum up what's missing: QEMU needs a physical device in Dom0 in order > to use it as a hard-drive, but since you are using a driver domain QEMU > cannot access this device at all (your case is even worse, because QEMU > tries to use the disk in Dom0). The only way to solve this is to teach > xl to attach the disk provided by the driver domain to Dom0 and then > pass that device to QEMU. As usual, patches are welcome :). > > Roger. > Hi Roger, Thanks for the explanation; that makes sense. I will try with stubdomains to see if I can get that to work, or otherwise stick with PV for now. Thanks, Alex _______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxx http://lists.xen.org/xen-users
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |