[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-devel] [RFC PATCH 00/17] Add support for qemu-xen runnning in a Linux-based stubdomain.
General idea is to allow freely set device_model_version and device_model_stubdomain_override and choose the right options based on this choice. Also, allow to specific path to stubdomain kernel/ramdisk, for greater flexibility. Right now when qemu-xen in stubdomain is selected, it is assumed it's Linux-based and few decisions are based on it, specifically: - qemu args encoding (\x1b as separator, to allow spaces in arguments) - save/restore stream access (specific FDs passed to qemu by a wrapper script) - QMP access, if any It may be a good idea to document "stubdomain API" somewhere. Is there such document for MiniOS one? Here is an initial version for Linux one: Assumptions about Linux-based stubdomain for qemu-xen: * qemu command line is stored by toolstack in /vm/<target-uuid>/image/dmargs xenstore entry, arguments are separated with \x1b character * qemu can access saved state (if any) at its FD 3 * qemu can write its state (for save/migration) to its FD 4 * qemu expects backend for serial console at /dev/hvc3 * disks configured for the target are available as /dev/xvd*, in configuration order * qemu can call /etc/qemu-ifup and /etc/qemu-ifdown to connect/disconnect network interface to appropriate network Initial version has no QMP support - in initial patches it is completely disabled, which means no suspend/restore. QMP normally would be used for PCI passthrough setup, but it is worked around with MiniOS-like control protocol over xenstore, which then is translated to QMP on stubdomain side. Some option is to use separate console for that, but that require xenstoled supporting multiple consoles per domain (the goal is to not have qemu in dom0 at all). Also, it would be preferable to evaluate how libxl handle potentially malicious QMP responses. Another option is to use xenstore - either translate everything needed to MiniOS-like thing, or write already json-formatted requests to xenstore and watch there for responses. When using separate xenstore dir for that, with per-command entries (command id as a key name?) that would solve concurrency problem. QMP support over separate console: patch "libxl: access QMP socket via console for qemu-in-stubdomain" implements some early PoC of this. Major limitation: only one connection at a time and no means of out of band reset (and re-negotiate). I've tried adding very simple `qmp_reset` command for in-band connection reset, but it is unreliable because of the first limitation - xl process running in background hold this connection open and every other xl instance interfere with it. On the other hand, for libvirt use case one connection could be enough (leaving alone libvirtd restart). Xenconsoled patches add support for multiple consoles in xenconsoled, to avoid the need for qemu in dom0 for this to work. Multiple consoles for a stubdomain are used for: - logs (console 0) - save + restore (console 1, 2) - qmp (console 3) - serial terminal to target domU (console 4) Xenconsoled patches are in fact a separate series, but I'm sending them here to ease dependencies handling (latter libxl patches use that). What qmp-libxenstat socket is for? PCI passthrough support require some more love. Right now, libxl tries to setup pcifront for both target HVM and stubdomain and the former fails (race condition): xen-pciback pci-259-0: 22 Couldn't locate PCI device (0000:00:1b.0)! perhaps already in-use? Fortunately it isn't needed. There is also a PCI related problem on domain shutdown - it looks like first stubdomain is paused and then libxl waits for pcifront there. Also, MSI doesn't work (qemu output): [00:05.0] xen_pt_msgctrl_reg_write: setup MSI (register: 81). [00:05.0] msi_msix_setup: requested pirq 55 for MSI (vec: 0, entry: 0) [00:05.0] msi_msix_setup: Error: Mapping of MSI (err: 1, vec: 0, entry 0) [00:05.0] xen_pt_msgctrl_reg_write: Warning: Can not map MSI (register: 80)! [00:05.0] Write-back to unknown field 0x44 (partially) inhibited (0x00) [00:05.0] If the device doesn't work, try enabling permissive mode [00:05.0] (unsafe) and if it helps report the problem to xen-devel The actual stubdomain implementation is here: https://github.com/marmarek/qubes-vmm-xen-stubdom-linux (branch for-upstream) See readme there for build instructions. Remaining parts for eliminating dom0's instance of qemu: - do not force QDISK backend for CDROM This patch series is third (fourth?) attempt to get rid of limitation "if you want to use stubdomain, you're stuck with qemu-traditional", done over years, by many people. I think it should be acceptable plan to gradually add features to qemu-xen+stubdomain configuration - not necessary waiting with committing any of those patches until full feature set of qemu-xen is supported. After all, right now "feature set supported by qemu-xen+stubdom" is empty, so wouldn't be worse. Cc: Simon Gaiser <simon@xxxxxxxxxxxxxxxxxxxxxx> Cc: Eric Shelton <eshelton@xxxxxxxxx> Cc: Ian Jackson <ian.jackson@xxxxxxxxxxxxx> Cc: Wei Liu <wei.liu2@xxxxxxxxxx> Eric Shelton (3): libxl: Add "stubdomain_version" to domain_build_info. libxl: Handle Linux stubdomain specific QEMU options. libxl: Build the domain with a Linux based stubdomain Marek Marczykowski-Górecki (13): libxl: fix qemu-trad cmdline for no sdl/vnc case libxl: create vkb device only for guests with graphics output libxl: add save/restore support for qemu-xen in stubdomain xl: add stubdomain related options to xl config parser libxl: use \x1b to separate qemu arguments for linux stubdomain xenconsoled: install xenstore watch for all supported consoles xenconsoled: add support for consoles using 'state' xenstore entry xenconsoled: make console_type->use_gnttab less confusing xenconsoled: add support for up to 3 secondary consoles xenconsoled: deduplicate error handling xenconsoled: add support for non-pty output libxl: access QMP socket via console for qemu-in-stubdomain libxl: use xenconsoled even for multiple stubdomain's consoles Simon Gaiser (1): libxl: use xenstore for pci hotplug qemu-in-linux-stubdom commands (and part of other commits, as noted in commit messages there) docs/man/xl.cfg.pod.5.in | 23 ++- tools/console/daemon/io.c | 222 ++++++++++++++++++++++++++++----- tools/libxl/libxl_create.c | 83 ++++++++++-- tools/libxl/libxl_dm.c | 182 ++++++++++++++++++--------- tools/libxl/libxl_dom_suspend.c | 10 +- tools/libxl/libxl_internal.c | 22 +++- tools/libxl/libxl_internal.h | 9 +- tools/libxl/libxl_mem.c | 6 +- tools/libxl/libxl_pci.c | 22 ++- tools/libxl/libxl_qmp.c | 52 +++++++- tools/libxl/libxl_types.idl | 10 +- tools/xl/xl_parse.c | 7 +- 12 files changed, 537 insertions(+), 111 deletions(-) base-commit: e3f667bc5f51d0aa44357a64ca134cd952679c81 -- git-series 0.9.1 _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxxx https://lists.xenproject.org/mailman/listinfo/xen-devel
|
![]() |
Lists.xenproject.org is hosted with RackSpace, monitoring our |