[Xen-devel] [PATCH v2 00/62] Comet: Run PV in PVH container

[Wei Liu <wei.liu2@xxxxxxxxxx>]

Hi all

This is a patch series to run PV guest inside a PVH container. It should
also support running under HVM mode but we've not extensively tested it.
The future plan is for this to merge with the Vixen work Amazon did.
 
The series can be found at:

    https://xenbits.xen.org/git-http/people/liuw/xen.git wip.pvshim-v2.3

The basic idea can be found at page 15 of the slides at [0]. This can
be used to mitigate Meltdown as stated in [1].

The difference between Comet and Vixen is discussed somewhere else. Please
consult those threads to pick the right short term solution.

This version now has a functional toolstack. A lot of comments in the previous
version are addressed, but there are leftovers.

Instructions on using the shim:

1. Git clone the branch and configure as you normally would.
2. A xen-shim binary would be built and installed into Xen's firmware
   directory, along side hvmloader and co.
3. Append the following options to guest config file
       type = 'pvh'
       pvshim = 1
       pvshim_cmdline = "xxxx" # if any
       pvshim_extra = "xxxx"
   Please see xl manpage for more information.
   I personally use
       pvshim_cmdline="pv-shim console=xen,pv loglvl=all guest_loglvl=all 
apic_verbosity=debug e820-verbose sched=null"
   to get more debugging output.
4. xl create -c guest.cfg

You should be able to see some Xen messages first and then guest kernel
messages.

The patch series is split into three parts. Please see the markers.

# Host patches

c2b5d19366 x86/svm: Offer CPUID Faulting to AMD HVM guests as well
de1db54d3d xen/x86: report domain id on cpuid
feaddf0628 tools/libxc: remove extraneous newline in xc_dom_load_acpi
2d73544d60 tools/libelf: fix elf notes check for PVH guest
605ecc28ab tools/libxc: Multi modules support
8daffd4bba xen/common: Widen the guest logging buffer slightly

# Shim patches

2008783b68 x86/time: Print a more helpful error when a platform timer can't be 
found
aa43415bec x86/link: Introduce and use SECTION_ALIGN
53d943eeb8 ACPICA: Make ACPI Power Management Timer (PM Timer) optional.
030f8d0fd6 xen/domctl: Return arch_config via getdomaininfo
79fcb598ae tools/ocaml: Expose arch_config in domaininfo
990438cd5d tools/ocaml: Extend domain_create() to take arch_domainconfig
bd2ccf4c6f x86/fixmap: Modify fix_to_virt() to return a void pointer
8c36939118 x86: Common cpuid faulting support
edcad6ef4b x86/Kconfig: Options for Xen and PVH support
a5446dbf31 x86/link: Relocate program headers
80b3a08f9c x86: introduce ELFNOTE macro
068933f37a x86: produce a binary that can be booted as PVH
3acd23e0a1 x86/entry: Early PVH boot code
cdd1de873d x86/boot: Map more than the first 16MB
3f6c1c33ec x86/entry: Probe for Xen early during boot
855856e857 x86/guest: Hypercall support
f204a73fef x86/shutdown: Support for using SCHEDOP_{shutdown,reboot}
3c9590135d x86/pvh: Retrieve memory map from Xen
85cfd2a735 xen/console: Introduce console=xen
47ad4ecf25 xen: introduce rangeset_claim_range
971b5c8d1f xen/pvshim: keep track of used PFN ranges
e3039f2df0 x86/guest: map shared_info page
3997881097 xen/guest: fetch vCPU ID from Xen
d693b10299 x86/guest: map per-cpu vcpu_info area.
c582962ac7 x86: xen pv clock time source
2e8afc1ead x86: APIC timer calibration when running as a guest
b5a6e58ff3 x86: read wallclock from Xen when running in pvh mode
bc3e5e866a x86: don't swallow the first command line item in guest mode
09cedc3af3 x86/guest: setup event channel upcall vector
8343a651ea x86/guest: add PV console code
ec8fb8f7a1 x86/guest: use PV console for Xen/Dom0 I/O
1ec222a153 x86/shim: Kconfig and command line options
0620686d72 tools/firmware: Build and install xen-shim
536339a994 xen/x86: make VGA support selectable
21f0a6ed97 xen/pvh: do not mark the low 1MB as IO mem
db104c0391 sched/null: skip vCPUs on the waitqueue that are blocked
549dba6a5a xen/pvshim: skip Dom0-only domain builder parts
1bba66e4c7 xen: mark xenstore/console pages as RAM
b907ed7322 xen/pvshim: modify Dom0 builder in order to build a DomU
867380c40e xen/pvshim: set correct domid value
d3682baafb xen/pvshim: forward evtchn ops between L0 Xen and L2 DomU
25f682b751 xen/pvshim: add grant table operations
c538c7f635 x86/pv-shim: shadow PV console's page for L2 DomU
85d142a053 xen/pvshim: add migration support
92612a97ef xen/pvshim: add shim_mem cmdline parameter
1f64261cca xen/pvshim: set max_pages to the value of tot_pages
f9f9265f0e xen/pvshim: support vCPU hotplug
7f723b6dff xen/pvshim: memory hotplug
5016c05fc9 xen/shim: modify shim_mem parameter behaviour
4f4cf3304a xen/pvshim: use default position for the m2p mappings
c7bde74df9 xen/shim: crash instead of reboot in shim mode
f842ed539a xen/shim: allow DomU to have as many vcpus as available

# Toolstack patches

5c43c5721c libxl: pvshim: Provide first-class config settings to enable shim 
mode
6833537050 libxl: pvshim: Introduce pvshim_extra
c128981e73 xl: pvshim: Provide and document xl config
4689dc2e29 xl: Default guest mode changed from PV to PVH with PV shim


Wei.

[0] 
https://www.slideshare.net/xen_com_mgr/xpdds17-keynote-towards-a-configurable-and-slimmer-x86-hypervisor-wei-liu-citrix
[1] https://xenbits.xen.org/xsa/advisory-254.html


 .gitignore                            |   5 +
 docs/man/xl.cfg.pod.5.in              |  35 ++
 docs/misc/xen-command-line.markdown   |  36 +-
 stubdom/grub/kexec.c                  |   7 +-
 tools/firmware/Makefile               |   9 +
 tools/firmware/xen-dir/Makefile       |  59 +++
 tools/firmware/xen-dir/shim.config    |  85 +++
 tools/helpers/init-xenstore-domain.c  |   4 +-
 tools/libxc/include/xc_dom.h          |  48 +-
 tools/libxc/include/xenctrl.h         |   1 +
 tools/libxc/xc_dom_compat_linux.c     |   2 +-
 tools/libxc/xc_dom_core.c             | 154 ++++--
 tools/libxc/xc_dom_x86.c              |  65 +--
 tools/libxc/xc_domain.c               |   1 +
 tools/libxl/libxl.h                   |   8 +
 tools/libxl/libxl_create.c            |  22 +-
 tools/libxl/libxl_dom.c               |  67 ++-
 tools/libxl/libxl_internal.h          |   4 +
 tools/libxl/libxl_types.idl           |   6 +-
 tools/ocaml/libs/xc/xenctrl.ml        |  31 +-
 tools/ocaml/libs/xc/xenctrl.mli       |  30 +-
 tools/ocaml/libs/xc/xenctrl_stubs.c   |  48 +-
 tools/xl/xl_parse.c                   |  25 +-
 xen/Makefile                          |  16 +-
 xen/arch/x86/Kconfig                  |  40 +-
 xen/arch/x86/Makefile                 |   9 +
 xen/arch/x86/acpi/lib.c               |   2 +-
 xen/arch/x86/apic.c                   |  38 +-
 xen/arch/x86/boot/build32.mk          |   1 +
 xen/arch/x86/boot/cmdline.c           |   5 +-
 xen/arch/x86/boot/head.S              |  47 ++
 xen/arch/x86/boot/trampoline.S        |   7 +
 xen/arch/x86/boot/x86_64.S            |   5 +-
 xen/arch/x86/compat.c                 |   4 +-
 xen/arch/x86/cpu/amd.c                |  16 +-
 xen/arch/x86/cpu/common.c             |  76 ++-
 xen/arch/x86/cpu/intel.c              |  81 +--
 xen/arch/x86/dom0_build.c             |  48 +-
 xen/arch/x86/domctl.c                 |   2 +
 xen/arch/x86/e820.c                   |   7 +-
 xen/arch/x86/efi/efi-boot.h           |   4 +
 xen/arch/x86/guest/Makefile           |   4 +
 xen/arch/x86/guest/hypercall_page.S   |  79 +++
 xen/arch/x86/guest/pvh-boot.c         | 139 +++++
 xen/arch/x86/guest/xen.c              | 388 ++++++++++++++
 xen/arch/x86/hvm/dom0_build.c         |   4 -
 xen/arch/x86/hvm/svm/svm.c            |   6 +
 xen/arch/x86/mm.c                     |  13 +-
 xen/arch/x86/mpparse.c                |   2 +-
 xen/arch/x86/msi.c                    |   3 +-
 xen/arch/x86/msr.c                    |   3 +-
 xen/arch/x86/platform_hypercall.c     |   2 +
 xen/arch/x86/pv/Makefile              |   1 +
 xen/arch/x86/pv/dom0_build.c          |  58 ++-
 xen/arch/x86/pv/hypercall.c           |  17 +
 xen/arch/x86/pv/shim.c                | 957 ++++++++++++++++++++++++++++++++++
 xen/arch/x86/setup.c                  |  84 ++-
 xen/arch/x86/shutdown.c               |  39 +-
 xen/arch/x86/smpboot.c                |   4 +
 xen/arch/x86/tboot.c                  |   4 +-
 xen/arch/x86/time.c                   | 124 ++++-
 xen/arch/x86/traps.c                  |   5 +
 xen/arch/x86/xen.lds.S                |  82 ++-
 xen/common/domain.c                   |  53 +-
 xen/common/event_channel.c            |  99 ++--
 xen/common/libelf/libelf-dominfo.c    |   9 +-
 xen/common/memory.c                   |  21 +
 xen/common/page_alloc.c               |  15 +
 xen/common/rangeset.c                 |  52 ++
 xen/common/sched_null.c               |  11 +-
 xen/common/schedule.c                 |   3 +-
 xen/drivers/acpi/apei/apei-io.c       |   2 +-
 xen/drivers/acpi/tables/tbfadt.c      |   5 +-
 xen/drivers/char/Makefile             |   2 +
 xen/drivers/char/console.c            |  72 +++
 xen/drivers/char/consoled.c           | 148 ++++++
 xen/drivers/char/ehci-dbgp.c          |   2 +-
 xen/drivers/char/ns16550.c            |   2 +-
 xen/drivers/char/xen_pv_console.c     | 208 ++++++++
 xen/drivers/video/Kconfig             |   8 +-
 xen/include/asm-x86/apicdef.h         |   2 +-
 xen/include/asm-x86/asm_defns.h       |  12 +
 xen/include/asm-x86/cpuid.h           |   3 -
 xen/include/asm-x86/dom0_build.h      |   4 +
 xen/include/asm-x86/e820.h            |   1 +
 xen/include/asm-x86/fixmap.h          |   6 +-
 xen/include/asm-x86/guest.h           |  37 ++
 xen/include/asm-x86/guest/hypercall.h | 206 ++++++++
 xen/include/asm-x86/guest/pvh-boot.h  |  57 ++
 xen/include/asm-x86/guest/xen.h       |  92 ++++
 xen/include/asm-x86/hypercall.h       |   3 +
 xen/include/asm-x86/processor.h       |   4 +-
 xen/include/asm-x86/pv/shim.h         | 107 ++++
 xen/include/asm-x86/setup.h           |   6 +
 xen/include/public/arch-x86/cpuid.h   |   2 +
 xen/include/public/domctl.h           |   3 +-
 xen/include/xen/consoled.h            |  27 +
 xen/include/xen/domain.h              |   1 +
 xen/include/xen/event.h               |  15 +
 xen/include/xen/pv_console.h          |  38 ++
 xen/include/xen/rangeset.h            |   4 +-
 xen/include/xen/sched.h               |   4 +-
 102 files changed, 4106 insertions(+), 418 deletions(-)
 create mode 100644 tools/firmware/xen-dir/Makefile
 create mode 100644 tools/firmware/xen-dir/shim.config
 create mode 100644 xen/arch/x86/guest/Makefile
 create mode 100644 xen/arch/x86/guest/hypercall_page.S
 create mode 100644 xen/arch/x86/guest/pvh-boot.c
 create mode 100644 xen/arch/x86/guest/xen.c
 create mode 100644 xen/arch/x86/pv/shim.c
 create mode 100644 xen/drivers/char/consoled.c
 create mode 100644 xen/drivers/char/xen_pv_console.c
 create mode 100644 xen/include/asm-x86/guest.h
 create mode 100644 xen/include/asm-x86/guest/hypercall.h
 create mode 100644 xen/include/asm-x86/guest/pvh-boot.h
 create mode 100644 xen/include/asm-x86/guest/xen.h
 create mode 100644 xen/include/asm-x86/pv/shim.h
 create mode 100644 xen/include/xen/consoled.h
 create mode 100644 xen/include/xen/pv_console.h

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel