Re: [Xen-devel] Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Wed, 10 Jan 2018, George Dunlap wrote:
> * Executive summary
> 
> - We've agreed on a "convergence" point for PV shim functionality that
>   covers as many users as possible:
>  - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
>    event channels, &c, booted via 'sidecar'
>  - 'PVH' functionality: boots in PVH mode, booted via toolstack
>    changes
> 
> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
>   each cover some users and not others; neither one (yet) covers all
>   users

Sorry for being punctilious, but neither one can cover all users: there
are users without VT-x on their platform, and both approaches require
VT-x.

Both can only be temporary stop-gaps. I made a couple of more comments
in this respect below.



> - Vixen is ready for immediate release; PVH shim still needs some
>   minor revision, and the toolstack component
> 
> - Proposal:
>  - Release "HOWTO" for Amazon shim v1 immediately (including a signed
>    tag on xenbits with the requisite patches)
>  - Release "HOWTO" for PVH shim as soon as it's ready (including a
>    signed tag or tags on xenbits with the requisite patches)
>  - Base future development on the PVH shim (porting over functionality
>    from the Amazon series)
> 
> * Discussion
> 
> In our discussions, we've generally identified four kinds of users /
> consumers of Xen:
> 
> 1. Those for whom upgrading to a newer version of Xen is the biggest
> problem
> 
> This could be people who have local changes they can't forward-port,
> people whose SLAs forbid updating versions, or any number of other
> things that make updating to a newer version of Xen unpalatable or
> impossible.
> 
> 2. Those for whom adding a QEMU instance to all PV guests is the
> biggest problem
> 
> Many users I've talked to extremely dislike the idea of adding QEMU
> instances to all their PV guests, and would rather do upgrades and/or
> modify guest types to avoid this
> 
> 3. Those for whom modifying guest parameters is the biggest problem
> 
> Many users I've talked to are using software which severely limits the
> flexibility they have in terms of how guests are created
> 
> 4. Those for whom some subset of features (migration, ballooning, vcpu
> hotplug, &c) is the biggest problem.
> 
> This includes users who rely on these features, as well as software
> providers that provide those features.

5. Those that cannot deploy neither Vixen nor PVShim due to lack of VT-x



> Amazon is in the first category, and have developed a "PV shim"
> series that addresses their needs.  They have also very kindly
> shared the patches publicly for the community to be able to use.
> 
> Citrix is in the fourth category, and have developed a "PV shim"
> series that addresses their needs.  This shim also satisfies category 2,
> and with some work could support category 3.
> 
> Going forward, the best solutions for new hypervisors (Xen 4.11+)
> would be to avoid needing QEMU, building the sidecar, and so on.  But
> we still want to be able to support people running the 'HVM shim with
> sidecar' version going forward.  So the "convergence point" we've all
> seemed to agree on is to have a shim capable of satisfying all groups:

Going forward, the best solutions for new hypervisors (Xen 4.11+) is to
fix PV in a way that is secure without needing VT-x.



> - Can run in HVM mode back as far as Xen 3.4
> - Can run in PVH mode, without QEMU or "sidecar", and doesn't require
>   many toolstack changes
> - Has feature parity with PV mode (migration, ballooning, vcpu
>   migration, &c)
>
> The issue has been made public for nearly a week now, so there is a
> lot of pressure to get a solution to our users as soon as possible; by
> the end of the week at absolute latest, but today if possible.
> 
> Amazon's v1 series:
> - Has support for 'legacy' event channels
> 
> - Has been tested by Amazon over a wide range of their hypervisor
>   versions and guest types (including pvgrub types) back as far as Xen
>   3.4
> 
> - Uses HVM mode, and so has the overhead and security implications of
>   running QEMU
> 
> - Requires no L0 hypervisor or toolstack changes
> 
> - Requires users to build a "boot sidecar" image for each unique guest
>   boot configuration
> 
> - Is missing many features, such as migration, memory ballooning, and
>   vcpu hotplug
> 
> - Has accurate 'stolen time' support
> 
> Citrix's series:
> - Has been extensively tested by XenServer's XenRT for Xen 4.7 (with
>   PVH backports) and 4.10
> - Uses PVH mode, and so doesn't have the overhead and security
>   implications of running QEMU
> - No need to build "boot sidecar"
> - Requires hypervisor changes for versions for versions before 4.10
>   which cannot reasonably be backported by the open-source team beyond
>   Xen 4.8
> - Requires toolstack changes in all versions
> - Has migration, memory ballooning, and vcpu hotplug (extensively
>   tested by XenRT)
> - Doesn't have accurate 'stolen time' support
> - Currently is missing a clean toolstack interface
> 
> With some tweaks, Citrix's version has been made to boot in HVM mode.
> However, this modified version:
> - Doesn't support the 'legacy' event channel mode, and so won't work
>   for versions as old as Amazon's
> - Hasn't been tested extensively on older versions
> 
> Ian suggested that we get something and check it into 4.10-staging
> as soon as possible.  But there has been a debate about whether we
> should start with Amazon's series and add PVH support, or start with
> Citrix's series and add HVM support (including legacy event channels,
> and so on).
> 
> No matter what, individual users will need to decide whether to take
> the 'HVM + sidecar' option or the 'PVH' option..
> 
> I'd like to propose a new appraoch:
> 
> 1. Immediately release Amazon's v1 series for people who can / prefer
> to use the HVM + sidecar option.
> - Advisory and HOWTO should include who should use this option, and
>   how to do it.
> - Check the series into a branch on xenbits, and add a signed tag
> - HOWTO-vixen, and sidecar script, to be included in the advisory.
> 
> 2. As soon as possible, release Citrix's series for people who can /
> prefer to use the PVH + toolstack option.
> - Immediate work should focus on getting PVH + toolstack functionality
>   ready to release
> - When ready, advisory and HOWTO should include who should use this
>   option and how
> - Either check the series into a branch on xenbits, or into
>   staging-4.10
> - HOWTO-pvh to be included in the advisory.
> 
> This should allow us to get a solution to the widest number of users
> in the shortest amount of time; it also allows us to leverage the
> testing efforts of both Amazon (for breadth of Xen versions) and
> Citrix (for depth of functionality).
> 
> That takes the pressure off us to check in one or the other version of
> the patch series.
> 
> Going forward, we could work towards the convergence of functionality
> from either patch series.  But it looks superficially at least like
> the Citrix series is closer to the convergence point, and so it seems
> like using that as a starting point would make the most sense.
> 
> Regardless of what we think of step 2, I think we should take step 1
> immediately.
> 
> Let me know what you think.
> 
>  -George
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel