Re: [Xen-devel] Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Anthony Liguori <anthony@xxxxxxxxxxxxx>]

On Wed, Jan 10, 2018 at 5:51 AM, Juergen Gross <jgross@xxxxxxxx> wrote:
> On 10/01/18 14:34, George Dunlap wrote:
>> * Executive summary
>>
>> - We've agreed on a "convergence" point for PV shim functionality that
>>   covers as many users as possible:
>>  - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
>>    event channels, &c, booted via 'sidecar'
>>  - 'PVH' functionality: boots in PVH mode, booted via toolstack
>>    changes
>>
>> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
>>   each cover some users and not others; neither one (yet) covers all
>>   users
>>
>> - Vixen is ready for immediate release; PVH shim still needs some
>>   minor revision, and the toolstack component
>>
>> - Proposal:
>>  - Release "HOWTO" for Amazon shim v1 immediately (including a signed
>>    tag on xenbits with the requisite patches)
>>  - Release "HOWTO" for PVH shim as soon as it's ready (including a
>>    signed tag or tags on xenbits with the requisite patches)
>>  - Base future development on the PVH shim (porting over functionality
>>    from the Amazon series)
>>
>> * Discussion
>>
>> In our discussions, we've generally identified four kinds of users /
>> consumers of Xen:
>>
>> 1. Those for whom upgrading to a newer version of Xen is the biggest
>> problem
>>
>> This could be people who have local changes they can't forward-port,
>> people whose SLAs forbid updating versions, or any number of other
>> things that make updating to a newer version of Xen unpalatable or
>> impossible.
>>
>> 2. Those for whom adding a QEMU instance to all PV guests is the
>> biggest problem
>>
>> Many users I've talked to extremely dislike the idea of adding QEMU
>> instances to all their PV guests, and would rather do upgrades and/or
>> modify guest types to avoid this
>>
>> 3. Those for whom modifying guest parameters is the biggest problem
>>
>> Many users I've talked to are using software which severely limits the
>> flexibility they have in terms of how guests are created
>>
>> 4. Those for whom some subset of features (migration, ballooning, vcpu
>> hotplug, &c) is the biggest problem.
>>
>> This includes users who rely on these features, as well as software
>> providers that provide those features.
>>
>> Amazon is in the first category, and have developed a "PV shim"
>> series that addresses their needs.  They have also very kindly
>> shared the patches publicly for the community to be able to use.
>>
>> Citrix is in the fourth category, and have developed a "PV shim"
>> series that addresses their needs.  This shim also satisfies category 2,
>> and with some work could support category 3.
>>
>> Going forward, the best solutions for new hypervisors (Xen 4.11+)
>> would be to avoid needing QEMU, building the sidecar, and so on.  But
>> we still want to be able to support people running the 'HVM shim with
>> sidecar' version going forward.  So the "convergence point" we've all
>> seemed to agree on is to have a shim capable of satisfying all groups:
>>
>> - Can run in HVM mode back as far as Xen 3.4
>> - Can run in PVH mode, without QEMU or "sidecar", and doesn't require
>>   many toolstack changes
>> - Has feature parity with PV mode (migration, ballooning, vcpu
>>   migration, &c)
>>
>> The issue has been made public for nearly a week now, so there is a
>> lot of pressure to get a solution to our users as soon as possible; by
>> the end of the week at absolute latest, but today if possible.
>>
>> Amazon's v1 series:
>> - Has support for 'legacy' event channels
>>
>> - Has been tested by Amazon over a wide range of their hypervisor
>>   versions and guest types (including pvgrub types) back as far as Xen
>>   3.4
>>
>> - Uses HVM mode, and so has the overhead and security implications of
>>   running QEMU
>>
>> - Requires no L0 hypervisor or toolstack changes
>>
>> - Requires users to build a "boot sidecar" image for each unique guest
>>   boot configuration
>>
>> - Is missing many features, such as migration, memory ballooning, and
>>   vcpu hotplug
>>
>> - Has accurate 'stolen time' support
>>
>> Citrix's series:
>> - Has been extensively tested by XenServer's XenRT for Xen 4.7 (with
>>   PVH backports) and 4.10
>> - Uses PVH mode, and so doesn't have the overhead and security
>>   implications of running QEMU
>> - No need to build "boot sidecar"
>> - Requires hypervisor changes for versions for versions before 4.10
>>   which cannot reasonably be backported by the open-source team beyond
>>   Xen 4.8
>> - Requires toolstack changes in all versions
>> - Has migration, memory ballooning, and vcpu hotplug (extensively
>>   tested by XenRT)
>> - Doesn't have accurate 'stolen time' support
>> - Currently is missing a clean toolstack interface
>>
>> With some tweaks, Citrix's version has been made to boot in HVM mode.
>> However, this modified version:
>> - Doesn't support the 'legacy' event channel mode, and so won't work
>>   for versions as old as Amazon's
>> - Hasn't been tested extensively on older versions
>>
>> Ian suggested that we get something and check it into 4.10-staging
>> as soon as possible.  But there has been a debate about whether we
>> should start with Amazon's series and add PVH support, or start with
>> Citrix's series and add HVM support (including legacy event channels,
>> and so on).
>>
>> No matter what, individual users will need to decide whether to take
>> the 'HVM + sidecar' option or the 'PVH' option..
>>
>> I'd like to propose a new appraoch:
>>
>> 1. Immediately release Amazon's v1 series for people who can / prefer
>> to use the HVM + sidecar option.
>> - Advisory and HOWTO should include who should use this option, and
>>   how to do it.
>> - Check the series into a branch on xenbits, and add a signed tag
>> - HOWTO-vixen, and sidecar script, to be included in the advisory.
>>
>> 2. As soon as possible, release Citrix's series for people who can /
>> prefer to use the PVH + toolstack option.
>> - Immediate work should focus on getting PVH + toolstack functionality
>>   ready to release
>> - When ready, advisory and HOWTO should include who should use this
>>   option and how
>> - Either check the series into a branch on xenbits, or into
>>   staging-4.10
>> - HOWTO-pvh to be included in the advisory.
>>
>> This should allow us to get a solution to the widest number of users
>> in the shortest amount of time; it also allows us to leverage the
>> testing efforts of both Amazon (for breadth of Xen versions) and
>> Citrix (for depth of functionality).
>>
>> That takes the pressure off us to check in one or the other version of
>> the patch series.
>>
>> Going forward, we could work towards the convergence of functionality
>> from either patch series.  But it looks superficially at least like
>> the Citrix series is closer to the convergence point, and so it seems
>> like using that as a starting point would make the most sense.
>>
>> Regardless of what we think of step 2, I think we should take step 1
>> immediately.
>>
>> Let me know what you think.
>
> I'm absolutely in favor of that idea.

I've pushed https://github.com/aliguori/xen/tree/vixen-upstream-v1.5 which
contains the original series + one fix from Wei for xl shutdown.

I think this is the branch to merge.

Regards,

Anthony Liguori

>
>
> Juergen
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxxx
> https://lists.xenproject.org/mailman/listinfo/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel