[Xen-devel] Proposal: Feature Maturity Lifecycle

[Lars Kurth <lars.kurth.xen@xxxxxxxxx>]

Hi all,

following up from 
http://lists.xenproject.org/archives/html/xen-devel/2015-06/msg01775.html I 
wanted to propose the following convention related to feature classification as 
a proposal for discussion. I tried to pretty much describe what we do now and 
hope my understanding is correct. I did propose a new Complete state, to cover 
for bigger new features which we may not want to mark as supported straight 
away, for various reasons.

I marked areas which with some of my thoughts or references to other documents 
in [Note: ...] brackets

Regards
Lars

----

        Feature Maturity Lifecycle
        ==========================

        Purpose and Scope
        -----------------

The purpose of this document is to define the possible states a feature. 
*Features* may be one of the following:
* End user Feature: aka a pice of functionality that is controllable 
  by users of Xen through a command line option exposed via xl or 
  libvirt, a config file or otherwise
* Support for a Hardware platform: e.g. x86 or ARM
* Support for a specific Hardware platforms, e.g. Thunder X, 
  Xilinx ZynqMP, etc.
* A set of APIs, e.g. men-event API, memory sharing APIs, etc.

It is also the intent, that this document is used to set expectations
of contributors to the Xen Project hypervisor, such that they can take 
appropriate steps to ensure that eventually a feature that they develop
is a supported by the community.

Further, it will help users of the Xen Project hypervisors understand
how developers use the feature states. This will help users make informed
decisions about the risk involved in using a feature. 

        Dependencies
        -------------

This document refers to definitions in other files and project conventions,
in particular:

1.      Status of subsystems in the MAINTAINERS file
        The MAINTAINERS file maps individuals against groups of files 
        in the source tree. In there context of this document, we say that
        a feature is *maintained*, iff all components of that feature
        are marked as one of the following
        Supported:  Someone is actually paid to look after the code.
        Maintained: Someone actually looks after the code.

2.      Classification of public APIs and interfaces: 
        APIs and other interfaces are declared stable by agreement
        on the xen-devel@ mailing list. We consider a feature as
        *stable*, iff all public APIs and interfaces a feature depends 
        on have been declared stable.

3.      Testing
        The Xen Project runs a continuous test and integration system 
        in the project's test lab. We consider a feature *tested*, iff 
        there are test cases that are run in our test lab, that test a  
        specific feature and consistently pass. For hardware platforms we 
        consider a platform *tested*, iff appropriate hardware for said 
        platform is live and in active use in our test lab and tests
        consistently pass against that platform.

        In some cases, it may not be possible to add hardware to the
        Xen Project test lab (e.g. for cost, space, or other reasons).
        In such cases, it is acceptable for a community member or 
        organisation to run their own tests on behalf of the community.
        In such cases, we consider a feature or platform *tested*
        if said community member tests a feature/platform using their
        own infrastructure and regularly and consistently reports results 
        to the community via xen-devel@. At a minimum, release candidates 
        should be tested.

4.      Documentation
        The Xen Project requires that documentation for user facing 
        features and in-code API documentation (or user guides as 
        appropriate) are provided in tree. We say that a feature as
        *documented*, if relevant documentation has been committed to
        the tree.

        State Definitions
        -----------------

        This section lists state definitions of a *Feature* in terms of
        properties. States are listed in order of increasing number
        of properties. Note that note all features will require to go 
        through each state: for example small and non-risky features
        may go straight from under development to supported. It is up to 
        the development community to judge and discuss, which states
        are necessary based on the size and risk of a feature.

1.      Preview
        - Partially completed, with missing functionality
        - May not be fully functional in all cases
        - May not be tested
        - APIs and interfaces may not be stable
        - The developer is actively looking for user feedback
        - Bugs and issues can be raised on xen-devel@ and will be
          handled on a best-effort basis

[Note:  the term prototype is listed in some cases in 
        http://wiki.xenproject.org/wiki/Xen_Project_Release_Features -
        I propose to purge prototype and replace with Preview]

2.      Experimental
        - Core functionality is fully functional
        - However, not all functionality or platform support may be
          present
        - May not be tested, although there is an expectation that a plan
          to improve testing is in place or worked on
        - APIs and interfaces may not be stable
        - Bugs and issues can be raised on xen-devel@ and will be
          handled on a best-effort basis. However, there is an expectation 
          that issues related to broken core functionality are addressed.

3.      Complete
[Note:  This is a state which has not existed in the past. It is aimed
        at larger new features, which may only be in use or of interest
        to a small number of contributors, or where not enough expertise
        exists in the community to treat the feature as *Supported*. It
        presents an opportunity for developers to prove over one or
        two release cycles that said feature can be *supported* in future.]
        - Intended functionality is fully implemented
        - Feature is *maintained*
        - Feature is *tested*
        - Feature is *stable*
        - Feature is *documented*
        - Bugs and issues can be raised on xen-devel@ and will be
          handled by the developers/maintainers behind the feature. There 
          is an expectation for the developers/maintainers to address 
          bugs and issues over a period of time. 
        - Regressions and blockers in a complete feature do *not* normally
          block new releases. It is at the discretion of the community
          and Release Manager to downgrade the feature. 
        - Security issues would *not* be handled by the security team.
    
4.      Supported
        - Intended functionality is fully implemented
        - Feature is *maintained*
        - Feature is *tested*
        - Feature is *stable*
        - Feature is *documented* 
        - Bugs and issues can be raised on xen-devel@ and will be
          handled by the developers/maintainers/committers within the 
          community.
        - Regressions and blockers in a complete feature do normally
          block new releases. 
        - Security issues would be handled by the security team.

5.      Deprecated
        There are typically two scenarios when a feature would be 
        deprecated. 
5.1.    - If the feature is not relevant any more or has been replaced
          by a new implementation (e.g. xm vs. xl)
5.2     - If we have lost the capability to support a feature.
          For example when we have lost the capability to *maintain*
          the feature, because we do not have maintainers. In such cases
          raising the issue usually will lead to a resolution, if there
          is enough usage by vendors in the eco-system. 
        Features in any state can be deprecated.

        State Changes
        -------------        
[Note:  this assumes that we keep a document in the source tree which
        provides a snapshot of information akin a snapshot of
        http://wiki.xenproject.org/wiki/Xen_Project_Release_Features
        in the source tree. I am volunteering to maintain the wiki
        and initially populate the file based on existing information.
        Andy Cooper suggested he is willing to put together a template
        with some examples. See 
http://lists.xenproject.org/archives/html/xen-devel/2015-06/msg01369.html]

        The intention here is to require that the central file is modified
        with a patch that introduces a feature (state, feature title,
        short description) and that developers which add functionality
        modify accordingly. The release manager and other stake-holders
        such as the community may also propose changes during the release
        cycle (in particular towards the end). 

        Also, if a feature has been for too long on in an incomplete 
        state (e.g. Preview or Experimental) or some of the assumptions
        associated with a state do not hold any more, community members 
        may propose to downgrade a feature. Typically a discussion on
        the list would be expected before a patch to the file is proposed.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel