Re: [Xen-devel] [PATCH 0/3 for 4.8] docs: feature documents for the schedulers

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 13/10/2016 22:06, Stefano Stabellini wrote:
>
>>>>> Credit2 **Supperted**, instead of experimental.
>>>> Supperted?  That's like supported right? ;p
>>>>
>>>>
>>>> It is fine for you to propose that a feature should be upgraded to
>>>> supported, and this is probably the best way to formally do so.
>>>>
>>>> However, final agreement of a feature becoming supported should
>>> include
>>>> input from the security team. (At the end of the day, it is us with
>>>> extra work if the feature isn't up to scratch.)
>>> Is this new? If so, should we formalize the change in process somewhere
>>> (patch to governance, etc.)?
>> This came about when we had .. XSA7? Which was the tmem one and came with 
>> the idea that anything that moves to Supported has to pass the security 
>> audit pass.

Off the top of my head, XSA-7 is the Intel silicon sysret bug.  ITYM
XSA-15, but your point still stands.

> Make sense. In that case we should definitely write it down somewhere.

My entire purpose with introducing feature docs to start with was to try
and do for feature what we now do very well for
docs/misc/xen-command-line.markdown  i.e. to keep the documentation
easy, and up to date.

This does involve the committers, maintainers and reviewers being
mindful to prompt for an update when necessary.  However, it is very
evident from the replies on-list these days that it has become second
nature to a lot of people to either patch the document in the first
place, or prompt others to do the same in submitted patches.

Overall, this is a very good position to be in, for the community as a
whole.  I am certainly hoping that in another couple of years, we as a
community have exactly the same "second nature" mindset when it comes to
keeping the feature docs up to date.

> I like the idea of keeping these info on pandoc on a git repo, like Lars
> did with the governance.

I should hasten to add that perhaps picking on the security team in
isolation was a poor move on my part, for which I apologise. There are
multiple involved parties with different interests, all of which need
attending to.

The feature submitter (should) have a vested interest in getting code
submitted (even as experimental/tech preview to start with), so that
others can trial/debug/extend the feature.  Even having code upstream
but disabled-by-default in Kconfig is a far better position for others
to start from, than to be provided with instructions saying "there was a
patch series several months ago on a mailing list".

The reviewers/maintainers (should) have a vested interest in keeping the
overall quality up.  Due to their appointment within the community, they
have demonstrated knowledge and expertise in their respective areas, and
the overall project relies on them to ask questions such as "How does $X
work with $Y?" or "Have you considered $Z as an alternative?", and to
come to a fair and unbiased opinion to the quality and worthiness of a
feature.

The release manager has conflicted vested interests.  On the one hand,
getting more features in a release is better on paper and in the press,
but can certainly backfire when features aren't up to scratch. 
Ultimately, if features in a release are substandard, the downstreams
and end users will be the ones to bear the pain.  It is in the release
managers best interest to ensure that the features which are finally
accepted are actually ready.

Nothing is ever perfect, but a whole lot of stuff is perfectly good in
common case, and useful for people in general.  This is why the feature
doc template specifically has sections for know issues to be fixed, and
areas for future improvement.  Any documentation (so long as it isn't
factually incorrect) is better than nothing.

There seems to be general consensus that a status of "Supported" means
"with security support", and I agree with this assessment.  Ultimately,
the security team is accountable to whatever the project as a whole
declares "Supported", but as the security team are all commiters and
maintainer there is a large overlap of responsibility.  OTOH, the
security team are also responsible for managing the fallout when things
go wrong.

XSAs are bad publicity, and are a problem for downstreams (remember that
we have plenty of downstreams who measure quantity of servers in units
of a datacenter).  Another problem we as a community have is that no
support status is written down; there is a 13ish? year backlog in this
regard.  Writing details down in feature docs is intended to get
everyone on the same, un-ambiguous, page.

We also have an unfortunate habit of new features appearing, being
accepted, and (intentionally or unintentionally) available to guests. 
Fastforward a bit, and it is discovered that said feature resembles a
sieve in terms of security holes, and the common recourse is to publish
an XSA stating "turn it off and don't use it with untrusted guests", or
"here is a patch to actually turn it off, and still don't use it with
untrusted guests", because we really can't re-engineer a feature in a
security patch.  (Sorry to pick on TMEM here, but it is now 4 years of
development later and it still has a lot of development left to go. 
Reworking features in the light of a security hole can be very
complicated, and lots of work.)

There should be a high barrier to "Supported" status, because the cost
of getting it wrong is equally high.  However, there are perfectly
legitimate intermediate stages such as "Supported in these limited set
of circumstances".  A number of features are not plausibly for use in
production environments, but otherwise function fine for
development/investigatory purposes.  In these cases, something like "no
security support, but believed to be working fine" might be appropriate.

Having said all of this, I don't wish to deter feature submitters from
striving for "Supported".  Attaining "Supported" should be a accolade
that a feature is good from a technical point of view, with no important
missing parts.  I will certainly continue to strive for my features to
fit into this category, and I hope that the rest of the community will
hold my code to the same high standard that I hold others to.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel