Re: [Xen-devel] Xen release cycle revisited

[George Dunlap <dunlapg@xxxxxxxxx>]

On Thu, Dec 14, 2017 at 2:11 PM, Jan Beulich <JBeulich@xxxxxxxx> wrote:
>>>> On 14.12.17 at 08:56, <jgross@xxxxxxxx> wrote:
>> 4. Should we try harder to negotiate embargo dates of security issues to
>>    match the (targeted) release dates?
>
> Personally I don't think embargo dates should be made match
> release dates; if anything, the other way around. Holding back
> security issues is just bad (and I'm saying that being well aware
> that we hold ones back longer than is actually necessary, for
> reasons which I don't want to discuss here).

We had a discussion at our team meeting last Thursday, and I think
fundamentally it takes at least week of testing to be reasonably
confident that we don't have unknown "howlers" (i.e., embarrassing
bugs in basic functionality).

Which means we have three basic options:

1. Release with no known bugs, and confident that there are no
"howlers".  This means we have to be willing to slip the release by
one week, every time a new bug is discovered, *for as long as it
takes*.

2. Release at a specific date, confident that there are no "howlers".
This means shipping with known bugs (i.e., any bugs for which patches
have been submitted 1 week before the target date aren't included).

3. Releasing with no known bugs, at a specific date.  This means
taking the risk that there is basic functionality that is completely
broken, because we didn't test it properly.

It would be nice if we didn't have to choose between these three
things (specific date, all known bugs fixed, confident of no
'howlers'); but that's not the universe we live in, and attempting to
deny that fact led us this time to de facto choose #3 (i.e., we
decided we were going to ship on 12 December, with all the known bugs
fixed, but not give enough time for testing to see if we had
introduced any 'howlers').  (NB that I'm not blaming anybody in
particular here -- Julien discussed his plan and it was agree upon by
several people including myself.)

This is a slight simplification.  For instance, we can always start
with #1 and switch to #2 or #3 at any time: i.e., allow the release
date to slip a few times, and then fix the date and release either
with known bugs or release insufficiently tested.  (That's essentially
what we did this time.)  We could also decide to ship with "minor"
bugs, but slip the release indefinitely for "major" bugs.

I don't have a strong preference between #1 and #2, but I think we
definitely want to avoid #3.  But to do that we (or perhaps the
Release Manager) needs to decide which option to choose (slip
indefinitely or ship with known bugs) and stick with it.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel