Re: [Xen-devel] [Notes for xen summit 2018 design session] Process changes: is the 6 monthly release Cadence too short, Security Process, ...

[Juergen Gross <jgross@xxxxxxxx>]

On 05/07/18 09:53, Wei Liu wrote:
> On Wed, Jul 04, 2018 at 03:26:16PM +0000, George Dunlap wrote:
>>
>>
>>> On Jul 3, 2018, at 11:07 AM, Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
>>>
>>> On Mon, Jul 02, 2018 at 06:03:39PM +0000, Lars Kurth wrote:
>>>> We then had a discussion around why the positive benefits didn't 
>>>> materialize:
>>>> * Andrew and a few other believe that the model isn't broken, but that the 
>>>> issue is with how we 
>>>>   develop. In other words, moving to a 9 months model will *not* fix the 
>>>> underlying issues, but 
>>>>   merely provide an incentive not to fix them.
>>>> * Issues highlighted were:
>>>>   * 2-3 months stabilizing period is too long
>>>
>>> I think one of the goals with the 6 month release cycle was to shrink
>>> the stabilizing period, but it didn't turn that way, and the
>>> stabilizing period is quite similar with a 6 or a 9 month release
>>> cycle.
>>
>> Right, and I think this was something that wasn’t quite captured in Lars’ 
>> summary.
>>
>> Everyone agreed:
>> 1. The expectation was that a shorter release cycle would lead to shorter 
>> stabilization periods
>> 2. This has not turned out to be the case, which means
>> 3 At the moment, our “time doing development” to “time fixing bugs for a 
>> release” ratio is far too low.
>>
>> One option to fix #3 is to go back to a 9-month cycle (or even a
>> 12-month cycle), which would increase the “development” part of the
>> equation.
> 
> You get more changes in, you also get more bugs.  Assuming bugs are
> introduced at a constant rate in relation to changes, moving back to 9
> months won't help.

Uuh, why not? It isn't as if no bugs are found and corrected in the
development period. As long as the development period is longer than
the average time between OSSTEST pushs the stabilization period should
be roghly constant. So a longer development period will result in a
better ratio development / stabilization.

> At least in my experience, a majority of time during the freeze is spent
> on *waiting*. Waiting for osstest to turn around, waiting for security
> issues to become public. Moving to 9 months won't change those factors.

But waiting isn't a factor, it is a constant, assuming the number of
unresolved bugs at the end of the development period is roughly the
same. And that will be the case if we don't:

a) pay no attention to OSSTEST results during development (i.e. trying
   to get pushs as often as possible)

b) rush most series in at the end of the development period

> A typical bug would need five working days (one week) to fix.
> 
> 1. Someone report or osstest reports a bug. (Day 1)
> 2. Someone analyses it and writes a patch. (Day 2)
> 3. Someone reviews it. (Day 2 or 3).
> 4. Someone commits it. (Day 3 or 4).
> 5. Osstest produces test results (Day 3 to 5).
> 
> For a simple bug, we might finish 1-4 in one day. But we still need to
> allow for at least two days to get a push.

In case of OSSTEST often enough multiple bugs are reported in parallel
and can (and should) be processed concurrently.

> In reality, a number of factors actually prolong getting things fixed
> (in the sense that patches are pushed to master): 1. bug fixes are
> incomplete; 2. hardware issues in test system; 3. other random hiccups.
> Should any of these happens, another 2 to 3 days is required to get
> patches pushed.

4. rarely triggering bugs which have been ignored before re-surface
and cause delays


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel