Re: [Xen-devel] [Notes for xen summit 2018 design session] Process changes: is the 6 monthly release Cadence too short, Security Process, ...

[Steven Haigh <netwiz@xxxxxxxxx>]

On Thursday, 5 July 2018 1:26:16 AM AEST George Dunlap wrote:
> > On Jul 3, 2018, at 11:07 AM, Roger Pau Monné <roger.pau@xxxxxxxxxx>
> > wrote:
> > 
> > On Mon, Jul 02, 2018 at 06:03:39PM +0000, Lars Kurth wrote:
> > 
> >> We then had a discussion around why the positive benefits didn't
> >> materialize:
 * Andrew and a few other believe that the model isn't
> >> broken, but that the issue is with how we>> 
> >>   develop. In other words, moving to a 9 months model will *not* fix the
> >>   underlying issues, but 
 merely provide an incentive not to fix them.
> >> 
> >> * Issues highlighted were:
> >> 
> >>   * 2-3 months stabilizing period is too long
> > 
> > 
> > I think one of the goals with the 6 month release cycle was to shrink
> > the stabilizing period, but it didn't turn that way, and the
> > stabilizing period is quite similar with a 6 or a 9 month release
> > cycle.
> 
> 
> Right, and I think this was something that wasn’t quite captured in Lars’
> summary.
 
> Everyone agreed:
> 1. The expectation was that a shorter release cycle would lead to shorter
> stabilization periods
 2. This has not turned out to be the case, which
> means
> 3 At the moment, our “time doing development” to “time fixing bugs for a
> release” ratio is far too low.
 
> One option to fix #3 is to go back to a 9-month cycle (or even a 12-month
> cycle), which would increase the “development” part of the equation.
 
> But Doug was advocating trying instead to attack the “time fixing bugs” part
> of the equation.  He said he was a big fan “continuous delivery” — of being
> *always* ready to release.  And I think there’s a fair amount of agreement
> that one of the reasons it takes so long to stabilize is that our testing
> isn’t reliably catching bugs for whatever reason.

On this point alone, release quickly, release often. The kernel for instance 
runs about 1 release per week on the stable branches - sometimes 2.

With regular process, releases should be easy to achieve - and that means 
automation, automation and more automation.

From my end as a 'consumer' of this process, a regular process makes it easy 
for me to rebase work based on a regular release. In fact, it usually takes me 
less than 30 minutes to update, compile, package and release kernel builds - 
and all of that is compile time. I check kernel.org every 6 hours, and fire 
off automated builds as needed. The reality is, I could easily make this 
hourly and reduce time from release to package to less than an hour - but at 
that point is it worth it? :)

The longest delay on getting this to the clients is the time it takes for non-
local mirrors to catch up.
 
> So a fair amount of the discussion was about what it would look like, and
> what it would take, to make it such that almost any push from osstest (or
> whatever testing infrasctructure we went with) could reasonably be
> released, and would have a very low expectation of having extraneous bugs.

The key here is testing. If we started at the release end of the tree, 
something as simple as a git tag triggers a tarball export of the tree 
versioned as per the tag may well be a huge step forward in the release.

Then time can be taken after this in tweaking the testing part as we find 
things that become obvious through the rapid release process.

> I seem to recall saying that even if we agreed that moving to continuous
> delivery was a goal we wanted to pursue, we would still be several years
> away from achieving anything like it; and so in the mean time, it would
> probably make sense to move back to a 9-month cycle while we attack the
> problem.

Honestly, as a packager, I don't see ground-breaking changes in any version of 
Xen that is currently released. Most are optimisations like the new PVH code.

My point here is that really, are there enough major features that would break 
everything to require a new major release every 9 months?

Would 12 months for major features be more suitable and keep smaller 
refinements / additions as point releases?

With a solid base via testing, there's no real reason why a weekly (or daily) 
release wouldn't be technically feasible - apart from not having enough 
changes to justify it ;)

-- 
Steven Haigh

📧 netwiz@xxxxxxxxx       💻 https://www.crc.id.au
📞 +61 (3) 9001 6090    📱 0412 935 897

Attachment: signature.asc
Description: This is a digitally signed message part.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel