Re: [Xen-devel] Xen project CI systems and committer workflow

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Thu, Apr 18, 2019 at 10:54:05AM -0700, Stefano Stabellini wrote:
> On Thu, 18 Apr 2019, Wei Liu wrote:
> > Hi all
> > 
> > We now have Gitlab CI as a complementary system to Osstest and have planned 
> > to
> > add bots. It's high time we think about how we integrate them and how it may
> > improve our workflow.
> > 
> > ## Requirements
> > 
> > 1. We want to have light weight build tests before a patch series is 
> > reviewed
> > or committed.
> > 
> > 2. We don't want to push broken patches to central repository such that
> > everyone is blocked.
> > 
> > 3. We don't want to significantly change committer workflow.
> > 
> > Requirement 2 means that our current branching model will need to be 
> > changed.
> > Details will follow.
> > 
> > ## Overview of automation systems
> > 
> > - Gitlab CI
> > 
> >     It is capable of running build tests with relatively short turnaround 
> > time.
> > 
> > - Bots like Patchew and Patchwork
> > 
> >     They are able to slurp patches from xen-devel and maybe run customised
> >     scripts on those patches.
> > 
> > - Osstest
> > 
> >     Pushgate, integration test system. It runs more sophisticated tests
> >     compared to Gitlab CI. Turnaround time is long.
> >     
> > ## New world
> > 
> > There will be an unified endpoint for committers and bots. Committers and 
> > bots
> > will have their git trees. Patches are committed to those trees. An 
> > automated
> > system will get patches from those trees and trigger CI runs.
> > 
> > The system will pick up commits from one of the trees, merge them with 
> > master
> > and send the merge to CI systems.
> > 
> > For bots, only Gitlab CI build tests will run. Results are sent back to
> > xen-devel.
> > 
> > For committers' trees, at first Gitlab CI build tests are run, if the 
> > result is
> > successful, the merge commit is submitted to osstest. If osstest deems the
> > merge is good, the merge is pushed (published) to xen.git, otherwise the 
> > merge
> > is discarded. Test results are sent back to xen-devel.
> > 
> > With this system, all published commits should have already passed Gitlab CI
> > and osstest tests.
> > 
> > ## Gaps
> > 
> > The component / system which merges trees and drives CI systems is missing.
> > 
> > Patchew and Patchwork don't do all the things we care about yet.
> 
> I understand the general principle and I am on board. But I missing some
> details.
> 
> Is the idea that each committer would have his own git tree and branch
> with a specific name which would be picked up automatically by the
> CI-loop daily? In the style of linux-next? If it passed, it is given to
> OSSTest automatically and eventually committed?

Branch / tree -- whichever works best.

If the target under test passes, it will eventually be committed to
xen.git; otherwise the target is discarded and submitters / committers
get notified what failed.

> 
> Or is it the idea that each committer would have a way to manually
> submit a branch to the CI-loop for testing when he wished to do so?  In
> the style of github pull-requests-based CI-loops? If this is the case,
> then we just need a way to trigger the CI-loops on a given branch?
> 

This is the same as above. How the system works internally has no
visible effect on committer workflow.

In both cases, you just need to follow instructions to commit to a tree
/ branch / repository.

> In any case, my only feedback would be to also allow CI-loops on
> branches that are *not* meant to be committed automatically if tests
> pass. So that we can apply a complex series to a branch and see if tests
> pass before doing reviews or running experiments.

Understood.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel