Re: [Xen-devel] Xen project CI systems and committer workflow

[George Dunlap <george.dunlap@xxxxxxxxxx>]

On 4/25/19 1:53 PM, Julien Grall wrote:
> Hi Jan,
> 
> On 25/04/2019 13:37, Jan Beulich wrote:
>>>>> On 18.04.19 at 16:02, <ian.jackson@xxxxxxxxxx> wrote:
>>> Andrew Cooper writes ("Re: Xen project CI systems and committer
>>> workflow"):
>>>> While everything presented here is fine to do as a matter of policy,
>>>> the
>>>> committers still need to retain the ability to actually push
>>>> directly to
>>>> the staging branches on xen.git
>>>
>>> Why ?
>>>
>>> In particular, this seems to presuppose that these "staging" branches
>>> continue to exist.
>>>
>>> One of our strange practices is that we push things to a non-rewinding
>>> branch before they have been tested.  Now, OK, we have been doing that
>>> for over a decade - since before people really invented the modern
>>> concept of "CI" - but that doesn't make it sensible.
>>
>> Now the question of course is - why do you consider this practice
>> "strange"? And what do you mean by "before they have been
>> tested"? Any submitter should have tested their patches. Yes, not
>> everyone tests all possible (basic) configurations, but we spot
>> such issues in review more often than they slip through, I think.
> 
> It is fairly common to see breakage on Arm because changes was only
> compiled on x86. The most common argument is "I don't have setup to
> build Arm".
> 
> The CI would allow us to test on every arch, different variation of
> .config (and possibly compiler).
> 
>> And I don't expect CI will come anywhere near testing all
>> _possible_ configurations, i.e. the risk of subtle breakage won't
>> get eliminated altogether anyway.
> 
> Nothing is fully bullet-proof, but this is an improvement over manual
> testing on most of the build configuration today. The CI actually proved
> the usefulness by catching build bug in random .config the last few months.

Furthermore, the long-term goal is to extend the boundary of things that
we test, such that what gets tested is far wider than it would even be
practical for submitters to test.  (This was partially the goal of the
SUPPORT.md, to have a place to enumerate testing status; and also
partially the goal of the osstest stretch upgrade, to enable osstest to
provide gcov reports to highlight testing deficiencies.)

Computers are good at boring, repetitive tasks like extensive testing;
humans are bad at it.  There's no reason to make humans do work that a
computer can do better.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel