Re: [Xen-devel] [xen-unstable test] 123831: trouble: broken/fail/pass

[Ian Jackson <ian.jackson@xxxxxxxxxx>]

Juergen Gross writes ("Re: [Xen-devel] [xen-unstable test] 123831: trouble: 
broken/fail/pass"):
> The same host (italia1) that had the failed xtf test yesterday. The two
> failures are looking very similar to me.
> 
> Again the question: should we do a force push?

I think the final decision is up to you, but I would reason along
these lines:

The point of the push gate is to stop regressions making it into
master.  However, osstest cannot currently handle heisenbugs well, so
it punts: when something is identified as a heisenbug, it is not
considered a regression.


There is this failure in 123831
(3960f3a52346348e6b0306f65d19375612bd35b9, staging)

 test-xtf-amd64-amd64-5 <job status> broken
 test-xtf-amd64-amd64-5 4 host-install(4) broken pass in 123670

This is an infrastructure problem.  It means that that xtf test didn't
run.  However, each flight runs the same battery of tests on 5
different hosts; so identical tests were run on other hosts.  We're
just missing 20% of the XTF test host diversity we would have had.

So on that basis a force push is justified, because we can see that
the failure in 123831 does not really give any reason to suspect a
regression and the test coverage was only slightly reduced compared to
what was planned.


However, there are these failures in 123799
(06f542f8f2e446c01bd0edab51e9450af7f6e05b, master)

 test-armhf-armhf-xl-arndale
   5 host-ping-check-native   fail
   REGR. vs. 123323

This is very likely the known arndale bug and not a cause for concern.

 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm
   14 guest-saverestore.2 fail
   REGR. vs. 123323b

This failure is being discussed in email.  Obviously this is not a
regression from master, since it's *in* master.  But it might be a
release critical bug.

If it is a release critical bug then branching (and consequently
opening staging again) might entrench the bug, both by allowing
effort to go to "shiny new stuff", and by making it harder to fix as
staging diverges from staging-4.11.


So I think overall, I would say this justifies a force push but if the
reason for wanting a force push was to enable branching, there is an
actual decision to be made, which is a matter of judgement.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel