[Xen-devel] arm64, laxton[01] (was Re: [xen-unstable-smoke test] 133030: trouble: blocked/broken/pass)

[Ian Jackson <ian.jackson@xxxxxxxxxx>]

Ian Jackson writes ("arm64, laxton[01] (was Re: [Xen-devel] [xen-unstable-smoke 
test] 133030: trouble: blocked/broken/pass)"):
> I have checked and there was no update to the osstest code.  I don't
> think there have been updates to the infrastructure config but I
> haven't searched all the infrastructure boxes etckeepers etc.

I have now checked the etckeeper and dpkg logs on the osstest (test
controller, tftp server) and infra (dhcp and dns server etc.) VMs and
there were no changes that could have caused this.

Credativ have checked the boot files being used against the backups
and they are file.

> So something has caused both machines to fail simultaneously.
> 
> Possible explanations to me seem to be:
> 
>   * Some kind of common physical cause (power surge corrupting the
>     firmware or something)
> 
>   * Some kind of common nonphysical cause external to the hosts or the
>     tests: bad files on the infrastructure hosts; a change to the
>     behaviour of the bootp/tftp servers; a new kind of broadcast
>     network packet (perhaps from other tests) which causes the laxton
>     firmware to malfunction; etc.
> 
>   * The new linux-4.9 kernel does something which has the effect of
>     often (but not always) corrupting the laxtons' firmware.
> 
>   * A firmware bug triggered by the passage of time (eg
>     clock-dependent)
> 
>   * Misunderstanding by me in my analysis of what ingredients are used
>     by the host installation failures.
> 
>   * Some other common-mode failure that I haven't thought of.
> 
> In the meantime I have unblessed the laxtons to avoid osstest
> repeatedly power cycling them to try to get them to work.  FTR this
> will still not allow the push gate to pass.

I don't have a good plan about what to do next.  I guess one thing we
could do would be to ask Yogesh to reflash the firmware on laxton0 and
see if that helps.

I think this issue is likely to be a protracted problem.  Any helpful
suggestions from people with more experience of these particular boxes
would be very welcome...

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel