[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Gitlab status on older branches (Inc some 4.18 blockers)
Hello, I've done various backports to staging-4.14 and later trying to improve the state of Gitlab testing. The good news is that 4.16 and 4.17 now pass. The bad news is that there are still bugs which need fixing, but lets start with the older branches. Also, I was forced to backport an update to SeaBIOS 1.16 to all branches in order to fix compile failures in build environments we supported at the time of each of these releases. I honestly don't know what we were failing to do testing wise back then, but whatever we missed ought to have been release blockers. 4.15: https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/834460832 Individual failure instances: 1) https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/4097232265 This is a -Werror=array-bounds failure in HVMLoader but the same job/container works in 4.16 and newer, and the underlying code is the same. There must be some change in the build environment, but I haven't worked out what yet. 2) https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/4097232266 This is a -Werror=array-bounds in iPXE. Probably needs an update like SeaBIOS too. 3) https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/4097232290 This is a Qemu upstream failure which I do vaguely recall. Bit it also means that Xen 4.15 had a dead-on-arrival version of qemu which call into question a number of our normal release activities. Probably the least-bad option is to backport the one fix relevant to this, because changing the version of qemu in the security-only trees is far riskier than changing one of the in-guest ROMs. 4) https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/4097232334 I have no idea what's going on here. If nothing else, we're failing to collect all the relevant log files from a build and that probably wants fixing and backporting. 5) https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/4097232324 This isn't so much about the failure, as the fact that the OpenSUSE Leap tests (which were replaced with tumbleweed in newer versions of Xen) probably want the same doing to them. And being marked as non-blocking. This is a failure somewhere in the middle of qemu. But, on top of all of ^, I discovered that we have a majority of tests being debian/unstable which, when we refreshed to fix the HTTPS issue, ended up retrofitting a newer-than-at-release-time build environment to the old trees. This has come up previously, and not addressed, so I'm now declaring it a blocker for 4.18. Only tests against a fixed disto version can be blocking; those against an unstable distro must be non-blocking, and most of the currently unstable things should be transformed into their stable alternative. For backports, we want to retrofit what debian/unstable was at the time of release, rather than what it currently is. Furthermore, the fixed distros we currently test in staging are old bordering on obsolete. Which is not a healthy position to be in as far as the 4.18 release goes. 4.14: https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/834461234 6) https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/4097236330 This is the only 4.14 failure I can see which isn't a duplicate of a 4.15 failure, but it is an OpenSUSE leap failure in qemu so perhaps related to #5 As a general note, we still have too much testing (and/or insufficient testing resource). It's very painful waiting 2h for each branch to complete. I'm very tempted to trim things down further on staging and backport the results. ~Andrew
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |