Re: Xen 4.18 release: Reminder about last posting date

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Wed, 9 Aug 2023, Andrew Cooper wrote:
> On 09/08/2023 11:34 pm, Stefano Stabellini wrote:
> > On Wed, 9 Aug 2023, Andrew Cooper wrote:
> >> On 09/08/2023 10:43 pm, Stefano Stabellini wrote:
> >>> On Wed, 9 Aug 2023, Andrew Cooper wrote:
> >>>> On 07/08/2023 2:24 am, Henry Wang wrote:
> >>>>> Hi everyone,
> >>>>>
> >>>>> Following the release schedule discussion in in April, I am sending 
> >>>>> this email
> >>>>> to remind that according to the release schedule [1], August 11 (this 
> >>>>> Friday)
> >>>>> will be the last posting date, when patches adding new features are 
> >>>>> expected
> >>>>> to be posted to the mailing list by this date.
> >>>>>
> >>>>> Also, note that we currently have 1 release blocker [2] which might need
> >>>>> some attention.
> >>>>>
> >>>>> [1] 
> >>>>> https://lore.kernel.org/xen-devel/AS8PR08MB79919F9CE0B2BF80E7103FB592609@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/
> >>>>> [2] https://gitlab.com/xen-project/xen/-/issues/114
> >>>> Off the top of my head.
> >>>>
> >>>> There are still unaddressed Gitlab bugs from the Eclair integration
> >>> The bug you managed to find it is now fixed (commit e55146071de9). I am
> >>> all for fixing Gitlab bugs so let me know if you find anything else! I
> >>> am not aware of any other issue with Eclair at the moment.
> >> I meant the one where Eclair is still running on `smoke` and twiddling
> >> its thumbs for 1h doing so each time OSSTest says yes to a push.  It
> >> will be a missing 'exclude' somewhere, but I haven't hand enough time to
> >> look.
> > Ah, yes, got it. I would call it "Eclair lagging behind". I am happy for
> > this to be a blocker.
> 
> Something else for the list is to see about trimming down the testing
> we're doing.
> 
> I had to cancel 7 pipelines (mostly patchew) in order to get a gitlab
> run on a late-breaking tweak on the security content yesterday.
> 
> Take
> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/959800176
> as an example.  Nearly 2h to run, and queued for 2h before that waiting
> to start.  This is not a rare occurrence right now.

Yeah...

So, an easy fix that would solve the problem for the rest of the tests
but not Eclair would be to reduce the number of tests. It is easy
because we can make them conditional on a CI/CD variable and *not*
define the variable in patchew. That way patchew gets a reduced set of
tests. I think that would be fine.

Another option we can also pursue in parallel is to throw more hardware
at the problem. I am not sure where we are on reusing Colo hardware for
this.

Eclair is a different issue because we cannot increase the hardware and
I don't think we want to reduce the set of rules to be scanned. So the
only idea I can come up with is to limit the number of Eclair jobs in
patchew to 1-2 per day. However it would be pretty random which ones are
getting executed. Or find a wat to prioritize non-patchew jobs? If you
or anyone else has better idea feel free to suggest.


> >>>> and other Gitlab bugs (use of unstable containers) which I'd unwilling
> >>>> to let 4.18 be released with, given the pain we've had on the stable
> >>>> trees trying to keep CI working.
> >>> That is fair enough. To make this more concrete and easier to track, the
> >>> following would need to be changed to using stable containers:
> >>>
> >>> - .qemu-arm64
> >>> - .qemu-arm32
> >>> (I am not counting .qemu-riscv64)
> >>>
> >>> Andrew, is that what you meant? Am I missing anything?
> >> Every debian unstable container, and the other containers (OpenSUSE)
> >> which are using an non-specific upstream version.  Upstreams which
> >> really are rolling distros (Arch, Tumbleweed) need to be made non-fatal.
> > I was more asking whether for 4.18 you want to fix only test.yaml or
> > also build.yaml. (Typically it is test.yaml that causes most issues
> > with rolling containers.)
> 
> Everything.  Because both bite equally hard in stable trees.
> 
> It's not hard to do - just needs some time.
 
OK