Re: [XEN PATCH v1 0/3] automation: add x86_64 test (linux argo)

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Thu, 17 Oct 2024, Andrew Cooper wrote:
> On 17/10/2024 6:18 pm, victorm.lira@xxxxxxx wrote:
> > From: Victor Lira <victorm.lira@xxxxxxx>
> >
> > Add x86_64 hardware test that creates a Xen Argo communication
> > connection between two PVH domains.
> >
> > To accomplish this, add new build artifacts for Linux and Argo, and
> > update the xilinx x86_64 test script.
> >
> > Victor Lira (3):
> >   automation: add linux 6.6.56 artifact
> >   automation: add linux argo test artifacts
> >   automation: add x86_64 test (linux argo)
> >
> >  automation/gitlab-ci/build.yaml               | 34 +++++++
> >  automation/gitlab-ci/test.yaml                | 10 ++
> >  .../scripts/xilinx-smoke-dom0-x86_64.sh       | 76 ++++++++++-----
> >  .../tests-artifacts/argo/6.6.56.dockerfile    | 95 +++++++++++++++++++
> >  .../tests-artifacts/kernel/6.6.56.dockerfile  | 54 +++++++++++
> >  5 files changed, 244 insertions(+), 25 deletions(-)
> >  create mode 100644 automation/tests-artifacts/argo/6.6.56.dockerfile
> >  create mode 100644 automation/tests-artifacts/kernel/6.6.56.dockerfile
> 
> I'm afraid that we need to stop doing this test-artefacts like this.
> 
> The *-export pattern is nonsense (wasting lots of runner network&disk
> and time just to copy data back into Github's artefact store), and the
> main reason why we're at ~2T of data total for Xen-project.
> 
> We need a project wide expires_in: setting, which will prune all of our
> not-most-recent content, and probably make most of that 2T disappear.

I looked at this for an hour and couldn't find a way to do it
unfortunately. expires_in can only be changed if you are an Admin of the
Gitlab instance, meaning an Admin of gitlab.com.

However, there is a setting in each repository (not one global at the
gitlab.com/xen-project level) called "Keep artifacts from most recent
successful jobs" that can be disabled. I did it manually for my repo.

The big offender is patchew. I went and disabled "Keep artifacts from
most recent successful jobs" but I didn't see any immediate changes in
storage utilization. But it is expected to take time to propagate.


> But, the test jobs stating what artefacts they want are perfectly
> capable of pulling artefacts from somewhere other than an earlier nop
> build job, meaning that we could get away with having one artefact
> shared by everything, not one artefact copied per user per pipeline into
> the artefact store.
> 
> 
> I was thinking of experimenting with a separate top-level repo that does
> nothing but has a few manual runs to populate artefacts, and having the
> Xen tests pull artefacts from here rather than from earlier build jobs.
> 
> But, I've not had a chance to play yet, so I don't know for sure if this
> is a workable direction.
>
> Alternatively, if anyone has a better idea, I'm all ears.  But, it is
> very clear that the *-export pattern is a giant bodge and there are
> better ways.
> 
> Other notes.  Your dockerfiles use syntax:1 but not heredocs, and
> they're also root containers.
> 
> Please follow the pattern in
> https://lore.kernel.org/xen-devel/178560672106e37648304f5e597cc0b16c8db6db.1729170005.git.javi.merino@xxxxxxxxx/T/#u
> or one of plenty other containers I've already converted in the past few
> months.
> 
> Please do not add a new build job just to get a new minor variation of
> Xen.  Just enable ARGO in general Xen build.

+1