Re: [PATCH 3/3] automation: add a QEMU based x86_64 Dom0/DomU test

[Anthony PERARD <anthony.perard@xxxxxxxxxx>]

On Mon, Oct 25, 2021 at 06:33:53PM -0700, Stefano Stabellini wrote:
> On Mon, 25 Oct 2021, Anthony PERARD wrote:
> > There is something I'm missing, how is it a problem to have a container
> > that is a bit bigger? What sort of problem could we have to deal with?
> 
> It takes time to clone the container in the gitlab-ci, the bigger the
> container the more time it takes. It is fetched over the network. Now we
> are fetching qemu (as part of the container) 10 times during the build
> although it is not needed.

I guess the issue would be more like we don't do enough caching with our
gitlab runners. So package installation is just a workaround.

> But we do have a severe problem at the moment with external sources: our
> "git clones" keep failing during the build on x86. That is definitely
> something worth improving (see my other email thread on the subject) and
> it is the main problem affecting gitlab-ci at the moment, I keep having
> to restart jobs almost daily to get the overall pipeline to "pass".
> 
> If you have any ideas on how to stop fetching things using "git" from
> external repositories in gitlab-ci that would be fantastic :-)
> The only thing I could think of to "fix it" is moving all external repos
> to gitlab repositories mirrors.

I don't think that would work, I've seen the initial clone/fetch of a
job fail as well, so from gitlab. If we could have a cache of those
external resources closer to the runners, that would be better.

> > > I am not entirely sure what is the best solution overall, but for this
> > > series at this stage I would prefer to keep the same strategy used for
> > > the ARM tests (i.e. reuse the debian unstable build container and
> > > apt-get the few missing packages.) If we do change the way we do it, I
> > > would rather change both x86 and ARM at the same time.
> > 
> > I'm pretty sure the best strategy would be to do as little as possible
> > during a job, download as little as possible and possibly cache as much
> > as possible and do as much as possible ahead of time. Feel free to
> > change the Arm test, but I don't think it is necessary to change the Arm
> > test at the same time as introducing an x86 test.
> 
> I agree.
> 
> At the same time it would be nice to follow the same strategy between
> x86 and ARM going forward: if one optimization is made for one, it is
> also made for the other.

Probably better, yes.

Thanks,

-- 
Anthony PERARD