Re: [PATCH v2 0/3] Yocto Gitlab CI

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Mon, 17 Oct 2022, Bertrand Marquis wrote:
> Hi Stefano,
> 
> > On 14 Oct 2022, at 21:27, Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote:
> > 
> > On Wed, 12 Oct 2022, Bertrand Marquis wrote:
> >> This patch series is a first attempt to check if we could use Yocto in
> >> gitlab ci to build and run xen on qemu for arm, arm64 and x86.
> >> 
> >> The first patch is creating a container with all elements required to
> >> build Yocto, a checkout of the yocto layers required and an helper
> >> script to build and run xen on qemu with yocto.
> >> 
> >> The second patch is creating containers with a first build of yocto done
> >> so that susbsequent build with those containers would only rebuild what
> >> was changed and take the rest from the cache.
> >> 
> >> The third patch is adding a way to easily clean locally created
> >> containers.
> >> 
> >> This is is mainly for discussion and sharing as there are still some
> >> issues/problem to solve:
> >> - building the qemu* containers can take several hours depending on the
> >>  network bandwith and computing power of the machine where those are
> >>  created
> >> - produced containers containing the cache have a size between 8 and
> >>  12GB depending on the architecture. We might need to store the build
> >>  cache somewhere else to reduce the size. If we choose to have one
> >>  single image, the needed size is around 20GB and we need up to 40GB
> >>  during the build, which is why I splitted them.
> >> - during the build and run, we use a bit more then 20GB of disk which is
> >>  over the allowed size in gitlab
> >> 
> >> Once all problems passed, this can be used to build and run dom0 on qemu
> >> with a modified Xen on the 3 archs in less than 10 minutes.
> > 
> > The build still doesn't work for me. I found the reason:
> > 
> >  create archive failed: cpio: write failed - Cannot allocate memory
> > 
> > It is a "silly" out of memory error. I tried to solve the problem by
> > adding:
> > 
> >  export RPM_BUILD_NCPUS=8
> > 
> > at the beginning of build-yocto.sh but it didn't work. I realize that
> > this error might be considered a workstation configuration error at my
> > end but I cannot find a way past it. Any suggestions?
> 
> 
> Can you give me more details on when this is happening ? Ie the full logs.
> 
> Can you try to apply the following:
> --- a/automation/build/yocto/build-yocto.sh
> +++ b/automation/build/yocto/build-yocto.sh
> @@ -107,6 +107,9 @@ IMAGE_INSTALL:append:pn-xen-image-minimal = " 
> ssh-pregen-hostkeys"
>  # Save some disk space
>  INHERIT += "rm_work"
> 
> +# Reduce number of jobs
> +BB_NUMBER_THREADS=2
> +
>  EOF
> 
>      if [ "${do_localsrc}" = "y" ]; then
> 
> This should reduce the number of parallel jobs during Yocto build.

It should be

BB_NUMBER_THREADS="2"

but that worked! Let me a couple of more tests.