Re: [PATCH v2 0/3] Yocto Gitlab CI

[Bertrand Marquis <Bertrand.Marquis@xxxxxxx>]

Hi Stefano,

> On 14 Oct 2022, at 21:27, Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote:
> 
> On Wed, 12 Oct 2022, Bertrand Marquis wrote:
>> This patch series is a first attempt to check if we could use Yocto in
>> gitlab ci to build and run xen on qemu for arm, arm64 and x86.
>> 
>> The first patch is creating a container with all elements required to
>> build Yocto, a checkout of the yocto layers required and an helper
>> script to build and run xen on qemu with yocto.
>> 
>> The second patch is creating containers with a first build of yocto done
>> so that susbsequent build with those containers would only rebuild what
>> was changed and take the rest from the cache.
>> 
>> The third patch is adding a way to easily clean locally created
>> containers.
>> 
>> This is is mainly for discussion and sharing as there are still some
>> issues/problem to solve:
>> - building the qemu* containers can take several hours depending on the
>>  network bandwith and computing power of the machine where those are
>>  created
>> - produced containers containing the cache have a size between 8 and
>>  12GB depending on the architecture. We might need to store the build
>>  cache somewhere else to reduce the size. If we choose to have one
>>  single image, the needed size is around 20GB and we need up to 40GB
>>  during the build, which is why I splitted them.
>> - during the build and run, we use a bit more then 20GB of disk which is
>>  over the allowed size in gitlab
>> 
>> Once all problems passed, this can be used to build and run dom0 on qemu
>> with a modified Xen on the 3 archs in less than 10 minutes.
> 
> The build still doesn't work for me. I found the reason:
> 
>  create archive failed: cpio: write failed - Cannot allocate memory
> 
> It is a "silly" out of memory error. I tried to solve the problem by
> adding:
> 
>  export RPM_BUILD_NCPUS=8
> 
> at the beginning of build-yocto.sh but it didn't work. I realize that
> this error might be considered a workstation configuration error at my
> end but I cannot find a way past it. Any suggestions?


Can you give me more details on when this is happening ? Ie the full logs.

Can you try to apply the following:
--- a/automation/build/yocto/build-yocto.sh
+++ b/automation/build/yocto/build-yocto.sh
@@ -107,6 +107,9 @@ IMAGE_INSTALL:append:pn-xen-image-minimal = " 
ssh-pregen-hostkeys"
 # Save some disk space
 INHERIT += "rm_work"

+# Reduce number of jobs
+BB_NUMBER_THREADS=2
+
 EOF

     if [ "${do_localsrc}" = "y" ]; then

This should reduce the number of parallel jobs during Yocto build.

Cheers
Bertrand