Re: [PATCH for-4.19 1/4] CI: Formalise the use of heredocs

[Andrew Cooper <amc96@xxxxxxxx>]

On 03/07/2024 3:34 pm, Roger Pau Monné wrote:
> On Wed, Jul 03, 2024 at 03:19:59PM +0100, Andrew Cooper wrote:
>> Commit b5739330d7f4 introduced the use of heredocs in the jessie/stretch
>> dockerfiles.
>>
>> It turns out this was introduced by BuildKit in 2018 along with a
>> standardisation of Dockerfile syntax, and has subsequently been adopted by 
>> the
>> docker community.
>>
>> Annotate all dockerfiles with a statement of the syntax in use, and extend
>> README.md details including how to activate BuildKit when it's available but
>> off by default.
>>
>> This allows the containers to be rebuilt following commit a0e29b316363 ("CI:
>> Drop glibc-i386 from the build containers").
>>
>> Fixes: b5739330d7f4 ("automation: fix jessie/stretch images to use 
>> archive.debian.org apt repos")
>> Signed-off-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
> Reviewed-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>

Thanks.

>
>> ---
>> CC: Anthony PERARD <anthony.perard@xxxxxxxxxx>
>> CC: Stefano Stabellini <sstabellini@xxxxxxxxxx>
>> CC: Doug Goldstein <cardoe@xxxxxxxxxx>
>> CC: Roger Pau Monné <roger.pau@xxxxxxxxxx>
>> CC: Oleksii Kurochko <oleksii.kurochko@xxxxxxxxx>
>> ---
>>  automation/build/README.md                             | 10 +++++++++-
>>  automation/build/alpine/3.18-arm64v8.dockerfile        |  1 +
>>  automation/build/alpine/3.18.dockerfile                |  1 +
>>  automation/build/archlinux/current-riscv64.dockerfile  |  1 +
>>  automation/build/archlinux/current.dockerfile          |  1 +
>>  automation/build/centos/7.dockerfile                   |  1 +
>>  .../build/debian/bookworm-arm64v8-arm32-gcc.dockerfile |  1 +
>>  automation/build/debian/bookworm-arm64v8.dockerfile    |  1 +
>>  automation/build/debian/bookworm-cppcheck.dockerfile   |  1 +
>>  automation/build/debian/bookworm-i386.dockerfile       |  1 +
>>  automation/build/debian/bookworm.dockerfile            |  1 +
>>  automation/build/debian/bullseye-ppc64le.dockerfile    |  1 +
>>  automation/build/debian/buster-gcc-ibt.dockerfile      |  1 +
>>  automation/build/debian/jessie-i386.dockerfile         |  1 +
>>  automation/build/debian/jessie.dockerfile              |  1 +
>>  automation/build/debian/stretch-i386.dockerfile        |  1 +
>>  automation/build/debian/stretch.dockerfile             |  1 +
>>  automation/build/fedora/29.dockerfile                  |  1 +
>>  automation/build/suse/opensuse-leap.dockerfile         |  1 +
>>  automation/build/suse/opensuse-tumbleweed.dockerfile   |  1 +
>>  automation/build/ubuntu/bionic.dockerfile              |  1 +
>>  automation/build/ubuntu/focal.dockerfile               |  1 +
>>  automation/build/ubuntu/trusty.dockerfile              |  1 +
>>  automation/build/ubuntu/xenial-xilinx.dockerfile       |  1 +
>>  automation/build/ubuntu/xenial.dockerfile              |  1 +
>>  automation/build/yocto/yocto.dockerfile.in             |  2 ++
>>  .../tests-artifacts/alpine/3.18-arm64v8.dockerfile     |  1 +
>>  automation/tests-artifacts/alpine/3.18.dockerfile      |  1 +
>>  .../tests-artifacts/kernel/5.19-arm64v8.dockerfile     |  1 +
>>  automation/tests-artifacts/kernel/6.1.19.dockerfile    |  1 +
>>  .../qemu-system-aarch64/6.0.0-arm64v8.dockerfile       |  1 +
>>  .../qemu-system-ppc64/8.1.0-ppc64.dockerfile           |  1 +
>>  32 files changed, 41 insertions(+), 1 deletion(-)
>>
>> diff --git a/automation/build/README.md b/automation/build/README.md
>> index 1c040533fdd8..12a2b4af1807 100644
>> --- a/automation/build/README.md
>> +++ b/automation/build/README.md
>> @@ -81,7 +81,14 @@ Building a container
>>  
>>  There is a makefile to make this process easier. You should be
>>  able to run `make DISTRO/VERSION` to have Docker build the container
>> -for you. If you define the `PUSH` environment variable when running the
>> +for you.
>> +
>> +Xen's dockerfiles use heredocs, which depend on the standardised dockerfile
>> +syntax introduced by [BuiltKit].  This should work by default starting with
>> +docker 23.0, or podman/buildah v1.33.  For older versions of docker, it can 
>> be
>> +activated with `DOCKER_BUILDKIT=1` in the environment.
>> +
>> +If you define the `PUSH` environment variable when running the
>>  former `make` command, it will push the container to the [registry] if
>>  you have access to do so and have your Docker logged into the registry.
>>  
>> @@ -101,6 +108,7 @@ env CONTAINER_NO_PULL=1 \
>>  make -C automation/build suse/opensuse-tumbleweed PUSH=1
>>  ```
>>  
>> +[BuildKit]: https://docs.docker.com/build/buildkit/
>>  [registry]: https://gitlab.com/xen-project/xen/container_registry
>>  [registry help]: https://gitlab.com/help/user/project/container_registry
>>  
>> diff --git a/automation/build/alpine/3.18-arm64v8.dockerfile 
>> b/automation/build/alpine/3.18-arm64v8.dockerfile
>> index 91e90220240f..19fe46f8418f 100644
>> --- a/automation/build/alpine/3.18-arm64v8.dockerfile
>> +++ b/automation/build/alpine/3.18-arm64v8.dockerfile
>> @@ -1,3 +1,4 @@
>> +# syntax=docker/dockerfile:1
> Will this fail to parse if not using BuildKit?

No.  To "the legacy builder" (the pre-BuildKit thing), it's just a comment.

But to anything semi recent, it defines the version of the dockerfile
syntax we intend to use, and in particular will prevent you from
accidentally using newer syntax in an older dockerfile.

~Andrew