|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [TEST_ARTIFACTS PATCH v1 1/1] build: add x86_64 xen artifacts (argo)
On Thu, 24 Oct 2024, victorm.lira@xxxxxxx wrote:
> From: Victor Lira <victorm.lira@xxxxxxx>
>
> Add container image build files:
> - alpine/x86_64-build
> - alpine/x86_64-rootfs
> - Makefile
>
> Add CI configuration and jobs to produce binaries for xen tests:
> - xen-argo Linux Xen Argo kernel module
> - bzImage Linux kernel
> - libargo Linux Argo shared library
> - argo-exec Linux Argo test program
> - initrd.tar.gz Alpine Linux rootfs
>
> Signed-off-by: Victor Lira <victorm.lira@xxxxxxx>
Reviewed-by: Stefano Stabellini <sstabellini@xxxxxxxxxx>
One comment below
> ---
> Changes v1:
> - Update dockerfile format to group dependencies
> - add user to build container
> ---
> Cc: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
> Cc: Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>
> Cc: Doug Goldstein <cardoe@xxxxxxxxxx>
> Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx
> ---
> .gitlab-ci.yml | 50 ++++++++++++++++++++
> binaries/.gitignore | 3 ++
> images/Makefile | 20 ++++++++
> images/alpine/x86_64-build.dockerfile | 31 +++++++++++++
> images/alpine/x86_64-rootfs.dockerfile | 4 ++
> scripts/x86_64-argo-linux.sh | 63 ++++++++++++++++++++++++++
> scripts/x86_64-kernel-linux.sh | 31 +++++++++++++
> scripts/x86_64-rootfs-alpine.sh | 58 ++++++++++++++++++++++++
> 8 files changed, 260 insertions(+)
> create mode 100644 .gitlab-ci.yml
> create mode 100644 binaries/.gitignore
> create mode 100644 images/Makefile
> create mode 100644 images/alpine/x86_64-build.dockerfile
> create mode 100644 images/alpine/x86_64-rootfs.dockerfile
> create mode 100755 scripts/x86_64-argo-linux.sh
> create mode 100755 scripts/x86_64-kernel-linux.sh
> create mode 100755 scripts/x86_64-rootfs-alpine.sh
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> new file mode 100644
> index 0000000..7e18a87
> --- /dev/null
> +++ b/.gitlab-ci.yml
> @@ -0,0 +1,50 @@
> +workflow:
> + name: "xen test artifacts"
> +
> +.artifacts:
> + stage: build
> + image:
> + name:
> registry.gitlab.com/xen-project/hardware/test-artifacts/${CONTAINER}
> + artifacts:
> + name: "${CI_JOB_NAME_SLUG}"
> + paths:
> + - binaries/
> + exclude:
> + - binaries/.gitignore
> +
> +.x86_64-artifacts:
> + extends: .artifacts
> + image:
> + docker:
> + platform: linux/amd64
> + user: xenproject
> + tags:
> + - x86_64
> + variables:
> + CONTAINER: alpine:x86_64-build
> +
> +x86_64-kernel-linux-6.6.56:
> + extends: .x86_64-artifacts
> + script:
> + - . scripts/x86_64-kernel-linux.sh
> + variables:
> + LINUX_VERSION: 6.6.56
> +
> +x86_64-rootfs-alpine-3.18:
> + extends: .x86_64-artifacts
> + image:
> + docker:
> + user: root
> + script:
> + - . scripts/x86_64-rootfs-alpine.sh
> + variables:
> + CONTAINER: alpine:x86_64-rootfs
> +
> +x86_64-argo-linux-6.6.56:
> + extends: .x86_64-artifacts
> + script:
> + - . scripts/x86_64-argo-linux.sh
> + variables:
> + LINUX_VERSION: 6.6.56
> + ARGO_SHA: "705a7a8a624b42e13e655d3042059b8a85cdf6a3"
> + ARGOEXEC_SHA: "d900429f6640acc6f68a3d3a4c945d7da60625d8"
> diff --git a/binaries/.gitignore b/binaries/.gitignore
> new file mode 100644
> index 0000000..95e2a2e
> --- /dev/null
> +++ b/binaries/.gitignore
> @@ -0,0 +1,3 @@
> +# Keep this directory around to be available in CI/CD jobs.
> +*
> +!.gitignore
> diff --git a/images/Makefile b/images/Makefile
> new file mode 100644
> index 0000000..42f231b
> --- /dev/null
> +++ b/images/Makefile
> @@ -0,0 +1,20 @@
> +
> +# The base of where these containers will appear
> +REGISTRY := registry.gitlab.com/xen-project/hardware/test-artifacts
> +CONTAINERS = $(subst .dockerfile,,$(wildcard */*.dockerfile))
> +
> +help:
> + @echo "Containers to build test artifacts."
> + @echo "To build one run 'make DISTRO/TAG'."
> + @echo "Available containers:"
> + @$(foreach file,$(sort $(CONTAINERS)),echo ${file};)
> + @echo "To push container builds, set the environment variable PUSH"
> +
> +%: %.dockerfile ## Builds containers
> + docker build --pull -t $(REGISTRY)/$(@D):$(@F) -f $< $(<D)
> + @if [ ! -z $${PUSH+x} ]; then \
> + docker push $(REGISTRY)/$(@D):$(@F); \
> + fi
> +
> +.PHONY: all
> +all: $(CONTAINERS)
> diff --git a/images/alpine/x86_64-build.dockerfile
> b/images/alpine/x86_64-build.dockerfile
> new file mode 100644
> index 0000000..3bfd172
> --- /dev/null
> +++ b/images/alpine/x86_64-build.dockerfile
> @@ -0,0 +1,31 @@
> +# syntax=docker/dockerfile:1
> +FROM --platform=linux/amd64 alpine:3.18
> +LABEL maintainer.name="The Xen Project" \
> + maintainer.email="xen-devel@xxxxxxxxxxxxxxxxxxxx"
> +
> +WORKDIR /build/
> +
> +RUN apk --no-cache add bash
> +
> +RUN bash -ex <<EOF
> + adduser -D xenproject --shell bash
> +
> + DEPS=(
> + musl-dev
> + build-base
> + libc6-compat
> + linux-headers
> + bash
> + git
> + curl
> + flex
> + bison
> + elfutils-dev
> + autoconf
> + automake
> + libtool
> + openssl-dev
> + )
> +
> + apk add --no-cache "\${DEPS[@]}"
> +EOF
> diff --git a/images/alpine/x86_64-rootfs.dockerfile
> b/images/alpine/x86_64-rootfs.dockerfile
> new file mode 100644
> index 0000000..b912e9c
> --- /dev/null
> +++ b/images/alpine/x86_64-rootfs.dockerfile
> @@ -0,0 +1,4 @@
> +# syntax=docker/dockerfile:1
> +FROM --platform=linux/amd64 alpine:3.18
> +LABEL maintainer.name="The Xen Project" \
> + maintainer.email="xen-devel@xxxxxxxxxxxxxxxxxxxx"
> diff --git a/scripts/x86_64-argo-linux.sh b/scripts/x86_64-argo-linux.sh
> new file mode 100755
> index 0000000..a110a33
> --- /dev/null
> +++ b/scripts/x86_64-argo-linux.sh
> @@ -0,0 +1,63 @@
> +#!/usr/bin/env bash
> +
> +if test -z "${LINUX_VERSION}"
> +then
> + >&2 echo "LINUX_VERSION must be set"; exit 1
> +fi
> +
> +set -ex -o pipefail
> +
> +BUILDDIR="${PWD}"
> +COPYDIR="${BUILDDIR}/binaries/"
> +
> +# Prepare Linux sources
> +curl -fsSLO \
> +
> https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"${LINUX_VERSION}".tar.xz
> +tar xJf linux-"${LINUX_VERSION}".tar.xz
> +cd linux-"${LINUX_VERSION}"
> +make ARCH=x86 defconfig
> +make ARCH=x86 xen.config
> +./scripts/config --enable BRIDGE
> +./scripts/config --enable IGC
> +./scripts/config --enable TUN
> +cp .config .config.orig
> +cat .config.orig \
> + | grep 'XEN' \
> + | grep '=m' \
> + | sed 's/=m/=y/g' \
> + >> .config
> +make ARCH=x86 olddefconfig
> +make ARCH=x86 modules_prepare
> +
> +# Build Linux kernel module for Xen Argo
> +cd "${BUILDDIR}"
> +git clone \
> + --depth=1 --branch=master \
> + https://github.com/OpenXT/linux-xen-argo.git
> +git -C "${BUILDDIR}/linux-xen-argo" switch --detach "${ARGO_SHA}"
> +make -C "linux-${LINUX_VERSION}" M="${BUILDDIR}/linux-xen-argo/argo-linux" \
> + CFLAGS_MODULE="-Wno-error" KBUILD_MODPOST_WARN=1
> +cp "linux-xen-argo/argo-linux/xen-argo.ko" "${COPYDIR}/xen-argo.ko"
> +
> +# Build Linux libargo shared library, applying fixes to build in Alpine Linux
> +cd "${BUILDDIR}/linux-xen-argo/libargo"
> +sed -i "s|AM_INIT_AUTOMAKE|AC_CONFIG_AUX_DIR(.)\nAM_INIT_AUTOMAKE|"
> configure.ac
> +sed -i "s/__SOCKADDR_COMMON (sxenargo_)/sa_family_t sxenargo_family/"
> src/libargo.h
> +sed -i "s/__SOCKADDR_COMMON_SIZE/(sizeof (unsigned short int))/"
> src/libargo.h
> +autoreconf --install
> +./configure --prefix="${COPYDIR}" CPPFLAGS="-I${PWD}/../argo-linux/include"
> +make
> +make install
> +
> +# Build Linux user program, modifying for xilinx argo test
> +cd "${BUILDDIR}"
> +wget "https://raw.githubusercontent.com/OpenXT/xenclient-oe/${ARGOEXEC_SHA}/\
> +recipes-openxt/argo-exec/argo-exec/argo-exec.c"
> +sed -i "s|#include <xen/xen.h>||" argo-exec.c
> +sed -i "s|ret = shuffle(s, fds\[0\], fds\[1\]);|ret = shuffle(s, 0, 1);|" \
> + argo-exec.c
Normally I would suggest to handle this as a patch file but given it is
just 2 changes, and it is nice to have everything contained within this
file, I think it is OK. In the future if we grow more changes, I'd
suggest to move it out.
> +gcc -I"${BUILDDIR}/linux-xen-argo/libargo/src" \
> + -I"${BUILDDIR}/linux-xen-argo/argo-linux/include" \
> + -L"${COPYDIR}/lib/" \
> + -o argo-exec argo-exec.c -largo
> +cp argo-exec "${COPYDIR}"
> diff --git a/scripts/x86_64-kernel-linux.sh b/scripts/x86_64-kernel-linux.sh
> new file mode 100755
> index 0000000..2a816f0
> --- /dev/null
> +++ b/scripts/x86_64-kernel-linux.sh
> @@ -0,0 +1,31 @@
> +#!/usr/bin/env bash
> +
> +if test -z "${LINUX_VERSION}"
> +then
> + >&2 echo "LINUX_VERSION must be set"; exit 1
> +fi
> +
> +set -ex -o pipefail
> +
> +WORKDIR="${PWD}"
> +COPYDIR="${WORKDIR}/binaries/"
> +
> +# Build Linux
> +curl -fsSLO \
> +
> https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"${LINUX_VERSION}".tar.xz
> +tar xJf linux-"${LINUX_VERSION}".tar.xz
> +cd linux-"${LINUX_VERSION}"
> +make ARCH=x86 defconfig
> +make ARCH=x86 xen.config
> +./scripts/config --enable BRIDGE
> +./scripts/config --enable IGC
> +./scripts/config --enable TUN
> +cp .config .config.orig
> +cat .config.orig \
> + | grep 'XEN' \
> + | grep '=m' \
> + | sed 's/=m/=y/g' \
> + >> .config
> +make ARCH=x86 olddefconfig
> +make -s -j "$(nproc)" ARCH=x86
> +cp arch/x86/boot/bzImage "${COPYDIR}"
> diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
> new file mode 100755
> index 0000000..28d8e30
> --- /dev/null
> +++ b/scripts/x86_64-rootfs-alpine.sh
> @@ -0,0 +1,58 @@
> +WORKDIR="${PWD}"
> +
> +apk update
> +
> +# xen runtime deps
> +apk add musl
> +apk add libgcc
> +apk add openrc
> +apk add busybox
> +apk add sudo
> +apk add dbus
> +apk add bash
> +apk add python3
> +apk add zlib
> +apk add ncurses
> +apk add yajl
> +apk add libaio
> +apk add xz
> +apk add util-linux
> +apk add argp-standalone
> +apk add libfdt
> +apk add glib
> +apk add pixman
> +apk add curl
> +apk add udev
> +apk add pciutils
> +apk add libelf
> +
> +# Xen
> +cd /
> +# Minimal ramdisk environment in case of cpio output
> +rc-update add udev
> +rc-update add udev-trigger
> +rc-update add udev-settle
> +rc-update add loopback sysinit
> +rc-update add bootmisc boot
> +rc-update add devfs sysinit
> +rc-update add dmesg sysinit
> +rc-update add hostname boot
> +rc-update add hwclock boot
> +rc-update add hwdrivers sysinit
> +rc-update add modules boot
> +rc-update add killprocs shutdown
> +rc-update add mount-ro shutdown
> +rc-update add savecache shutdown
> +rc-update add local default
> +cp -a /sbin/init /init
> +echo "ttyS0" >> /etc/securetty
> +echo "hvc0" >> /etc/securetty
> +echo "ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100" >> /etc/inittab
> +echo "hvc0::respawn:/sbin/getty -L hvc0 115200 vt100" >> /etc/inittab
> +echo > /etc/modules
> +passwd -d "root" root
> +
> +# Create rootfs
> +cd /
> +tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
> + bin dev etc home init lib mnt opt root sbin usr var
> --
> 2.25.1
>
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |