Xen project Mailing List

[Xen-devel] [PATCH v3 7/8] osstest: introduce a script to create a FreeBSD flight

From: Roger Pau Monne <roger.pau@xxxxxxxxxx>

Date: Fri, 2 Jun 2017 14:37:51 +0100

Cc: ian.Jackson@xxxxxxxxxxxxx, Roger Pau Monne <roger.pau@xxxxxxxxxx>

Delivery-date: Fri, 02 Jun 2017 13:38:10 +0000

List-id: Xen developer discussion <xen-devel.lists.xen.org>

This requires changes in several places in order to accommodate the FreeBSD build, which although it's a build job, it doesn't have the same set of dependencies as the current builds. First, a new build recipe is added to sg-run-job, and accordingly sg-run-job is also made aware about the differences between a Linux and a FreeBSD build job. A Linux build job requires ts-host-allocate + ts-host-install-twice + ts-xen-build-prep, while a FreeBSD build job requires ts-freebsd-set-hostflags + ts-host-allocate + ts-freebsd-host-install. All the current build jobs are kept to use the Linux build recipe, while the newly added build-freebsd is using the new FreeBSD recipe. Finally, the logic to create a FreeBSD build job is added to make-freebsd-flight. This includes creating a FreeBSD build job, and also testing the output of that build job (by creating another build job that depends on the output of the first). Note that the FreeBSD build job needs some input in order to setup a FreeBSD host, and that can be fetched from different places: 1. Env variable FREEBSD_BUILDJOB: use the output from a previous build-<arch>-freebsd. 2. Env variables FREEBSD_DIST and FREEBSD_VERSION: set before calling into make-flight, provide the path to the installer image and sets folder and the version being installed. 3. Config file FreeBSDDist and FreeBSDVersion: same as 2. except that they are set on the config file. Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx> --- Changes since v1: - Replace freebsd_buildjob with freebsdbuildjob. - Replace FREEBSD_SETS/IMAGE with a single FREEBSD_DIST that points to a folder that should contain both things. - Document the FreeBSDDist and FreeBSDVersion config file options in the README file. --- README | 11 +++++++ ap-common | 4 +++ make-freebsd-flight | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++ sg-run-job | 43 ++++++++++++++++++------- 4 files changed, 138 insertions(+), 12 deletions(-) create mode 100755 make-freebsd-flight diff --git a/README b/README index b45058da..51243974 100644 --- a/README +++ b/README @@ -423,6 +423,17 @@ HostGroupFlags_<group> DebianPreseed Text to add to the debian-installer preseed file. Optional. +FreeBSDDist + Path to the folder that contains the FreeBSD install image and + the FreeBSD compressed install sets, together with the MANIFEST + file that holds the checksums. This is required in order to run + a FreeBSD host install if no previous FreeBSD buildjob is + available (ie: for example when running in standalone mode). + +FreeBSDVersion + Numeric value holding the major FreeBSD version of the media + provided in FreeBSDDist (ie: 12). + ======================================== Config settings relevant only to standalone mode diff --git a/ap-common b/ap-common index cbb815ce..d4fa7aef 100644 --- a/ap-common +++ b/ap-common @@ -37,6 +37,10 @@ : ${PUSH_TREE_XTF:=$XENBITS:/home/xen/git/xtf.git} : ${BASE_TREE_XTF:=git://xenbits.xen.org/xtf.git} +: ${TREE_FREEBSD:=git://github.com/freebsd/freebsd.git} +: ${PUSH_TREE_FREEBSD:=$XENBITS:/home/xen/git/freebsd.git} +: ${BASE_TREE_FREEBSD:=git://xenbits.xen.org/freebsd.git} + : ${TREE_LIBVIRT:=git://libvirt.org/libvirt.git} : ${PUSH_TREE_LIBVIRT:=$XENBITS:/home/xen/git/libvirt.git} : ${BASE_TREE_LIBVIRT:=git://xenbits.xen.org/libvirt.git} diff --git a/make-freebsd-flight b/make-freebsd-flight new file mode 100755 index 00000000..93bb2cf3 --- /dev/null +++ b/make-freebsd-flight @@ -0,0 +1,92 @@ +#!/bin/bash + +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2017 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + + +set -e -o posix + +branch=$1 +xenbranch=$2 +blessing=$3 +buildflight=$4 + +flight=`./cs-flight-create $blessing $branch` + +. ./cri-common +. ./ap-common +. ./mfi-common + +arch=amd64 + +job_create_build_filter_callback () { + : +} + +get_freebsdjob_flags () { + arch=$1 + + # Figure out where are the installer binaries. The order is the following: + # + # 1. Env variable FREEBSD_BUILDJOB: use the output from a previous + # build-<arch>-freebsd. + # + # 2. Env variables FREEBSD_DIST, FREEBSD_VERSION: set before calling into + # make-flight, provide the path to the installer image, the sets to install + # and the version being installed. + # + # 3. Config file FreeBSDDist, FreeBSDVersion: same as 2. except that they + # are set on the config file. + # + if [ -n "$FREEBSD_BUILDJOB" ]; then + flags="freebsdbuildjob=$FREEBSD_BUILDJOB" + elif [ -n "$FREEBSD_DIST" ] && [ -n "$FREEBSD_VERSION" ]; then + flags="freebsd_distpath=$FREEBSD_DIST freebsd_version=$FREEBSD_VERSION" + else + distpath=`getconfig "FreeBSDDist"` + version=`getconfig "FreeBSDVersion"` + flags="freebsd_distpath=$distpath freebsd_version=$version" + fi + + echo $flags +} + +flags=`get_freebsdjob_flags $arch` +job_create_build build-$arch-freebsd build-freebsd \ + arch=$arch \ + $RUNVARS $BUILD_RUNVARS $BUILD_FREEBSD_RUNVARS $arch_runvars \ + tree_freebsd=$TREE_FREEBSD \ + revision_freebsd=$REVISION_FREEBSD \ + extra_hostflags=arch-$arch,purpose-build \ + $flags + +# Create an identical job that's going to use the build output from +# the previous one. +job_create_build build-$arch-freebsd-again build-freebsd \ + arch=$arch \ + $RUNVARS $BUILD_RUNVARS $BUILD_FREEBSD_RUNVARS $arch_runvars \ + extra_hostflags=arch-$arch,purpose-build \ + tree_freebsd=$TREE_FREEBSD \ + revision_freebsd=$REVISION_FREEBSD \ + freebsdbuildjob=build-$arch-freebsd + +echo $flight + +# Local variables: +# mode: sh +# sh-basic-offset: 2 +# indent-tabs-mode: nil +# End: diff --git a/sg-run-job b/sg-run-job index ceb79800..239446b8 100755 --- a/sg-run-job +++ b/sg-run-job @@ -53,12 +53,15 @@ proc run-job {job} { set skip_globs [jobdb::read-runvar $flight $job skip_testids] set nh [need-hosts/$jobinfo(recipe)] - if {![string compare $nh BUILD]} { + if {![string compare $nh BUILD_LINUX]} { set need_xen_hosts {} - set need_build_host 1 + set need_build_host LINUX + } elseif {![string compare $nh BUILD_FREEBSD]} { + set need_xen_hosts {} + set need_build_host FREEBSD } else { set need_xen_hosts $nh - set need_build_host 0 + set need_build_host "" } set nested_layers_hosts {} @@ -69,7 +72,11 @@ proc run-job {job} { eval run-ts broken = ts-hosts-allocate + $need_xen_hosts } - if {$need_build_host} { catching-otherwise broken prepare-build-host } + if {![string compare $need_build_host LINUX]} { + catching-otherwise broken prepare-build-host-linux + } elseif {![string compare $need_build_host FREEBSD]} { + catching-otherwise broken prepare-build-host-freebsd + } if {$ok} { setstatus running } @@ -89,7 +96,7 @@ proc run-job {job} { set need_xen_hosts [lunappend nested_layers_hosts] } - if {$need_build_host && $anyfailed} { + if {[string length $need_build_host] && $anyfailed} { run-ts !broken capture-logs ts-logs-capture + host } @@ -106,7 +113,7 @@ proc run-job {job} { if {$ok} { setstatus pass } - if {$need_build_host && $ok} { jobdb::preserve-task 90 } + if {[string length $need_build_host] && $ok} { jobdb::preserve-task 90 } if {$anyfailed} { jobdb::logputs stdout "at least one test failed" @@ -535,11 +542,12 @@ proc need-hosts/host-examine-linux {} { #---------- builds ---------- -proc need-hosts/build {} { return BUILD } -proc need-hosts/build-kern {} { return BUILD } -proc need-hosts/build-libvirt {} { return BUILD } -proc need-hosts/build-rumprun {} { return BUILD } -proc need-hosts/build-xtf {} { return BUILD } +proc need-hosts/build {} { return BUILD_LINUX } +proc need-hosts/build-kern {} { return BUILD_LINUX } +proc need-hosts/build-libvirt {} { return BUILD_LINUX } +proc need-hosts/build-rumprun {} { return BUILD_LINUX } +proc need-hosts/build-xtf {} { return BUILD_LINUX } +proc need-hosts/build-freebsd {} { return BUILD_FREEBSD } proc run-job/build {} { run-ts . = ts-xen-build @@ -566,13 +574,24 @@ proc run-job/build-xtf {} { run-ts . = ts-xtf-build } -proc prepare-build-host {} { +proc run-job/build-freebsd {} { + run-ts . = ts-freebsd-build +} + +proc prepare-build-host-linux {} { global jobinfo run-ts broken = ts-hosts-allocate + host run-ts broken host-install(*) ts-host-install-twice run-ts . host-build-prep ts-xen-build-prep } +proc prepare-build-host-freebsd {} { + global jobinfo + run-ts broken = ts-freebsd-set-hostflags + run-ts broken = ts-hosts-allocate + host + run-ts broken host-install(*) ts-freebsd-host-install +} + proc need-hosts/coverity {} { return BUILD } proc run-job/coverity {} { run-ts . = ts-coverity-build + host -- 2.11.0 (Apple Git-81) _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx https://lists.xen.org/xen-devel

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.