[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [XEN PATCH] automation: add ECLAIR pipeline
Patchew automatically picked it up from xen-devel and started a pipeline
here:
https://gitlab.com/xen-project/patchew/xen/-/pipelines/939440592
However the eclair-x86_64 job failed with:
ERROR: Uploading artifacts as "archive" to coordinator... 413 Payload
Too Large
In the next patch, I will avoid uploading the ECLAIR database
in GitLab artifacts. Of course, the databases can still be accessed via the links in the pipeline logs.
Also the eclair-ARM64 job failed but it is not clear to me why.
The reason is that the WTOKEN variable is missing. This masked variable should be set and visible to *all* repositories (xen-project/xen, xen-project/people/.*).
I think at least initially we should mark the two Eclair jobs with:
allow_failure: true
until we are sure they work reliably all the time. Otherwise we end up
blocking the whole Xen staging pipeline if we make any mistakes here. We
can remove "allow_failure: true" once we are sure it works well all the
time.
Ok.
The second thing I noticed is that the build phase didn't start until
the analyze phase was concluded. This is not good because it would
increase the overall time significantly. We need the build/test phases
to start in parallel. To do that you need to add the following change to
this patch:
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index c401f62d61..f01e2c32bb 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -11,6 +11,7 @@
- '*.log'
- '*/*.log'
when: always
+ needs: []
except:
- master
- smoke
Thanks.
> ---
> .gitlab-ci.yml | 2 ++
> automation/gitlab-ci/analyze.yaml | 38 +++++++++++++++++++++++++++++++
> automation/scripts/eclair | 26 +++++++++++++++++++++
> 3 files changed, 66 insertions(+)
> create mode 100644 automation/gitlab-ci/analyze.yaml
> create mode 100755 automation/scripts/eclair
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index c8bd7519d5..ee5430b8b7 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -1,7 +1,9 @@
> stages:
> + - analyze
> - build
> - test
>
> include:
> + - 'automation/gitlab-ci/analyze.yaml'
> - 'automation/gitlab-ci/build.yaml'
> - 'automation/gitlab-ci/test.yaml'
> diff --git a/automation/gitlab-ci/analyze.yaml b/automation/gitlab-ci/analyze.yaml
> new file mode 100644
> index 0000000000..be96d96e71
> --- /dev/null
> +++ b/automation/gitlab-ci/analyze.yaml
> @@ -0,0 +1,38 @@
> +.eclair-analysis:
> + stage: analyze
> + tags:
> + - eclair-analysis
> + - eclair
> + - misrac
I would only use 1 tag, eclair-analysis or eclair, up to you
Ok.
> + variables:
> + ECLAIR_OUTPUT_DIR: "ECLAIR_out"
> + ANALYSIS_KIND: "normal"
> + ECLAIR_REPORT_URL: "saas.eclairit.com"
> + ENABLE_ECLAIR_BOT: "n"
> + AUTOPRBRANCH: "staging"
> + AUTOPRREPOSITORY: "xen-project/xen"
> + artifacts:
> + when: always
> + paths:
> + - "${ECLAIR_OUTPUT_DIR}"
> + - '*.log'
> + reports:
> + codequality: gl-code-quality-report.json
> +
> +eclair-x86_64:
> + extends: .eclair-analysis
> + variables:
> + LOGFILE: "eclair-x86_64.log"
> + VARIANT: "X86_64"
> + RULESET: "Set1"
> + script:
> + - ./automation/scripts/eclair 2>&1 | tee "${LOGFILE}"
allow_failure: true
> +eclair-ARM64:
> + extends: .eclair-analysis
> + variables:
> + LOGFILE: "eclair-ARM64.log"
> + VARIANT: "ARM64"
> + RULESET: "Set1"
> + script:
> + - ./automation/scripts/eclair 2>&1 | tee "${LOGFILE}"
allow_failure: true
> diff --git a/automation/scripts/eclair b/automation/scripts/eclair
> new file mode 100755
> index 0000000000..d7f0845aec
> --- /dev/null
> +++ b/automation/scripts/eclair
> @@ -0,0 +1,26 @@
> +#!/bin/bash -eu
> +
> +# ECLAIR configuration files are maintened by BUGSENG
> +export GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no"
> +[ -d ECLAIR_scripts ] || git clone ssh://git@xxxxxxxxxxxxxxx/eclair/scripts/XEN ECLAIR_scripts
> +(cd ECLAIR_scripts; git pull --rebase)
> +
> +ECLAIR_DIR=ECLAIR_scripts/ECLAIR
> +ECLAIR_OUTPUT_DIR=$(realpath "${ECLAIR_OUTPUT_DIR}")
> +
> +ECLAIR_scripts/prepare.sh "${VARIANT}"
> +
> +ex=0
> +"${ECLAIR_DIR}/analyze.sh" "${VARIANT}" "${RULESET}" || ex=$?
> +"${ECLAIR_DIR}/action_log.sh" ANALYSIS_LOG \
> + "ECLAIR analysis log" \
> + "${ECLAIR_OUTPUT_DIR}/ANALYSIS.log" \
> + "${ex}"
> +"${ECLAIR_DIR}/action_log.sh" REPORT_LOG \
> + "ECLAIR report log" \
> + "${ECLAIR_OUTPUT_DIR}/REPORT.log" \
> + "${ex}"
> +[ "${ex}" = 0 ] || exit "${ex}"
> +"${ECLAIR_DIR}/action_push.sh" "${WTOKEN}" "${ECLAIR_OUTPUT_DIR}"
> +
> +rm -rf "${ECLAIR_OUTPUT_DIR}/.data"
> --
> 2.34.1
>
|
