Re: [XEN PATCH] automation: add ECLAIR pipeline

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Thu, 20 Jul 2023, Simone Ballarin wrote:
> Add two pipelines that analyze an ARM64 and a X86_64 build with the
> ECLAIR static analyzer on the guidelines contained in Set1.
> 
> The tool configuration is kept external to the xen repository for
> practical reasons, it will be included in a subsequent phase.
> 
> All commits on the xen-project/xen:staging branch will be analyzed
> and their artifacts will be stored indefinitely; the integration will
> report differential information with respect to the previous analysis.
> 
> All commits on other branches or repositories will be analyzed and
> only the last ten artifacts will be kept; the integration will report
> differential information with respect to the analysis done on the common
> ancestor with xen-project/xen:staging (if available).
> 
> Currently the pipeline variable ENABLE_ECLAIR_BOT is set to "n".
> Doing so disables the generation of comments with the analysis summary
> on the commit threads. The variable can be set to "y" if the a masked
> variable named ECLAIRIT_TOKEN is set with the impersonation token of
> an account with enough privileges to write on all repositories.
> 
> Additionaly any repository should be able to read a masked variable
> named WTOKEN with the token provided by BUGSENG.
> 
> Signed-off-by: Simone Ballarin <simone.ballarin@xxxxxxxxxxx>

Thanks for the patch!

Patchew automatically picked it up from xen-devel and started a pipeline
here:

https://gitlab.com/xen-project/patchew/xen/-/pipelines/939440592

However the eclair-x86_64 job failed with:

ERROR: Uploading artifacts as "archive" to coordinator... 413 Payload
Too Large

Also the eclair-ARM64 job failed but it is not clear to me why.

I think at least initially we should mark the two Eclair jobs with:

  allow_failure: true

until we are sure they work reliably all the time. Otherwise we end up
blocking the whole Xen staging pipeline if we make any mistakes here. We
can remove "allow_failure: true" once we are sure it works well all the
time.


The second thing I noticed is that the build phase didn't start until
the analyze phase was concluded. This is not good because it would
increase the overall time significantly. We need the build/test phases
to start in parallel. To do that you need to add the following change to
this patch:


diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index c401f62d61..f01e2c32bb 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -11,6 +11,7 @@
       - '*.log'
       - '*/*.log'
     when: always
+  needs: []
   except:
     - master
     - smoke



> ---
>  .gitlab-ci.yml                    |  2 ++
>  automation/gitlab-ci/analyze.yaml | 38 +++++++++++++++++++++++++++++++
>  automation/scripts/eclair         | 26 +++++++++++++++++++++
>  3 files changed, 66 insertions(+)
>  create mode 100644 automation/gitlab-ci/analyze.yaml
>  create mode 100755 automation/scripts/eclair
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index c8bd7519d5..ee5430b8b7 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -1,7 +1,9 @@
>  stages:
> +  - analyze
>    - build
>    - test
>  
>  include:
> +  - 'automation/gitlab-ci/analyze.yaml'
>    - 'automation/gitlab-ci/build.yaml'
>    - 'automation/gitlab-ci/test.yaml'
> diff --git a/automation/gitlab-ci/analyze.yaml 
> b/automation/gitlab-ci/analyze.yaml
> new file mode 100644
> index 0000000000..be96d96e71
> --- /dev/null
> +++ b/automation/gitlab-ci/analyze.yaml
> @@ -0,0 +1,38 @@
> +.eclair-analysis:
> +  stage: analyze
> +  tags:
> +    - eclair-analysis
> +    - eclair
> +    - misrac

I would only use 1 tag, eclair-analysis or eclair, up to you


> +  variables:
> +    ECLAIR_OUTPUT_DIR: "ECLAIR_out"
> +    ANALYSIS_KIND: "normal"
> +    ECLAIR_REPORT_URL: "saas.eclairit.com"
> +    ENABLE_ECLAIR_BOT: "n"
> +    AUTOPRBRANCH: "staging"
> +    AUTOPRREPOSITORY: "xen-project/xen"
> +  artifacts:
> +    when: always
> +    paths:
> +      - "${ECLAIR_OUTPUT_DIR}"
> +      - '*.log'
> +    reports:
> +      codequality: gl-code-quality-report.json
> +
> +eclair-x86_64:
> +  extends: .eclair-analysis
> +  variables:
> +    LOGFILE: "eclair-x86_64.log"
> +    VARIANT: "X86_64"
> +    RULESET: "Set1"
> +  script:
> +    - ./automation/scripts/eclair 2>&1 | tee "${LOGFILE}"

allow_failure: true


> +eclair-ARM64:
> +  extends: .eclair-analysis
> +  variables:
> +    LOGFILE: "eclair-ARM64.log"
> +    VARIANT: "ARM64"
> +    RULESET: "Set1"
> +  script:
> +    - ./automation/scripts/eclair 2>&1 | tee "${LOGFILE}"

allow_failure: true


> diff --git a/automation/scripts/eclair b/automation/scripts/eclair
> new file mode 100755
> index 0000000000..d7f0845aec
> --- /dev/null
> +++ b/automation/scripts/eclair
> @@ -0,0 +1,26 @@
> +#!/bin/bash -eu
> +
> +# ECLAIR configuration files are maintened by BUGSENG
> +export GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no"
> +[ -d ECLAIR_scripts ] || git clone 
> ssh://git@xxxxxxxxxxxxxxx/eclair/scripts/XEN ECLAIR_scripts
> +(cd ECLAIR_scripts; git pull --rebase)
> +
> +ECLAIR_DIR=ECLAIR_scripts/ECLAIR
> +ECLAIR_OUTPUT_DIR=$(realpath "${ECLAIR_OUTPUT_DIR}")
> +
> +ECLAIR_scripts/prepare.sh "${VARIANT}"
> +
> +ex=0
> +"${ECLAIR_DIR}/analyze.sh" "${VARIANT}" "${RULESET}" || ex=$?
> +"${ECLAIR_DIR}/action_log.sh" ANALYSIS_LOG \
> +                             "ECLAIR analysis log" \
> +                             "${ECLAIR_OUTPUT_DIR}/ANALYSIS.log" \
> +                             "${ex}"
> +"${ECLAIR_DIR}/action_log.sh" REPORT_LOG \
> +                             "ECLAIR report log" \
> +                             "${ECLAIR_OUTPUT_DIR}/REPORT.log" \
> +                             "${ex}"
> +[ "${ex}" = 0 ] || exit "${ex}"
> +"${ECLAIR_DIR}/action_push.sh" "${WTOKEN}" "${ECLAIR_OUTPUT_DIR}"
> +
> +rm -rf "${ECLAIR_OUTPUT_DIR}/.data"
> -- 
> 2.34.1
>