Issue with check-endbr.sh during Xen build

[Luca Fancellu <Luca.Fancellu@xxxxxxx>]

Hi all,

I’ve encountered an issue building Xen for x86_64, basically the script 
check-endbr.sh was returning an error:
“Error: More valid-addrs than all-addrs”.

I have found a solution but I’m writing anyway to you to make you aware of the 
issue.

In my setup I’m using a docker container with base image from ubuntu:22.04, 
when I build from the container it
Is failing, instead when I’m building from my machine it is working fine.

So I’ve set -x in the script to trace it and I have this for my host machine 
(xen_x86-docker is just xen-syms):

$ ./xen/xen/tools/check-endbr.sh tmp_files/xen_x86-docker
+ MSG_PFX=check-endbr.sh xen_x86-docker
+ OBJCOPY=objcopy
+ OBJDUMP=objdump
+ ADDR2LINE=addr2line
+ mktemp -d
+ D=/tmp/tmp.3TmVp2idCm
+ trap rm -rf /tmp/tmp.3TmVp2idCm EXIT
+ TEXT_BIN=/tmp/tmp.3TmVp2idCm/xen-syms.text
+ VALID=/tmp/tmp.3TmVp2idCm/valid-addrs
+ ALL=/tmp/tmp.3TmVp2idCm/all-addrs
+ BAD=/tmp/tmp.3TmVp2idCm/bad-addrs
+ echo X
+ grep -aob X -q
+ perl_re=true
+ echo X
+ grep -aobP \x58 -q
+ objdump -j .text tmp_files/xen_x86-docker -d -w
+ grep  endbr64 *$
+ cut -f 1 -d :
+ objdump -j .text tmp_files/xen_x86-docker -h
+ awk $2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), 
substr($4, 9, 16)}
+ eval vma_hi=ffff82d0 vma_lo=40200000
+ vma_hi=ffff82d0 vma_lo=40200000
+ objcopy -j .text tmp_files/xen_x86-docker -O binary 
/tmp/tmp.3TmVp2idCm/xen-syms.text
+ true
+ LC_ALL=C grep -aobP \xf3\x0f\x1e(\xfa|\xfb)|\x66\x0f\x1f\x01 
/tmp/tmp.3TmVp2idCm/xen-syms.text
+ awk -F: {printf "%s%x\n", "ffff82d0", int(0x40200000) + $1}
+ wait
+ stat -c %s /tmp/tmp.3TmVp2idCm/valid-addrs
+ val_sz=24004
+ stat -c %s /tmp/tmp.3TmVp2idCm/all-addrs
+ all_sz=24004
+ [ 24004 -eq 0 ]
+ [ 24004 -eq 0 ]
+ [ 24004 -lt 24004 ]
+ sort /tmp/tmp.3TmVp2idCm/valid-addrs /tmp/tmp.3TmVp2idCm/all-addrs
+ uniq -u
+ wc -l
+ nr_bad=0
+ [ 0 -eq 0 ]
+ exit 0
+ rm -rf /tmp/tmp.3TmVp2idCm

And I have this from my docker container:

# ./xen/xen/tools/check-endbr.sh tmp_files/xen_x86-docker
+ MSG_PFX='check-endbr.sh xen_x86-docker'
+ OBJCOPY=objcopy
+ OBJDUMP=objdump
+ ADDR2LINE=addr2line
++ mktemp -d
+ D=/tmp/tmp.tR0Vt8l2ie
+ trap 'rm -rf /tmp/tmp.tR0Vt8l2ie' EXIT
+ TEXT_BIN=/tmp/tmp.tR0Vt8l2ie/xen-syms.text
+ VALID=/tmp/tmp.tR0Vt8l2ie/valid-addrs
+ ALL=/tmp/tmp.tR0Vt8l2ie/all-addrs
+ BAD=/tmp/tmp.tR0Vt8l2ie/bad-addrs
+ echo X
+ grep -aob X -q
+ perl_re=true
+ echo X
+ grep -aobP '\x58' -q
+ objdump -j .text tmp_files/xen_x86-docker -d -w
+ grep '        endbr64 *$'
+ cut -f 1 -d :
++ objdump -j .text tmp_files/xen_x86-docker -h
++ awk '$2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), 
substr($4, 9, 16)}'
+ eval vma_hi=ffff82d0 vma_lo=40200000
++ vma_hi=ffff82d0
++ vma_lo=40200000
+ objcopy -j .text tmp_files/xen_x86-docker -O binary 
/tmp/tmp.tR0Vt8l2ie/xen-syms.text
+ true
+ LC_ALL=C
+ grep -aobP '\xf3\x0f\x1e(\xfa|\xfb)|\x66\x0f\x1f\x01' 
/tmp/tmp.tR0Vt8l2ie/xen-syms.text
+ awk -F: '{printf "%s%x\n", "ffff82d0", int(0x40200000) + $1}'
+ wait
++ stat -c %s /tmp/tmp.tR0Vt8l2ie/valid-addrs
+ val_sz=24004
++ stat -c %s /tmp/tmp.tR0Vt8l2ie/all-addrs
+ all_sz=19686
+ '[' 24004 -eq 0 ']'
+ '[' 19686 -eq 0 ']'
+ '[' 19686 -lt 24004 ']'
+ echo 'check-endbr.sh xen_x86-docker Error: More valid-addrs than all-addrs'
check-endbr.sh xen_x86-docker Error: More valid-addrs than all-addrs
+ exit 1
+ rm -rf /tmp/tmp.tR0Vt8l2ie


After spending a bit of time to understand what the script does, I’ve checked 
the awk version
and seems that the script is failing in the Docker container because there is 
mawk there:
# awk -W version
mawk 1.3.4 20200120
Copyright 2008-2019,2020, Thomas E. Dickey
Copyright 1991-1996,2014, Michael D. Brennan

random-funcs:       srandom/random
regex-funcs:        internal
compiled limits:
sprintf buffer      8192
maximum-integer     2147483647

Instead in my host machine there is gawk:
$ awk -W version
GNU Awk 4.1.4, API: 1.1 (GNU MPFR 4.0.1, GNU MP 6.1.2)
[…]


So there seems to be a dependency on the interpreter (or at least it doesn’t 
work with that version of mawk).

My solution was to install gawk in the container and everything was working 
fine afterwards.

Cheers,
Luca