Re: Error around cc-option on AArch64

[Jan Beulich <jbeulich@xxxxxxxx>]

On 07.09.2021 15:03, Penny Zheng wrote:
> Hi Julien and Stefano
> 
> I found that the "cc-option/cc-option-add" is not working for "-march=xxx" 
> option on a few very common aarch64 compilers.
> Here is what I got when trying to compile XEN on r82 platform.
> ```
> diff --git a/xen/arch/arm/arch.mk b/xen/arch/arm/arch.mk
> index 11caec86ba..d2d71baef4 100644
> --- a/xen/arch/arm/arch.mk
> +++ b/xen/arch/arm/arch.mk
> @@ -10,6 +10,10 @@ $(call cc-option-add,CFLAGS,CC,-Wnested-externs)
> CFLAGS-$(CONFIG_ARM_32) += -msoft-float
> CFLAGS-$(CONFIG_ARM_32) += -mcpu=cortex-a15
> 
> +#ifeq ($(CONFIG_ARM64_V8R),y)
> +    CFLAGS-$(CONFIG_ARM_64) += $(call 
> cc-option,$(CC),-march=armv8-r,-march=armv8.4-a)
> +#endif
> +
> CFLAGS-$(CONFIG_ARM_64) += -mcpu=generic
> CFLAGS-$(CONFIG_ARM_64) += -mgeneral-regs-only # No fp registers etc
> $(call cc-option-add,CFLAGS-$(CONFIG_ARM_64),CC,-mno-outline-atomics)
> ```
> My gcc compiler version is as follows:
> ```
> Using built-in specs.
> COLLECT_GCC=aarch64-linux-gnu-gcc
> COLLECT_LTO_WRAPPER=/usr/lib/gcc-cross/aarch64-linux-gnu/9/lto-wrapper
> Target: aarch64-linux-gnu
> Configured with: ../src/configure -v --with-pkgversion='Ubuntu 
> 9.3.0-17ubuntu1~20.04'
> --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs 
> --enable-languages=c,ada,c++,
> go,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only 
> --program-suffix=-9
> --enable-shared --enable-linker-build-id --libexecdir=/usr/lib 
> --without-included-gettext
> --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ 
> --enable-clocale=gnu
> --enable-libstdcxx-debug --enable-libstdcxx-time=yes 
> --with-default-libstdcxx-abi=new
> --enable-gnu-unique-object --disable-libquadmath 
> --disable-libquadmath-support --enable-plugin
> --enable-default-pie --with-system-zlib --without-target-system-zlib 
> --enable-libpth-m2 --enable-multiarch
> --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release 
> --build=x86_64-linux-gnu
> --host=x86_64-linux-gnu --target=aarch64-linux-gnu 
> --program-prefix=aarch64-linux-gnu- 
> --includedir=/usr/aarch64-linux-gnu/include
> Thread model: posix
> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
> ```
> However. even if my gcc compiler is indeed not supporting "-march=armv8-r", 
> above check will not identify it
> "unrecognized option" and seek to the "-march=armv8.4-a" as expected.
> See the resulting errors, it still uses "-march=armv8-r":
> ```
> make[3]: Entering directory 
> '/home/penny/FVP/r82/fvp_aemv8r_cortexr82_0.0_6347_gcc6.4/xen/xen'
> aarch64-linux-gnu-gcc -MMD -MP -MF ./.asm-offsets.s.d -DBUILD_ID 
> -fno-strict-aliasing -std=gnu99 -Wall
> -Wstrict-prototypes -Wdeclaration-after-statement 
> -Wno-unused-but-set-variable -Wno-unused-local-typedefs
> -O1 -fno-omit-frame-pointer -nostdinc -fno-builtin -fno-common -Werror 
> -Wredundant-decls -Wno-pointer-arith -Wvla
> -pipe -D__XEN__ -include 
> /home/penny/FVP/r82/fvp_aemv8r_cortexr82_0.0_6347_gcc6.4/xen/xen/include/xen/config.h
> -g -march=armv8-r -mgeneral-regs-only -mno-outline-atomics 
> -I/home/penny/FVP/r82/fvp_aemv8r_cortexr82_0.0_6347_gcc6.4
> /xen/xen/include -fno-stack-protector -fno-exceptions 
> -fno-asynchronous-unwind-tables -fcf-protection=none
> -Wnested-externs '-D__OBJECT_FILE__="asm-offsets.s"' -S -g0 -o 
> asm-offsets.s.new -MQ asm-offsets.s arch/arm/arm64/asm-offsets.c
> cc1: error: unknown value 'armv8-r' for '-march'
> cc1: note: valid arguments are: armv8-a armv8.1-a armv8.2-a armv8.3-a 
> armv8.4-a armv8.5-a; did you mean 'armv8-a'?
> make[3]: *** [Makefile:413: asm-offsets.s] Error 1
> ```
> And the reason is that in the implementation of "cc-option": To handle this 
> we do a test compile, passing the option-under-test,
> on a code fragment that will always produce a warning (integer assigned to 
> pointer). We then grep for the option-under-test in
> the compiler's output, the presence of which would indicate an "unrecognized 
> command-line option" warning/error.
> ```
> cc-option = $(shell if test -z "`echo 'void*p=1;' | \
>                      $(1) $(2) -S -o /dev/null -x c - 2>&1 | grep -- $(2) 
> -`"; \
>                      then echo "$(2)"; else echo "$(3)"; fi ;)
> ```
> Here, that is, we are trying to grep "-march=armv8-r" in output message of 
> the test compile, and since the output
> message(unknown value 'armv8-r' for '-march') doesn't contain the COMPLETE 
> string, it thought the compiler supported this
> option, but it actually not...
> 
> Tracing back to the commit about the implementation of "cc-option", it 
> discards the linux method(using exit code to tell) for dealing with
> disable-warning options.
> See 
> https://github.com/xen-project/xen/commit/28f8fb7d2b3fde2f5cbe5526ac4f1c932e3f5d26
>  for more details.
> 
> To fix this issue, I could loose the "grep check", instead trying to grep the 
> COMPLETE option string("-march=armv8-r"), only partial(armv8-r) is
> enough. But, hmmm, the way of greping output message to check is kinds of 
> reliable, IMO, since there is no standard for that.

Since we need to be careful about not relaxing this too much, my
immediate suggestion here would be to split option strings at =
signs, and then use grep to check that every resulting piece is
present in the error message coming from the compiler. Afaict
this should cover your case and might also make a few others
less fragile. Unfortunately I expect this will make for quite a
bit more complicated a make macro. I wonder whether Linux hasn't
run into the same problem at some point, and has found some more
elegant and/or robust solution.

Jan