Re: [XEN PATCH 1/4] build: xen/include: use if_changed

[Bertrand Marquis <Bertrand.Marquis@xxxxxxx>]

Hi,

> On 9 Jun 2022, at 13:16, Jan Beulich <jbeulich@xxxxxxxx> wrote:
> 
> On 09.06.2022 13:51, Bertrand Marquis wrote:
>>> On 9 Jun 2022, at 11:26, Jan Beulich <jbeulich@xxxxxxxx> wrote:
>>> On 09.06.2022 12:16, Bertrand Marquis wrote:
>>>>> On 1 Jun 2022, at 17:59, Anthony PERARD <anthony.perard@xxxxxxxxxx> wrote:
>>>>> 
>>>>> Use "define" for the headers*_chk commands as otherwise the "#"
>>>>> is interpreted as a comment and make can't find the end of
>>>>> $(foreach,).
>>>>> 
>>>>> Adding several .PRECIOUS as without them `make` deletes the
>>>>> intermediate targets. This is an issue because the macro $(if_changed,)
>>>>> check if the target exist in order to decide whether to recreate the
>>>>> target.
>>>>> 
>>>>> Removing the call to `mkdir` from the commands. Those aren't needed
>>>>> anymore because a rune in Rules.mk creates the directory for each
>>>>> $(targets).
>>>>> 
>>>>> Remove "export PYTHON" as it is already exported.
>>>> 
>>>> With this change, compiling for x86 is now ending up in:
>>>> CHK     include/headers99.chk
>>>> make[9]: execvp: /bin/sh: Argument list too long
>>>> make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
>>>> 
>>>> Not quite sure yet why but I wanted to signal it early as other might be 
>>>> impacted.
>>>> 
>>>> Arm and arm64 builds are not impacted.
>>> 
>>> Hmm, that patch has passed the smoke push gate already, so there likely is
>>> more to it than there being an unconditional issue. I did build-test this
>>> before pushing, and I've just re-tested on a 2nd system without seeing an
>>> issue.
>> 
>> I have the problem only when building using Yocto, I did a normal build and 
>> the
>> issue is not coming.
>> 
>> Doing a verbose compilation I have this (sorry for the long lines):
>> 
>> for i in include/public/vcpu.h include/public/errno.h include/public/kexec.h 
>> include/public/argo.h include/public/xen.h include/public/nmi.h 
>> include/public/xencomm.h include/public/xenoprof.h 
>> include/public/device_tree_defs.h include/public/version.h 
>> include/public/memory.h include/public/features.h include/public/sched.h 
>> include/public/xen-compat.h include/public/callback.h 
>> include/public/vm_event.h include/public/grant_table.h 
>> include/public/physdev.h include/public/tmem.h include/public/hypfs.h 
>> include/public/platform.h include/public/pmu.h include/public/elfnote.h 
>> include/public/trace.h include/public/event_channel.h 
>> include/public/io/vscsiif.h include/public/io/kbdif.h 
>> include/public/io/protocols.h include/public/io/ring.h 
>> include/public/io/displif.h include/public/io/fsif.h 
>> include/public/io/blkif.h include/public/io/console.h 
>> include/public/io/sndif.h include/public/io/fbif.h 
>> include/public/io/libxenvchan.h include/public/io/netif.h 
>> include/public/io/usbif.h include/public/io/pciif.h 
>> include/public/io/tpmif.h include/public/io/xs_wire.h 
>> include/public/io/xenbus.h include/public/io/cameraif.h 
>> include/public/hvm/pvdrivers.h include/public/hvm/e820.h 
>> include/public/hvm/hvm_xs_strings.h include/public/hvm/dm_op.h 
>> include/public/hvm/ioreq.h include/public/hvm/hvm_info_table.h 
>> include/public/hvm/hvm_vcpu.h include/public/hvm/hvm_op.h 
>> include/public/hvm/params.h; do x86_64-poky-linux-gcc  
>> --sysroot=/home/bermar01/Development/xen-dev/build/profile-qemu-x86_64.prj/tmp/work/core2-64-poky-linux/xen/4.17+git1-r0/recipe-sysroot
>>   -x c -ansi -Wall -Werror -include stdint.h -S -o /dev/null $i || exit 1; 
>> echo $i; done >include/headers.chk.new; mv include/headers.chk.new 
>> include/headers.chk
>> |       rm -f include/headers99.chk.new;  echo "#include 
>> "\"include/public/io/9pfs.h\" | x86_64-poky-linux-gcc  
>> --sysroot=/home/bermar01/Development/xen-dev/build/profile-qemu-x86_64.prj/tmp/work/core2-64-poky-linux/xen/4.17+git1-r0/recipe-sysroot
>>   -x c -std=c99 -Wall -Werror -include stdint.h  -include string.h -S -o 
>> /dev/null - || exit $?; echo include/public/io/9pfs.h >> 
>> include/headers99.chk.new;  echo "#include "\"include/public/io/pvcalls.h\" 
>> | x86_64-poky-linux-gcc  
>> --sysroot=/home/bermar01/Development/xen-dev/build/profile-qemu-x86_64.prj/tmp/work/core2-64-poky-linux/xen/4.17+git1-r0/recipe-sysroot
>>   -x c -std=c99 -Wall -Werror -include stdint.h  -include string.h -S -o 
>> /dev/null - || exit $?; echo include/public/io/pvcalls.h >> 
>> include/headers99.chk.new; mv include/headers99.chk.new include/headers99.chk
>> | make[9]: execvp: /bin/sh: Argument list too long
>> | make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
>> | make[9]: *** Waiting for unfinished jobs....
>> 
>> So the command passed to the sub shell by make is quite long.
>> 
>> No idea why this comes out only when building in Yocto but I will dig a bit.
> 
> Maybe Yocto has an unusually low limit on command arguments' total size?
> The whole thing is just over 2500 chars, which doesn't look to be unusually
> long for Unix-like environments.
> 

Actually the command to generate headers++.chk is 15294 characters when I build 
xen normally.
In Yocto it becomes a lot bigger as CC includes a sysroot parameter with a path.

Bertrand