Re: [Xen-devel] Automatic dependencies are out of sync

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 06.09.18 at 09:34, <jgross@xxxxxxxx> wrote:
> On 05/09/18 16:05, Juergen Gross wrote:
>> I think we have a major problem in our build system regarding
>> automatic dependencies.
>> 
>> Starting with a new tree (after git clone or make clean) I have
>> no dependency files (*.d2) anywhere:
>> 
>> $ make clean
>> $ find . -name '*.d2' | wc -l
>> 0
>> 
>> Doing "make" will produce only some of them in a very limited number of
>> directories:
>> 
>> $ make
>> $ find . -name '*.d2' | wc -l
>> 279
>> $ find . -name '*.d2' | xargs -n 1 dirname | sort -u
>> ./tools/firmware/xen-dir/xen-root/xen/arch/x86
>> ./tools/firmware/xen-dir/xen-root/xen/arch/x86/x86_64
>> ./tools/firmware/xen-dir/xen-root/xen/common
>> ./tools/firmware/xen-dir/xen-root/xen/common/compat
>> ./xen/arch/x86
>> ./xen/arch/x86/x86_64
>> ./xen/common
>> ./xen/common/compat
>> 
>> And only after the next "make" we have all of the *.d2 files available:
>> 
>> $ make
>> $ find . -name '*.d2' | wc -l
>> 969
>> 
>> I guess the reason for that is that the *.d2 files only depend on the
>> *.d files which are built together with the *.o files. They are not
>> needed for building the product files, so they seem to be always one
>> make step behind.
>> 
>> Now comes the weird part (that's why I started to look into this):
>> When I now do:
>> 
>> $ touch tools/libxc/include/xenctrl.h
>> $ make
>> 
>> I get build failures in tools/tests/depriv (lots of unknown types like
>> uint64_t). The .d file in the depriv directory suddenly contains only
>> 7 instead of 44 entries, while the .d2 file is still okay (it is one
>> make "older", do you remember?):
>> 
>> $ wc .*.d*
>>    7   14  546 .depriv-fd-checker.d
>>   44   88 1923 .depriv-fd-checker.d2
>> 
>> And doing another make will succeed again, but now both dependency files
>> are cut down to 7 entries (sure they are: .d2 is rebuilt from .d, while
>> .d would only be rebuilt if .o is being built, which isn't happening as
>> no file it depends on has changed):
>> 
>> $ make
>> $ wc .*.d*
>>   7  14 546 .depriv-fd-checker.d
>>   7  14 336 .depriv-fd-checker.d2
>> 
>> I have no immediate idea how to solve that.
> 
> I've setup a little example Makefile solving the problem (just to show
> the correct dependencies, needs to be adapted for naming the .d and .d2
> files and how to build the .d2):
> 
> -->8 snip here 8<--
> 
> DEPS := tst.d2
> 
> all: tst $(DEPS)

-include $(DEPS) already ought to have the effect of such a dependency,
since all makefiles are checked for rules of how to re-make them.

> %.d2: %.d
>         cp $< $@

Such a rule already exists in ./Config.mk.

> %.o %.d: %.c
>         gcc -MMD -o $(patsubst %.c,%.o,$<) -c $<

Doesn't this result in gcc to be invoked twice, perhaps resulting in
corrupt .o and/or .d? I think %.d wants to depend on %.o, without
a command.

> %: %.o
>         gcc $< -o $@
> 
> -include $(DEPS)
> 
> -->8 snip here 8<--
> 
> So the basic ideas are:
> 
> - add a rule for constructing the .d files
> - let the build depend on the .d2 files

IOW I wonder whether this really is any different from what we
do now (minus bugs/quirks in make itself, of course). And from this
as well as your original mail I still don't understand what's actually
broken with the current approach.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel