Re: [Xen-devel] Automatic dependencies are out of sync

[Juergen Gross <jgross@xxxxxxxx>]

On 05/09/18 16:05, Juergen Gross wrote:
> I think we have a major problem in our build system regarding
> automatic dependencies.
> 
> Starting with a new tree (after git clone or make clean) I have
> no dependency files (*.d2) anywhere:
> 
> $ make clean
> $ find . -name '*.d2' | wc -l
> 0
> 
> Doing "make" will produce only some of them in a very limited number of
> directories:
> 
> $ make
> $ find . -name '*.d2' | wc -l
> 279
> $ find . -name '*.d2' | xargs -n 1 dirname | sort -u
> ./tools/firmware/xen-dir/xen-root/xen/arch/x86
> ./tools/firmware/xen-dir/xen-root/xen/arch/x86/x86_64
> ./tools/firmware/xen-dir/xen-root/xen/common
> ./tools/firmware/xen-dir/xen-root/xen/common/compat
> ./xen/arch/x86
> ./xen/arch/x86/x86_64
> ./xen/common
> ./xen/common/compat
> 
> And only after the next "make" we have all of the *.d2 files available:
> 
> $ make
> $ find . -name '*.d2' | wc -l
> 969
> 
> I guess the reason for that is that the *.d2 files only depend on the
> *.d files which are built together with the *.o files. They are not
> needed for building the product files, so they seem to be always one
> make step behind.
> 
> Now comes the weird part (that's why I started to look into this):
> When I now do:
> 
> $ touch tools/libxc/include/xenctrl.h
> $ make
> 
> I get build failures in tools/tests/depriv (lots of unknown types like
> uint64_t). The .d file in the depriv directory suddenly contains only
> 7 instead of 44 entries, while the .d2 file is still okay (it is one
> make "older", do you remember?):
> 
> $ wc .*.d*
>    7   14  546 .depriv-fd-checker.d
>   44   88 1923 .depriv-fd-checker.d2
> 
> And doing another make will succeed again, but now both dependency files
> are cut down to 7 entries (sure they are: .d2 is rebuilt from .d, while
> .d would only be rebuilt if .o is being built, which isn't happening as
> no file it depends on has changed):
> 
> $ make
> $ wc .*.d*
>   7  14 546 .depriv-fd-checker.d
>   7  14 336 .depriv-fd-checker.d2
> 
> I have no immediate idea how to solve that.

I've setup a little example Makefile solving the problem (just to show
the correct dependencies, needs to be adapted for naming the .d and .d2
files and how to build the .d2):

-->8 snip here 8<--

DEPS := tst.d2

all: tst $(DEPS)

%.d2: %.d
        cp $< $@

%.o %.d: %.c
        gcc -MMD -o $(patsubst %.c,%.o,$<) -c $<

%: %.o
        gcc $< -o $@

-include $(DEPS)

-->8 snip here 8<--

So the basic ideas are:

- add a rule for constructing the .d files
- let the build depend on the .d2 files

I hope I didn't miss anything.

For cases like xen/arch/x86/mm/guest_walk_*.o obviously some special
handling is needed, but this should be rather straightforward.


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel