Re: [Xen-devel] [PATCH] x86emul: fix test harness and fuzzer build dependencies

[Ian Jackson <ian.jackson@xxxxxxxxxx>]

Jan Beulich writes ("Re: [PATCH] x86emul: fix test harness and fuzzer build 
dependencies"):
> On 20.12.18 at 15:46, <ian.jackson@xxxxxxxxxx> wrote:
> > I think this introduces a `make -j' hazard ?  The problem is that this
> > branch of the Makefiles tree might enter tools/include while
> > another branch is also doing so, resulting in two simultaneous
> > executions in the same directory.
> 
> What is "another branch" here? So far I was under the impression
> that the ability of building x86 emulator fuzzer and test harness
> independently is an exception, and that all other parts of the
> tools/ subtree are supposed to be built by going through the top
> level. Otherwise further dependency issues might arise, due to
> top level Makefile's %-tools-public-headers rule.
> 
> Hence whether there's a "make -j" hazard here depends on what
> that top level rule's purpose is.

I don't follow.

The top-level %-tools-public-headers rule is there to be something
that you can write in the dependencies of other subdirs, to arrange
that $(MAKE) -C tools/include is run before that other subdir.

Ie, it is there to satisfy the requirement I mention above, that the
dependency directory is built first.


If one wants to be build the x86 emulator fuzzer but not rebuild other
things, it is OK to run make in just that subdirectory.  That is a
general rule with recursive make: you can run make in a subdirectory,
at the cost of perhaps not rebuilding everything that was changed.

But when one does that, one must have built the rest of the things
already because otherwise the x86emul directory has to $(MAKE) -C back
to tools/include, rentering tools/include.

In the general case if one is changing things in different places and
rebuilding frequently, one may have to
   make -C place1 && make -C place2
in one's ad-hoc command line.

On that basis,

    On 14/12/2018 08:49, Jan Beulich wrote:
    > Commit fd35f32b4b ("tools/x86emul: Use struct cpuid_policy in
    > the userspace test harnesses") didn't account for the
    > dependencies of cpuid-autogen.h to potentially change between
    > incremental builds.  Putting the make invocation to produce the
    > header together with the directory tree creation therefore does
    > not work. Introduce a separate goal.

I think that may have been a misconceived attempt to improve on the
usual UI rule for recursive make, which I describe above.


I wrote:

   (It is possible to violate this rule without creating races but it
  is tricky and inadvisable.)

If we are determined that it must be possible to run make in the x86
emulator fuzzer directory *without having previously built the rest of
the tree normally*, then perhaps it is necessary to do this
$(MAKE) -C thing.

But in that case we need to make sure that either:

 A. 1. The top-level Makefiles ensure that *a* build of
       tools/include completes *before* starting to enter
       tools/fuzz/x86_instruction_emulator.  (Which I
       think is the case.)

    AND

    2. make -C tools/include is definitely completely read-only if the
       thing has already been built.  (This is somewhat hard to check
       and maintain, and would need a comment in that Makefile to
       ensure that this property is preserved.)

 OR

 B. The tools/include Makefile gains explicit synchronisation; it
    would have to re-invoke itself.  I have never tries this, but it
    seems like it would be possible.  We would gain a new built-time
    dependency on a shell synchronisation/locking utility which would
    perhaps not be available on ten-year-old enterprise Linuces.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel