Re: [MirageOS-devel] tests, coverage, and the modern duniverse

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

On 9 Apr 2019, at 00:02, Mindy Preston <mindy@xxxxxxxxxxxxxxxxxxx> wrote:
> 
> Hi all,
> 
> I recently got inspired to revisit test coverage information generation. It 
> seems that the current state of this ecosystem uses `dune` for building and 
> some instructions in `preprocess` stanzas to invoke `bisect_ppx`, which 
> itself has logic to only produce coverage information if an environment 
> variable is set when called in a certain mode.
> 
> Unfortunately, while the *invocation* of `bisect_ppx` can be set 
> conditionally (see bisect_ppx's instructions[1]) for details), the dependency 
> on `bisect_ppx` is unconditional - even if the environment variable isn't 
> set, `dune build` will fail if `bisect_ppx` is not installed.
> 
> It seems that most projects using `bisect_ppx` use a solution that involves 
> some pre-release massaging of `dune` files to remove `bisect_ppx` from 
> `preprocess (pps` stanzas, and then release an `opam` file that doesn't 
> mention `bisect_ppx`, but keep `bisect_ppx` in their repository-local `opam` 
> files.
> 
> I think this kind of workflow is OK for repositories that have one or two 
> very involved maintainers, but it seems error-prone for MirageOS 
> repositories, where there's a team of maintainers that have varying amounts 
> of involvement.  I can very easily imagine myself going to make a release of 
> a repository with this strategy and accidentally releasing the bisected 
> version.
> 
> I'm interested if anyone has a solution in mind for this that's a bit more 
> automatic.
> 

This is indeed pending a fix in Dune; see 
https://github.com/ocaml/dune/issues/57

I’ve copied Rudi in case he can comment on any blockers from the dune end and 
whether this is a candidate to target for dune 1.10 (the 1.9 release is just 
about ready to go out of the door in the next few days).

regards,
Anil


_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/mirageos-devel