[MirageOS-devel] tests, coverage, and the modern duniverse

[Mindy Preston <mindy@xxxxxxxxxxxxxxxxxxx>]

Hi all,

I recently got inspired to revisit test coverage information generation. 
It seems that the current state of this ecosystem uses `dune` for 
building and some instructions in `preprocess` stanzas to invoke 
`bisect_ppx`, which itself has logic to only produce coverage 
information if an environment variable is set when called in a certain mode.
Unfortunately, while the *invocation* of `bisect_ppx` can be set 
conditionally (see bisect_ppx's instructions[1]) for details), the 
dependency on `bisect_ppx` is unconditional - even if the environment 
variable isn't set, `dune build` will fail if `bisect_ppx` is not installed.
It seems that most projects using `bisect_ppx` use a solution that 
involves some pre-release massaging of `dune` files to remove 
`bisect_ppx` from `preprocess (pps` stanzas, and then release an `opam` 
file that doesn't mention `bisect_ppx`, but keep `bisect_ppx` in their 
repository-local `opam` files.
I think this kind of workflow is OK for repositories that have one or 
two very involved maintainers, but it seems error-prone for MirageOS 
repositories, where there's a team of maintainers that have varying 
amounts of involvement.  I can very easily imagine myself going to make 
a release of a repository with this strategy and accidentally releasing 
the bisected version.
I'm interested if anyone has a solution in mind for this that's a bit 
more automatic.
[1] https://github.com/aantron/bisect_ppx#instructions

(Much of the text of this message is copied from a comment on 
https://github.com/mirage/mirage-tcpip/issues/160 , filed in 2015 and 
maybe due for a resolution...)
-Mindy


_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/mirageos-devel