Re: [MirageOS-devel] tests, coverage, and the modern duniverse

[Hannes Mehnert <hannes@xxxxxxxxxxx>]

Hi Mindy,

On 09/04/2019 01:02, Mindy Preston wrote:
> I recently got inspired to revisit test coverage information generation.
> It seems that the current state of this ecosystem uses `dune` for
> building and some instructions in `preprocess` stanzas to invoke
> `bisect_ppx`, which itself has logic to only produce coverage
> information if an environment variable is set when called in a certain
> mode.
> 
> Unfortunately, while the *invocation* of `bisect_ppx` can be set
> conditionally (see bisect_ppx's instructions[1]) for details), the
> dependency on `bisect_ppx` is unconditional - even if the environment
> variable isn't set, `dune build` will fail if `bisect_ppx` is not
> installed.
> 
> It seems that most projects using `bisect_ppx` use a solution that
> involves some pre-release massaging of `dune` files to remove
> `bisect_ppx` from `preprocess (pps` stanzas, and then release an `opam`
> file that doesn't mention `bisect_ppx`, but keep `bisect_ppx` in their
> repository-local `opam` files.
> 
> I think this kind of workflow is OK for repositories that have one or
> two very involved maintainers, but it seems error-prone for MirageOS
> repositories, where there's a team of maintainers that have varying
> amounts of involvement.  I can very easily imagine myself going to make
> a release of a repository with this strategy and accidentally releasing
> the bisected version.
> 
> I'm interested if anyone has a solution in mind for this that's a bit
> more automatic.

I'm aware of this workaround -
https://github.com/mirage/arp/blob/master/src/dune - which uses
"Jbuild_plugin.V1.send" to evaluate the environment variable
BISECT_ENABLE, and if true, run the bisect_ppx preprocessor. In the opam
file (https://github.com/mirage/arp/blob/master/arp.opam), mentioning
'"bisect_ppx" {with-test}' seems to work fine.

The downside is that if you accidentally have BISECT_ENABLE set to true
and do a release, so it doesn't fully solve your concern above. And a
possible solution I can think of for this is even more ugly, to check
e.g. in dune-release that BISECT_ENABLE is false /o\


Maybe I just need more coffee,

hannes

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/mirageos-devel