Re: [MirageOS-devel] code coverage testing with Bisect

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

I'm just wondering where the best place to document this sort of stuff is.  We have all sorts of black magic for OASIS, OPAM, C stub compilation and now code coverage.

The medium term plan is for the Assemblage tool to take on all of this, but would stashing this stuff on the Mirage wiki help instead?  I'm happy to kick off the structure and cut-and-paste the information in.  I'm not proposing the OPAM wiki since some of this is Mirage-specific, such as how to get C stubs cross-compiling between Xen and Unix.

-anil

On 12 Mar 2015, at 12:51, Luke Dunstan <lukedunstan81@xxxxxxxxx> wrote:

I had the same idea with sed on _oasis, and coveralls: https://github.com/mirage/ocaml-dns/issues/33

Then I read this article that Anil tweeted: http://www.hammerlab.org/2015/03/04/creating-ocaml-code-coverage-reports-for-ketrew/

Based on a modified version of that "myocamlbuild.ml", I got it working without sed: https://github.com/infidel/ocaml-mdns/blob/master/myocamlbuild.ml

Luke

On 12 March 2015 at 17:34, David Scott <scott.dj@xxxxxxxxx> wrote:

Hi,

I've been having a lot of fun with bisect[1] recently-- I've been using it to see which parts of the mirage-block-volume[2] and shared-block-ring[3] libraries are completely untouched by their unit tests. I found the following game to be quite addictive:

1. "make coverage", and load result in web-browser

2. spot a big chunk of obvious red (danger, danger, danger)

3. (thinking carefully about what could go wrong) devise an interesting test to stress the red bits (obviously you could cover it with a 'noddy' test but there is probably no point)

4. "make coverage", reload in browser and see the red go green!

I've hooked it up with coveralls.io (admittedly in a bit of a hacky way[4]) such that the master branch is firmly in "development" mode, linking against bisect, and without checking in the oasis autogen rubbish. The .travis.yml runs both the travisci-skeleton script and then invokes ocveralls[5] to upload the results.

I've added a separate "make release" step which removes bisect and checks in the autogen (presumably into a release branch). Perhaps eventually this could make a github pull request (with the "hub" tool?) and make an opam package?

I think the game is made even more addictive when the coveralls badge changes colour, see:

https://coveralls.io/r/mirage/mirage-block-volume?branch=master

Here's an example bisect report (the code is a work-in-progress):

http://dave.recoil.org/tmp/report/file0000.html

If you haven't given bisect a go -- I recommend playing with it.

Also, if you can think of a nicer way to integrate this -- I admit using 'sed' on the _oasis file is a bit of a hack -- please let me know!

Cheers,

Dave

[1] http://bisect.x9c.fr

[2] https://github.com/mirage/mirage-block-volume

[3] https://github.com/mirage/shared-block-ring

[4] https://github.com/mirage/shared-block-ring/commit/67b9f3100be8e4e9732dd79b7c1cc5352a61d478

[5] https://github.com/sagotch/ocveralls

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel