[MirageOS-devel] Notes on testing

[Richard Mortier <richard.mortier@xxxxxxxxxxxx>]

A little late, but finally got round to writing up some notes on the
conversation a couple of weeks ago on how we might approach testing in
Mirage.

"""
# Notes

+ Testing is used for a variety of purposes. Of note for us are:
reproduction of reported issues; regression testing of previously
fixed issues; unit testing within a component; and continuous
integration testing of an entire component

+ Perhaps the key problem at this stage is that the stack does not appear to
have an explicit design guideline: it is unclear whether it is intended to be
copying/non-copying; to accept and provide aligned/unaligned buffers to/from
applications; to be blocking/non-blocking; or buffer ownership policy.

  + Confusion was expressed over whether `NetIf.write` is intended to
assemble a list of buffer fragments or transmit a list of complete
packet buffers. Magnus confirms: `NetIf.writev` assembles a list of
buffer fragments to a new buffer, `NetIf.write` just writes a buffer.
I think `Netif` is the module that is currently responsible for
assembling in `writev`, as the list of buffers seems to be just passed
on to `Netif` in the other modules.

+ There is evidence that at least some parts of the stack currently
require blocking behaviour, eg., `tcpip` deadlocks if blocking is not
introduced -- from Magnus: "mirage-tcpip requires that Netif calls the
listener callback with Lwt.async/ignore_result, as tcpip blocks in
some of the listener threads while it is waiting for more data (e.g.
for an mvar to be updated)". `NetIf` and `Io-page.1.3.0` recently
changed API to always copy data.

+ Longer term there is a perceived need to start splitting protocol
libraries to provide separate front- and back-end implementations
following the practice in `vchan`. Separation of `tcpip` into several
submodules that permit independent testing (`SlidingWindow`, etc) was
also recommended.

+ Inter-layer testing is a pain on Travis but may be ultimately
supported using `VNetIf` and a front/backend separation. Also required
for regular performance testing so we can build `is-mirage-fast`.

  + From Magnus: "mirage-vnetif has now been released in opam and
tests for connect and iperf in travis have been added to tcpip:

https://github.com/mirage/mirage-tcpip/tree/master/lib_test

I've also added a backend to test trailing bytes. The tests can be run
in alcotest verbose mode to see the measured speed."

+ Similarly, it was also suggested that a non-blocking API is
ultimately preferable as it provides more explicit control over queues
and pushback throughout stack, vs the current practice where listen
threads accumulate data and eventually return.

# Actions

[djs]: Tell us where to look in the mirage-xen front/backend code, and
to write down what actually happens there. Find out about XenRT
graphs.

[magnus]: Document `NetIf`. Send link to blog post network setup; verify iperf
over `VNetIf` works. Send pointers to tests for NSDI/Jitsu.

  + "I've added documentation and examples for the mirage-vnetif
network setup in the README:
https://github.com/MagnusS/mirage-vnetif/blob/master/README.md Tests
for jitsu/synjitsu are under development ... but not quite there yet
:-) We are tracking progress here:
https://github.com/mirage/jitsu/issues/11";

[mort]: Document that `mirage-net-pcap` should be used to gather
`.pcap` files for a bug report. File issue on `ocaml-vchan` for
@samoht to use alcotest. File issue for `TROVE` to indicate tests and
passes, Travis status, and maintainer.

[masoud]: Setup to run perf tests daily and automatically report findings.

[thomas]: Create `ocaml-test-skeleton` basing off "best practice" in
vchan/ alcotest/ocaml-irmin.

[mindy]: Build a simple example of testing for a single-layer of the
stack (ARP) using Irmin.
"""

-- 
Richard Mortier
richard.mortier@xxxxxxxxxxxx

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel