[MirageOS-devel] [RFC] TCP/IP stack MTU, layering, and write allocation discipline changes

[Hannes Mehnert <hannes@xxxxxxxxxxx>]

Dear hackers,

now that MirageOS 3.4.1 is released, which separated the ethernet and
the arp layer into distinct opam packages and git repositories, I hereby
propose changes to the layering of the TCP/IP stack in MirageOS. The
motivation behind this is further decoupling of the layers to support
802.1q etc. more easily. The changes do not modify the receive path.

A brief list of changes:

- Ethernet encapsulation is done in Ethernet - currently done in Arp and
IPv4 and IPv6 libraries
- MTU is passed from the network interface upwards - currently passed
into the Ethernet layer
- Write buffer allocation is done by the network interface, which knows
best about page boundary restrictions, etc. - currently done in IPv4,
IPv6, ARP using Io_page.get 1
- write functions signature change throughout the stack up to Ip: the
buffer to send is no longer given directly, instead a size argument and
a fill function of type (buffer -> int).
- IPv4 applies fragmentation!
- no longer used function are removed


I did performance evaluation as follows (I'm pretty sure this is not
exhaustive, and likely others have better ideas on how to actually do
such a benchmark):
- Server: the MirageOS unikernel iperf_server (TCP, thanks to Taka),
https://github.com/hannesm/mirage_iperf, virtualized with Solo5/hvt,
deployed on a PC Engines (AMD Embedded G series GX-412TC, 1 GHz quad
Jaguar core)
- Client: iperf on Unix running on my Lenovo X250, arguments -i 1 -t 20
- Connection was a 1 GBit/s copper cable (a switch was in between the
devices)

The latest released opam packages (mirage 3.4.1, tcpip 3.7.0):
Throughput = 16.97 [MBs/sec]
gc minor words 214425271.000000 promoted words 4421898.000000 major
words 4428911.000000 minor collections 884 major collections 131
heap_words 61440

With the changes above:
Throughput = 16.85 [MBs/sec]
gc minor words 212194711.000000 promoted words 4286488.000000 major
words 4293501.000000 minor collections 876 major collections 132
heap_words 61440


These changes already span over several opam packages, a PR that tracks
all changes is https://github.com/mirage/mirage/pull/968. Initial
discussion of the adjusted mirage-net interface is at
https://github.com/mirage/mirage-net/pull/13 (now superseeded).

A consistent repository of opam packages is available at
https://github.com/mirage/mirage-dev#layering for your convenience (you
can test this by opam repo add layering <url> followed by opam install
mirage).

A CI run using mirage-skeleton has been completed successfully at
https://travis-ci.org/mirage/mirage-dev/builds/491058735

I'd be happy to get these PRs merged rather sooner than later. If you're
interested in the API changes, please take a look at the changes to
mirage-net and mirage-protocols.
The mirage-net-xen changes have not been tested (they compile, though).
It also lacks a way to retrieve the MTU from Xenstore.
I extended tuntap to provide the MTU information, and mirage-net-unix to
use that information.


best,

hannes

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/mirageos-devel