[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[MirageOS-devel] [RFC] TCP/IP stack MTU, layering, and write allocation discipline changes

Dear hackers,

now that MirageOS 3.4.1 is released, which separated the ethernet and
the arp layer into distinct opam packages and git repositories, I hereby
propose changes to the layering of the TCP/IP stack in MirageOS. The
motivation behind this is further decoupling of the layers to support
802.1q etc. more easily. The changes do not modify the receive path.

A brief list of changes:

- Ethernet encapsulation is done in Ethernet - currently done in Arp and
IPv4 and IPv6 libraries
- MTU is passed from the network interface upwards - currently passed
into the Ethernet layer
- Write buffer allocation is done by the network interface, which knows
best about page boundary restrictions, etc. - currently done in IPv4,
IPv6, ARP using Io_page.get 1
- write functions signature change throughout the stack up to Ip: the
buffer to send is no longer given directly, instead a size argument and
a fill function of type (buffer -> int).
- IPv4 applies fragmentation!
- no longer used function are removed

I did performance evaluation as follows (I'm pretty sure this is not
exhaustive, and likely others have better ideas on how to actually do
such a benchmark):
- Server: the MirageOS unikernel iperf_server (TCP, thanks to Taka),
https://github.com/hannesm/mirage_iperf, virtualized with Solo5/hvt,
deployed on a PC Engines (AMD Embedded G series GX-412TC, 1 GHz quad
Jaguar core)
- Client: iperf on Unix running on my Lenovo X250, arguments -i 1 -t 20
- Connection was a 1 GBit/s copper cable (a switch was in between the

The latest released opam packages (mirage 3.4.1, tcpip 3.7.0):
Throughput = 16.97 [MBs/sec]
gc minor words 214425271.000000 promoted words 4421898.000000 major
words 4428911.000000 minor collections 884 major collections 131
heap_words 61440

With the changes above:
Throughput = 16.85 [MBs/sec]
gc minor words 212194711.000000 promoted words 4286488.000000 major
words 4293501.000000 minor collections 876 major collections 132
heap_words 61440

These changes already span over several opam packages, a PR that tracks
all changes is https://github.com/mirage/mirage/pull/968. Initial
discussion of the adjusted mirage-net interface is at
https://github.com/mirage/mirage-net/pull/13 (now superseeded).

A consistent repository of opam packages is available at
https://github.com/mirage/mirage-dev#layering for your convenience (you
can test this by opam repo add layering <url> followed by opam install

A CI run using mirage-skeleton has been completed successfully at

I'd be happy to get these PRs merged rather sooner than later. If you're
interested in the API changes, please take a look at the changes to
mirage-net and mirage-protocols.
The mirage-net-xen changes have not been tested (they compile, though).
It also lacks a way to retrieve the MTU from Xenstore.
I extended tuntap to provide the MTU information, and mirage-net-unix to
use that information.



MirageOS-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.