[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[MirageOS-devel] rfc: irmin + mirageos talk submission

Hi folks,

Some of us have put together a talk submission for the OCaml Workshop in September (alongside ICFP). In the ~20 minutes remaining before the submission deadline, I'd be happy to hear anyone's feedback on the proposal.

General thoughts on the subject would also be welcome, as we have lots of time to explore this space and think about what to address in more depth if the talk is accepted :)

Proposal below:

It is traditionally challenging to build and debug reliable distributed applications due to the difficulty of reasoning about the ordering and failure modes of remote communications. In this talk, we will describe an alternative to network simulators and emulators by using several OCaml libraries to build a persistent, interactive network stack from off-the-shelf OCaml libraries that are present in OPAM.

Irmin (the database that never forgets) presents users with an API allowing DVCS-like access to ordinary program data [1]. Using Irmin, we can construct data structures that are commonly found in network protocols (such as distributed hash tables), and manipulate them across nodes via DVCS pull/push/merge operations.

MirageOS [2] is a library operating system written in OCaml where all structures are ordinary program data, even ones that normally live deep within the kernel in traditional architectures. Mirage provides a network stack that provides a complete TCP/IP implementation, including Ethernet, ARP, ICMP, IP, UDP, TCP, and DHCP.

We combine both Irmin and the existing MirageOS stack to build a persistent network stack. With Irmin, we can manage updates for structures like the ARP cache, which is mutated by both incoming network traffic and the passage of time, with clear semantics, history, and conflict resolution. It also becomes possible, through Irmin's sync interface or Git filesystem backend, to make this data persistent in a consistent and convenient way.

Using Irmin's Git filesystem backend, it's also possible to interact with this data with Git tools. One can, for example, examine the history of the cache with `gitk` and observe whether an expiration thread is correctly removing old entries on its branch. It's even possible to make edits to the serialized cache data and commit them from the command line, resulting in live changes to the cache!

From an implemented example of an ARP cache, we'll motivate further possibilities in more complicated protocols and their related data stores, including TCP and its associated state. We will also discuss potential future uses in the context of many MirageOS unikernels running in a hypervisor, including coordination that traditionally uses broadcast traffic across a local area network.

A fuller blog post series on this work is available online [3].

[1] http://irmin.io
[2] https://mirage.io
[3] http://somerandomidiot.com/blog/2015/04/24/what-a-distributed-version-controlled-ARP-cache-gets-you/


MirageOS-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.