Re: [MirageOS-devel] Rust unikernels

[Geoffroy Couprie <contact@xxxxxxxxxxxxxxxxxxx>]

Hi Thomas,

On Fri, Jul 24, 2015 at 3:02 PM, Thomas Leonard <talex5@xxxxxxxxx> wrote:
> On 24 July 2015 at 13:00, Geoffroy Couprie <contact@xxxxxxxxxxxxxxxxxxx> 
> wrote:
>> Hello!
>>
>> A chat on Twitter showed some interest in unikernels developed in
>> Rust, and I was pointed here for further discussion.
>>
>> First, I want to acknowledge the amazing work done on MirageOS by
>> everybody here. I have followed it from afar for some time, and there
>> has been great progress to make it available and usable.
>>
>> Second, I know making a unikernel system means years of work and a
>> team of developers and testers. I am not planning to attempt it alone.
>> What I want is to sort out the skills needed, the big pitfalls, the
>> important milestones, and know about the unknown unknowns. Then push
>> for that project in the Rust community.
>>
>> From the beginning, people experimented with OS development in Rust
>> (cf https://github.com/ryanra/RustOS
>> https://github.com/thepowersgang/rust-barebones-kernel or
>> https://github.com/charliesome/rustboot ). There is also a very active
>> IRC channel, #rust-osdev on irc.mozilla.org
>>
>> From what I understand, the bare minimum to experiment would be an OS
>> booting and communicating with a network card, scheduling and task
>> switching, remote debugging, a network stack. Also, tools to help in
>> building, deploying and testing applications.
>> The ryanra/RustOS project seems well advanced, but it could use a network 
>> stack.
>
> Hi Geoffroy,
>
> Have you thought about running Mirage on top of a minimal Rust OS
> kernel to start with? Then you could implement new drivers in Rust
> alongside existing Mirage code. As a garbage-collected language, OCaml
> can't be used to implement the very lowest levels of Mirage (e.g.
> malloc) and we currently use (my branch of) Mini-OS [1] for running as
> a unikernel on Xen. A common question is whether we could replace this
> C code with Rust. I'd be interested to help out if so - I've been
> meaning to take another look at Rust now it's stable.

This is interesting, someone else proposed me that today too. This
would be a good strategy to get something up and running quickly
enough. Basically, Rust can build to a static library which can be
linked to C code, but until there's a memory allocator available,
things can be tricky.
This is how RustOS does it: a small part written in C, which links to
a Rust library that will handle the rest.

The core parts of MiniOS could probably not be reimplemented right
away, but drivers would be doable. From what I see, the first step is
to communicate with the Xenbus and the Xenstore, right? Then have some
ring buffers and a way for xen to signal that some data is available
(I'm currently looking at netfront.c).

>
>> A rough plan could be to take that project and:
>> - strenghten the rtl8139 driver
>
> If you target running under Xen, you only need to provide a simple Xen
> network driver, whatever the physical hardware is. Likewise for
> storage. This makes it easier to get started, and you'll support EC2,
> etc.

This makes sense. Writing drivers directly for hardware is a huge
task, while hypervisors provide standardised interfaces.
>
>> - write parsers and serializers for ARP, IP, TCP (it looks like
>> MirageOS does not even need DHCP, since the IP address is passed at
>> compilation time, is that right?).
>
> Mirage can use DHCP or static addresses.
>
>> This is something I am already
>> pushing for a parser combinators project I launched (
>> https://github.com/Geal/nom )
>> - building the API above that network stack
>> - the system can be monotask at first, but preemtive threading should
>> happen at some point
>
> If you have promises (and make all your low-level APIs non-blocking)
> then you can get a very long way without preemptive threading,
> although it would be good to have it eventually.

Could you tell me more about this? This approach looks interesting.

>
>> - a part of Rust standard library is already available, I do not know
>> how much is needed to support the use case of a HTTP server
>> - building monitoring tools inside the kernel
>
> When running under Xen, we share a trace buffer with dom0. This allows
> you to see what the system is doing, even if it is hung or crashed.
> See: https://mirage.io/wiki/profiling

That visualization tool looks useful! Do you have remote debuggers
too? With a serial port or something similar.

I guess the first thing I could do is to integrate some Rust code in
MiniOS. Is there a part of the code you would recommend? Something
well contained, without too many dependencies?

Thank you for your help!

Cheers,

Geoffroy Couprie

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel