Re: [MirageOS-devel] Rust unikernels

[Thomas Leonard <talex5@xxxxxxxxx>]

On 27 July 2015 at 15:54, Thomas Leonard <talex5@xxxxxxxxx> wrote:
> On 27 July 2015 at 09:36, Geoffroy Couprie <contact@xxxxxxxxxxxxxxxxxxx> 
> wrote:
>> Hi,
>>
>> On Sat, Jul 25, 2015 at 12:55 PM, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>>> On 24 July 2015 at 23:28, Geoffroy Couprie <contact@xxxxxxxxxxxxxxxxxxx> 
>>> wrote:
>>>> Hi Thomas,
>>>>
>>>> On Fri, Jul 24, 2015 at 3:02 PM, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>>>>> On 24 July 2015 at 13:00, Geoffroy Couprie <contact@xxxxxxxxxxxxxxxxxxx> 
>>>>> wrote:
>>>>>> Hello!
>>>>>>
>>>>>> A chat on Twitter showed some interest in unikernels developed in
>>>>>> Rust, and I was pointed here for further discussion.
>>>>>>
>>>>>> First, I want to acknowledge the amazing work done on MirageOS by
>>>>>> everybody here. I have followed it from afar for some time, and there
>>>>>> has been great progress to make it available and usable.
>>>>>>
>>>>>> Second, I know making a unikernel system means years of work and a
>>>>>> team of developers and testers. I am not planning to attempt it alone.
>>>>>> What I want is to sort out the skills needed, the big pitfalls, the
>>>>>> important milestones, and know about the unknown unknowns. Then push
>>>>>> for that project in the Rust community.
>>>>>>
>>>>>> From the beginning, people experimented with OS development in Rust
>>>>>> (cf https://github.com/ryanra/RustOS
>>>>>> https://github.com/thepowersgang/rust-barebones-kernel or
>>>>>> https://github.com/charliesome/rustboot ). There is also a very active
>>>>>> IRC channel, #rust-osdev on irc.mozilla.org
>>>>>>
>>>>>> From what I understand, the bare minimum to experiment would be an OS
>>>>>> booting and communicating with a network card, scheduling and task
>>>>>> switching, remote debugging, a network stack. Also, tools to help in
>>>>>> building, deploying and testing applications.
>>>>>> The ryanra/RustOS project seems well advanced, but it could use a 
>>>>>> network stack.
>>>>>
>>>>> Hi Geoffroy,
>>>>>
>>>>> Have you thought about running Mirage on top of a minimal Rust OS
>>>>> kernel to start with? Then you could implement new drivers in Rust
>>>>> alongside existing Mirage code. As a garbage-collected language, OCaml
>>>>> can't be used to implement the very lowest levels of Mirage (e.g.
>>>>> malloc) and we currently use (my branch of) Mini-OS [1] for running as
>>>>> a unikernel on Xen. A common question is whether we could replace this
>>>>> C code with Rust. I'd be interested to help out if so - I've been
>>>>> meaning to take another look at Rust now it's stable.
>>>>
>>>> This is interesting, someone else proposed me that today too. This
>>>> would be a good strategy to get something up and running quickly
>>>> enough. Basically, Rust can build to a static library which can be
>>>> linked to C code, but until there's a memory allocator available,
>>>> things can be tricky.
[...]
>>>> The core parts of MiniOS could probably not be reimplemented right
>>>> away, but drivers would be doable. From what I see, the first step is
>>>> to communicate with the Xenbus and the Xenstore, right? Then have some
>>>> ring buffers and a way for xen to signal that some data is available
>>>> (I'm currently looking at netfront.c).
>>>
>>> The only driver needed for Mirage is the console driver, for early
>>> boot messages. Mirage has its own OCaml implementations of Xenbus and
>>> Xenstore, so replacing them isn't needed to get rid of the C. Much of
>>> Mini-OS is optional and unused (and it would be better to package the
>>> optional bits separately really).
>>>
>>> [...]
>>
>> So Mirage only needs the console and a way to talk with the Xenbus.
>
> Actually, Mirage doesn't even use Mini-OS's Xenbus, but has its own:
>
>   https://github.com/mirage/shared-memory-ring
>
>> Is the scheduling handled by MiniOS or by Mirage?
>
> Mirage:
>
> https://github.com/mirage/mirage-platform/blob/master/xen/lib/main.ml
>
> Mini-OS does provide preemptive threading, but we don't use it currently.

Correction: Mini-OS's threading is also cooperative. However, it
should be fairly easy to change it. You'd just need to preserve more
registers, I think, at least on ARM (I know very little about x86).

>>>>>> - the system can be monotask at first, but preemtive threading should
>>>>>> happen at some point
>>>>>
>>>>> If you have promises (and make all your low-level APIs non-blocking)
>>>>> then you can get a very long way without preemptive threading,
>>>>> although it would be good to have it eventually.
>>>>
>>>> Could you tell me more about this? This approach looks interesting.
>>>
>>> Every function in Mirage that might need to wait instead immediately
>>> returns a promise. e.g. the read function is defined as:
>>>
>>> val read: flow -> [`Ok of buffer | `Eof | `Error of error ] io
[...]
>>> See https://mirage.io/wiki/tutorial-lwt for more details.
>>>
>>
>> From what I understand, Lwt is a green threads library. I would have
>> thought Mirage needed to emulate OS threads, but it looks like this
>> would be enough for most use cases.
>
> Yep. Hardware is easiest to drive with callbacks, but traditional OS
> kernels go to a lot of trouble to present a blocking, synchronous API
> to userspace. Green threading libraries then have to mess around with
> OS threads to recover the asynchronous callback-based API. With
> Mini-OS, all that goes away.
>
>> Are multiple cores supported?
>
> No. Currently the recommendation is to spin up multiple VMs if you
> need multiple cores. Rust should be good at supporting this though.
>
>>>>>> - a part of Rust standard library is already available, I do not know
>>>>>> how much is needed to support the use case of a HTTP server
>>>>>> - building monitoring tools inside the kernel
>>>>>
>>>>> When running under Xen, we share a trace buffer with dom0. This allows
>>>>> you to see what the system is doing, even if it is hung or crashed.
>>>>> See: https://mirage.io/wiki/profiling
>>>>
>>>> That visualization tool looks useful! Do you have remote debuggers
>>>> too? With a serial port or something similar.
>>>
>>> Mirage can also be compiled to a Unix process and debugged with gdb
>>> and similar in most cases. I believe Xen also provides for debugging
>>> of guests from dom0, but I've never used that.
>>>
>>>> I guess the first thing I could do is to integrate some Rust code in
>>>> MiniOS. Is there a part of the code you would recommend? Something
>>>> well contained, without too many dependencies?
>>>
>>> I'd probably start with the page allocator:
>>>
>>> https://github.com/talex5/mini-os/blob/master/mm.c
>>>
>>> /*
>>>  * Initialise allocator, placing addresses [@min,@max] in free pool.
>>>  * @min and @max are PHYSICAL addresses.
>>>  */
>>> static void init_page_allocator(unsigned long min, unsigned long max)
>>>
>>> This gets called early during boot with all the free RAM as the
>>> argument. It provides alloc_pages and free_pages to allocate from this
>>> range.
>>>
>> I'll take a look
>>
>>>
>>> Another possible starting place is the console driver:
>>>
>>> https://github.com/talex5/mini-os/blob/master/console/console.c
>>>
>>> That's a bit harder though because you're writing to memory shared
>>> with dom0, so you need to be careful about memory barriers and atomic
>>> operations. I assume Rust can handle this nicely? It would be fun to
>>> see how to use Rust's ownership types to ensure that other bits of
>>> code can't write to parts of a shared buffer that dom0 may be reading,
>>> etc.
>>>
>>
>> There are atomic types available, and a way to represent memory
>> fences: https://doc.rust-lang.org/std/sync/atomic/
>
> Great :-)

One other thing I should mention: Mini-OS runs in kernel mode and can
be interrupted at any time. On x86, the interrupt handler uses the
same stack as the rest of the code. Therefore, you must compile with
the Rust equivalent of -mno-red-zone so Rust doesn't assume it can
store things below the current stack pointer.


-- 
Dr Thomas Leonard        http://roscidus.com/blog/
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel