Re: mirage support for dynamically loaded modules

[Andrew Lekar <sam.and.tetris@xxxxxxxxx>]

"Got it -- so you want to interface with hardware drivers via passthrough to a driver domain, I think."

No! I do not have any domains with drivers or kernels - only dom0 with mirage. So I have to implement hardware interaction all by myself with a thin layer of xen.

For now I've got working baremetal ocaml runtime and I teared off xenstore support from mirage to run it on dom0. You can think of it as a true baremetal mirage since xen does not offer anything but some debug io and timer support.

Therefore I'm interested in deploying hardware drivers separately from mirage kernel and I'd prefer to have in theese drivers as less C code as possible. Will you give me any suggestions about such mirage configuration?

2013/9/10 Anil Madhavapeddy <anil@xxxxxxxxxx>

Got it -- so you want to interface with hardware drivers via passthrough to a driver domain, I think.

The best way to think of Mirage is as a growing bunch of libraries, glued together by a minimal runtime for whatever platform is targetted and an increasingly sophisticated constraint-based package manager (OPAM).

For example, consider these backends:

- UNIX --- the compiler output is an ELF executable, and started/stopped via a helper binary that passes it file descriptors and performs privileged operations.  It can be passed raw packets via tuntap, or invoke kernel sockets, depending on how it's built.

- Xen -- the compiler output is an object file that is linked to a minimal Xen OS with a linker script to make it PV bootable.  All the drivers are normal OCaml libraries (including the shared memory ring handling).

- _javascript_ -- the compiler output is a JS blob which has externals that are satisfied via other _javascript_ functions made available, and the runtime used is the _javascript_ one.

- kFreeBSD -- the compiler output is linked with the OCaml GC and fixed-point floating point and kernel module scaffolding.  The internal kernel APIs are bound as OCaml libraries.

The final one is the most relevant to you...the easiest way to manage hardware in the first instance is to use FreeBSD as a driver domain (no userspace), and take advantage of existing Mirage libraries to manage the kernel lifecycle.

The most immediate thing to get working in the kernel guest is Xenstore.  This is exposed to a HVM guest via a special MSR which gives you the shared_info page, which in turns gives access to hypercalls and the Xenstore event channel.  Once this is set up, you can use the existing OCaml Xenstore library to handle all the communication to/from the driver domain, without having to worry about reimplementing the protocol in the kernel.  The same goes for other virtual devices such as Netfront, Blkfront, and (thanks to Dave) even a VNC framebuffer if you want a PV framebuffer.

On your other points about filesystems -- Thomas Gazagnaire is working on a Mirage filesystem at the moment (a branch-consistent one dubbed Irminsule), and Dave has a FAT32 implementation, and I wrote a simple static one (in mirage-fs).  We're finding that filesystems aren't often required by our current applications, however.

Hope this helps -- if you let us know more specific use-cases, I can guide you towards more specific solutions.  

cheers,

Anil

On 9 Sep 2013, at 18:21, Andrew Lekar <sam.and.tetris@xxxxxxxxx> wrote:

Thank you Anil.

I am working with baremetal port of mirage (well on top of xen in dom0 domain) and I'm expecting to have a lot of hardware drivers which I'd like to deploy and debug without rebuild of entire mirage image. Also I'd like to have a way of upgrading software and it is better done with in-system compiler or bytecode interpreter. As far as I understood your answer mirage ocaml does have bytecode interpreter but does not support filesystem or other modules sources, am I right?

Unikernel solution is good for me for now but it is way too long to debug anything with unikernel and I expect it to become worse in future.

2013/9/9 Anil Madhavapeddy <anil@xxxxxxxxxx>

On 8 Sep 2013, at 20:25, Andrew Lekar <sam.and.tetris@xxxxxxxxx> wrote:

> Hello everyone.
>
> Is it possible to support dynamically load ocaml modules in mirage? So that I do not rebuild entire mirage kernel but make it once and add additional drivers later. What are the steps required to make it possible? I guess to make it work I should implement filesystem support, bytecode interpreter or ocaml compiler in mirage.

Hi Andrew,

It depends which backend you're thinking of.  In Xen, we have some patches to further make the address space immutable post-boot (by dropping the ability to alter the guest page tables or switch CR3).  In that case, you can't do dynamic loading by design.

However, without this patch, and with a suitable filesystem, network or vchan source for the dynamic module, you could do dynamic loading.  The real question is why you'd want that at all, given it's easier to recompile a Mirage instance and redeploy it rather than go through all the trouble of dynamic loading.  In many ways, that's the whole point of the Xen backend ;-)

-anil