Re: [Publicity] rump kernel and Rumprun unikernel proposed post

[Antti Kantee <pooka@xxxxxx>]

[added mato to cc]

On 31/07/15 17:27, Anil Madhavapeddy wrote:
> (Thanks Russ for sorting out my login)
>
> Great post! Some quick thoughts from vacation:
>
> - In the "the goal of rump was not unikernels", it might be worth pointing out that Rump 
> provides a long term baseline for other unikernel projects that have been focussing on the high 
> level structure.  For example, Martin's work on Mirage+rump now lets us slowly deprecate the old 
> MiniOS runtime in favour of Rump, and get KVM and baremetal support in addition to Xen, and focus 
> on the "new horizons" rather than low level device driver hackery.
Hmm, I wonder if I can succinctly work that in.  I think I should stress 
that the idea to is to provide a proper architecture so that projects 
using rump kernels don't need to fork their own copies, and encourage 
discussion if the architecture or implementation falls short of 
someone's needs.  Forking is nice for a quick thrill, but when you hit 
the maintenance stage it's suddenly not so fun.
I assume that Martin will write about "Mirump" in detail himself, but I 
guess I can mention it to strengthen the above argument.  Would 
something in the order of "yaddayadda... the MirageOS unikernel has 
already started taking advantage of this feature [Martin Lucina]" be 
appropriate?  Plus a link to:
http://permalink.gmane.org/gmane.os.mirage.devel/954

> - Link to the third-party applications Rump github repo?
ah, definitely.

> - A TL;DR with some hrefs into the rump kernel book on some of the terminology 
> would be useful for the more casual reader.  You mention in one blog post -- 
> rump kernel, unikernel and anykernel -- quite a lot to take in one post :) Your 
> FOSDEM 2013 interview on the topic is good: 
> https://archive.fosdem.org/2013/interviews/2013-antii-kantee/ or your New 
> Directions in Operating Systems 2014 talk.
The problem with the tl;dr is that it's hard to explain things to a 
global audience when I don't know what their *assumptions* are and which 
specific piece they are interested in -- even though the pieces are 
bite-sized, there's a lot of them.  Mini-anecdote: it took years for 
people to stop saying "oh you've reinvented FUSE".  So, the book is the 
only thing I trust to explain things in a hard-to-misassume way.  But 
I'll add some tl;dr if I can figure out what to add.
I'll see if I can somehow remove "anykernel".  My in-house reviewer 
complained about that as well.  The problem is that I doubt the figure 
works if "anykernel" is not there, so I need to think of something else.
> - if you have time, linking to the xen-devel mailing list posts from the 
> archives would be really useful.  There were some great threads there that you 
> reference.
Ah, right, there was the thread where Ian Jackson got so excited that he 
forgot that the other Ian wasn't Ian Jackson.  I'll see if the thread is 
useful.  Most of the interesting discussion may have been in private 
email :(
> - nowhere does the article mention what sort of existing applications rump 
> kernels are bad at -- worth mentioning that fork-heavy applications don't do so 
> well?  Might be out of scope.
I won't say that rump kernels have issues with fork because that's not 
true.  Rump kernels work perfectly with fork+exec *if the underlying 
system supports it*.  Rump kernels do not provide you with fork or exec 
because I don't consider fork or exec to be protocol translators (a.k.a. 
drivers).  However, rump kernels do support *the application* performing 
fork or exec.  Same goes for mmap.
If you mean the Rumprun unikernel, which is the one example of an 
underlying system which does not do VM, then yes.  But, as you can 
gather from above, the discussion on the topic gets convoluted quickly, 
and will confuse more than clarify unless I spend several paragraphs on 
it.  So, I think it's better that people read the book if they really 
want to understand.
The only thing that I can think of which really doesn't fit into the 
rump kernel model of splitting execution and [POSIX] drivers is signals. 
 But we'll soon have to figure out how to cram that in.  I actually 
sort of have a plan for that already, but just need to brave enough to 
try to implement it -- it requires touching the rump kernel CPU 
scheduler, and I'm very afraid of that code.
> - As a random aside, 2007 seems to have been the year that lots of projects 
> started.  I remember sitting in a room in Cambridge with Andrew Tolmach and 
> Adam Wick in Jan 2007 and using MiniOS to create the first Mirage and HalVM 
> kernels with networking (just ethernet) as well.  Trying to dig out an online 
> reference to that meeting -- ISTR we had a photo of the whiteboard somewhere!
Hahaa, "commit early, commit often" saved me there.  I think this was 
the first sketch of the mtools "unikernel" which ran both on NetBSD and 
Linux (not that the infrastructure is strictly speaking that of a 
unikernel, but anyway):
http://cvsweb.netbsd.org/bsdweb.cgi/~checkout~/src/sys/rump/fs/bin/fsconsole/Attic/fsconsole.c?rev=1.4&content-type=text/plain

And, urgh, handwritten syscall handlers and namei (ukfs).  I don't want 
to go back there.  Though, ukfs displays the quintessential philosophy 
of rump kernels: either figure out how to use existing code unmodified, 
or massively simplify it for the case at hand; port-by-copypaste 
(forking) is evil.  The problem with ukfs was that syscall handlers and 
namei are complex because they're complex, so it wasn't possible to 
simplify them and still have them, well, work (like really work, not 
just superficially work).  So the ukfs experiment failed, c'est la 
research, and I later figured out how to include the unmodified 
counterparts.

Anyway, got a bit sidetracked.  I'll send a ping to the list once I've 
finished editing everyone's suggestions in.  Might be a few days.
  - antti

_______________________________________________
Publicity mailing list
Publicity@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/publicity