Re: mirage + froc = self-scaling?

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

On 28 Mar 2013, at 11:33, Raphael Proust <raphlalou@xxxxxxxxx> wrote:

> On Thu, Mar 28, 2013 at 11:18 AM, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>> Some background on FRP, and specifically the FROC self-adjusting library:
>> http://ambassadortothecomputers.blogspot.co.uk/2010/05/how-froc-works.html
>> 
>> The goal of this work is to encode the external dependencies of a Mirage 
>> unikernel explicitly, so that changes can ripple through the various 
>> computation units in a well-defined manner.  The tricky bit is that I/O 
>> threads ('the outside world') need to be strongly separated from FROC ('the 
>> configuration space')
>> 
>> Consider the case of an ARP input packet, which needs to update a 
>> configuration variable (the ARP table), but is in the middle of doing some 
>> I/O.  However, we want to safely weave together static input (a static ARP 
>> configuration) with the dynamic inputs, which is where Froctol comes from.  
>> There are many, many cases of such undefined behaviour in conventional 
>> operating systems: what happens to existing network sockets when a DHCP 
>> server rebinds an IP address on your machine?
>> 
>> Raphael, are your experiments with this from last summer available anywhere? 
>>  I couldn't spot them in your Github repo list.
> 
> They are in a repo of yours: https://github.com/avsm/ocaml-ld

Ha! Right under my nose. I need a local search of the hundreds of repositories 
now...

> 
> The following comment (in lib/froc_lwt) is interesting. It list things
> that should be avoided when mixing the two paradigms
> (cooperative-threading and reactive).
> 
> 
> (*DOs and DONTs
> *
> * DONT expose writables (because it exposes the next DONT to the user)
> * DONT change and yield (or otherwise switch thread)
> * This can lead to partial changes being propagated. If you need to change
> * several values before propagation, treat all the changes as a critical
> * section, and make sure you propagate before yielding.
> * DONT bypass Froc_sa.bind by creating a new changeable, manually write and
> * propagate.
> * DONT implement events over changeables. This restriction ensures that the
> * whole system (in particullar, the data-plan) will be reactive (although
> * theoretically appealing, it is better, for performance reasons, to
> * restrict the reactive part to the configuration-plan).
> * DONT use read under a bind (to avoid inconsistencies) but
> * DO use read when you are in the data-plane (and thus not threading the
> * changeables around).
> *)
> 
> One thing that is quite important with Froc: you can set several
> changes and push them in one go. Thus, when reconfiguring it is
> important to not yield in between different changes (otherwise another
> thread might push).

Right... I think that the prototype should actually use more privilege 
separation
to help nail the semantics down.  Each Froc cycle should spin up a new process
that runs its own Lwt thread, with explicit handoff between them for resources.

This will obviously be pretty heavyweight, but it's far easier to optimise it
within the same process later than it is to untangle a single I/O threadpool
into multiple ones.

-anil