Re: ocaml-cohttp client

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

On 7 Apr 2012, at 18:53, Thomas Gazagnaire wrote:

> Just a side question:
> 
>> - client/ (cohttp client, Lwt_unix)
>> - server/ (cohttp server, Lwt_unix)
> 
> How do you plan to port that to Xen later ? Do you plan to reimplement lwt_io 
> channels on Xen (or maybe that's already the case) ?

The current thinking is to try and expose as much of the protocol in an 
iteratee style, so that the UNIX wrappers are essentially just configuration 
setup.

In Lwt, this means using the UNIX/Xen wrappers to output Lwt_streams of input 
and output buffers. These Lwt_streams are OS-independent, and so can live in 
the protocol-independent bit.

The ocaml-workflow library is an experiment in this direction. The programmer 
defines listeners and connectors, and the workflow engine deals with accepting 
external I/O and mapping them into this world. It can also manufacture 
shared-memory connections if the two communicating parties are on the same 
host. There are some subtleties around making the shared memory transport 
faster than TCP (due to the transmit buffers being more limited on a shmem 
connection, versus a TCP connection). I'm just knocking up a logging library so 
we can do 'tcptrace'-style windowing graphs to eliminate these bottlenecks by 
visualising the performance profiles more easily.

So if someone programs to this workflow model, they won't know where their data 
is actually coming from --- the full Xen stack, or just a UNIX container, and 
the engine can take care of workload balancing.  Dave is currently refactoring 
xenstored to be the name-service for ocaml-workflow (so it will even run 
without Xen being installed, which is very useful for development!).  

Note that cohttp is not structured fully in this model yet: the client, for 
example, has too much protocol logic in it. Some of that needs to be moved into 
the library so it isn't duplicated in the Xen backend.  This will also help the 
compilation into Javascript.

-a