Re: new Cohttp interface progress

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

On Tue, Aug 07, 2012 at 09:55:24PM +0100, Richard Mortier wrote:
> > Good question. It's because each threading library has different
> > tradeoffs in terms of performance.
> 
> hm- do you have pointers to any examination of this, or is it just well
> known?

It's simply because they have different semantics.  There are some numbers
in this paper from 2010 on different concurreny mechanisms in OCaml:
http://hal.inria.fr/docs/00/49/32/13/PDF/lwc.pdf
It doesn't cover Async as it has only be open-sourced this year, but you
get the idea I hope...

> > Lwt has very fairly fine-grained errors, and propagates exceptions as
> > values (i.e. "fail" vs "raise").
> > 
> > Async has a different mechanism called Monitors, where errors are
> > caught by a separate function that keeps an eye on a chunk of
> > computation. It has also deprecated streams, and has Reader/Writer
> > pipes in preference.
> 
> in some ways the different ways they do things like error handling is
> (to my mind) a more interesting distinction :)  (unless the perf
> differences are genuinely significant and different in multiple real
> applications.)

So I've been hacking away at this in recent days, and (thanks to help from
David Sheets) have a pretty workable repo now at:
http://github.com/avsm/ocaml-cohttp/tree/v2-interface

The lib/ directory has the core HTTP parser and no dependency on either
Lwt or Async.  Instead, it defines an IO monad and all the modules are
functorised against that.

Then, we have lwt/ and async/ directories which implement concrete
versions of the library.  You can see the Lwt interface here:

https://github.com/avsm/ocaml-cohttp/blob/v2-interface/lwt/cohttp_lwt.mli

There is a more low-level interface available via Cohttp_lwt_raw, but most
people will probably just use the Client/Server interface here.  The major
difference between Lwt and Async is how they handle streams.  You will
need the very latest Lwt (2.4.1dev) as they added an
Lwt_stream.on_terminate that we use to close the socket connection once a
body has been consumed by the client.  I have a mirror on avsm/lwt with
this required change.

I'm porting my avsm/ocaml-github bindings over to use the new API to make
sure any issues are shaken out, and then I will write a Mirage Net.Channel
version that we can use on the website.  Then, any core protocol parsing
changes will be shared across all the various backends, and we have a nice
reference for what Async, Lwt and Mirage-net code all looks like in one
place.

> > The other *big* thing we would like is that the same protocol library
> > works on both strings and Io_pages. It should be possible to hide away
> > the precise choice of what we're writing into behind IO.M.
> 
> by strings do you mean bigarray thingies?  if so then i buy that - it
> would be nice to have a single encapsulated layer that meant that all
> the mirage protocol libraries could take a single dependency on it and
> be usable from any other ocaml program.

Yeah. I haven't actually done this yet as I'd like to stabilise everything
else. However, it shouldn't be too hard to define a sub-set of the String
module and use that to directly read/write from Bigarray too.  We will
have to port the Re regexp library first, but that should also be very
quick.

-- 
Anil Madhavapeddy                                 http://anil.recoil.org