Re: Request: Apache Thrift

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

At a quick glance, the Thrift OCaml code generator looks pretty inefficient -- 
those object invocations won't be inlined, nor do they look necessary.

The only real requirement to make a Thrift serializer Mirage-friendly is to 
make it use Bigarrays rather than strings, and to make it pure OCaml.  If you 
can make it work on UNIX with Lwt_bytes, it will be pretty easy to port to Xen 
afterwards.

-anil

On 28 Jul 2013, at 14:28, rektide@xxxxxxxxxxxxxxx wrote:

> Hi Miragistas, much love & respect for your Unikernalist & Cross-everything 
> cause-
> 
> I'm not capable of getting this project going myself, and am filing a broad 
> request, or at least a
> registrance of interest here: I'd love to see Thrift support on Mirage. 
> Thrift has a fairly wide
> presence in the "big data"/computing-services world, and serves as a 
> serialization protocol as well
> as an RPC mechanism. I'd love for the ability to host Thrift services in 
> Mirage, and to make
> connections to external Thrift services.
> 
> I've run the Thrift generator on their tutorial's "idl" files, the 
> interface-descriptor-language
> files, producing some OCaml code. It produces some relatively simple looking 
> OCaml code, backed by a
> Thrift library which is where the real implementation is.
> 
> I'm hosting the IDL files & their generated OCaml output:
> https://github.com/rektide/thrift-tutorial-ocaml
> 
> Thrift's OCaml code is at:
> https://github.com/apache/thrift/tree/master/lib/ocaml/src
> 
> Some of the projects using Thrift:
> * HBase (data-storage)
> * Hive (data-querying)
> * Spark (data-storage)
> * Scribe (logging)
> * Cassandra (data-storage)
> * Tachyon (memory-store)
> 
> I've spent a couple hours trying to make a start at this, and wouldn't mind 
> taking the project on
> myself if I had some high level design to point to authored by someone with a 
> reasonable familiarity
> with options for threading (lwt &c) in Mirage and with a confidence that they 
> have any idea
> whatsoever what they were doing and what a good final product might look 
> like. I definitely am not
> familiar or comfortable enough with Mirage to have felt like I was doing 
> anything more than casually
> slinging codes around, mostly owing to not really grokking the concurrency 
> models available.
> 
> I suspect for someone that is better intimate with Mirage, porting the Thrift 
> OCaml code could be
> short day or two or hacking, and it'd be lovely to get to use Mirage 
> alongside some of the big
> computing stacks out there. I personally am gung ho for playing with some of 
> the Amp Lab stuff-
> Spark, & Tachyon, and would love to be doing it in Mirage, but I found myself 
> not widely versed
> enough to be up to the task of porting the OCaml Thrift code myself, unguided.
> 
> I'll continue idling on IRC, or as you prefer reply here. Thanks.
> 
> Regards,
> -rektide
>