Re: [MirageOS-devel] MPL compiler and bigarray

[Goswin von Brederlow <goswin-v-b@xxxxxx>]

On Sun, Oct 05, 2014 at 11:03:40PM +0100, Anil Madhavapeddy wrote:
> [ccing the mirageos-devel list]
> 
> On 29 Jul 2014, at 10:29, Goswin von Brederlow <goswin-v-b@xxxxxx> wrote:
> > 
> > I've looked at your melange paper [1] and it says that the MPL
> > compiler will use a basis library for I/O and buffer management
> > functions. You then show the data type for a light-weight packet
> > environment record showing that the actual payload is stored in a
> > string.
> > 
> > I want to use MPL for a high througput protocol that involves lots of
> > disk I/O with the linux AIO libarry as well and I need to use page
> > aligned bigarrays for that.
> > 
> > I'm wondering how flexible this is. Would it be easy to replace the
> > string with a bigarray in the library or replace the library
> > altogether? The advantage of a bigarray is that the data is allocated
> > outside the ocaml heap and will never be moved by the GC. This allows
> > using it in C stubs while the ocaml runtime lock is released or for
> > DMA. Bigarrays can also be created to be page aligned or from an
> > existing block of data. Which strings all of that requires copying the
> > data.
> > 
> > The MPL generated code is already fast despite the extra copying the
> > string requires. I wonder how much using bigarrays for true zero-copy
> > networking and I/O would gain on top of that.
> 
> Sorry about the delay getting back to you on this.
> 
> While it would be easy to replace MPL with a Bigarray rather than a
> string, it's probably not worth the effort to resurrect that tool.
> 
> We're leaning towards lightweight views over Bigarrays via Cstruct
> these days, and using GADTs+metaprogramming to convert packet grammars
> into parsers.   The GADT portion of ASN1 parsing for example:
> http://openmirage.org/blog/introducing-asn1
> 
> What's not fully integrated yet is the use of Metacaml or ppx/camlp4
> to perform the metaprogramming.  MPL's introduction of concrete syntax
> can then disappear, and packets can be manipulated directly in the
> host language as an EDSL.
> 
> For simpler protocols like IPv4 or TCP parsing, Cstruct provides a
> camlp4 extension that lets you write things like:
> 
> cstruct ipv4 {
>   uint8_t        hlen_version;
>   uint8_t        tos;
>   uint16_t       len;
>   uint16_t       id;
>   uint16_t       off;
>   uint8_t        ttl;
>   uint8_t        proto;
>   uint16_t       csum;
>   uint8_t        src[4];
>   uint8_t        dst[4]
> } as big_endian
> 
> 
> and retrieve the values from a Bigarray representation using the
> efficient compiler builtins (4.01.0+) for integer conversions.
> See https://github.com/mirage/ocaml-cstruct for more on that.
> 
> -anil

Cool. That sounds even better.

MfG
        Goswin

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel