Re: pa_cstruct first cut available

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

No manual free: the Bigarray buffers are internally reference counted, so as 
soon as all of the views onto a buffer are no longer used, the Gc will free the 
underlying resource.  The runtime generates a 'caml_ba_proxy' which is a shared 
reference count on the underlying data.

One thing I might need to extend the bigarray stubs for is that you can't 
register a *custom* finaliser. Each bigarray can either be CAML_BA_MANAGED 
(i.e. the runtime allocated the bigarray and should free it), or 
CAML_BA_MAPPED_FILE (mmap, so needs an munmap), or CAML_BA_EXTERNAL (something 
else, take no action).

Right now, the runtime takes no action for CAML_BA_EXTERNAL, but we need to be 
able to register a custom finaliser, so that we can do something interesting 
with the underlying resource in Xen (mostly putting it back into the Io_page 
pool for re-use).  This would also be useful in UNIX to similarly pool pages 
rather than going to expensive malloc and free all the time.

Are you familiar with that code? Something upstreamable for ocaml-4.0 (or 
indeed, if there's another way to do what I want) would be helpful so we don't 
need to maintain a custom patch...

Regarding your section question, the get_* functions just make a copy into the 
local heap (but in most cases, it's a cheap unboxed integer in the minor heap), 
and get_buffer calls are references and not copies.  It's all pretty cheap and 
comparable to Bitstring, which allocates a fair few tuples.

-anil

On 10 May 2012, at 17:04, Thomas Gazagnaire wrote:

> Neat!
> 
> So you need to manually free the objects you don't use anymore ?
> Aslo, do the get_* functions makes copy into the ocaml heap or there is a 
> clever trick there as well ?
> 
> Thomas
> 
> On May 10, 2012, at 5:42 PM, Anil Madhavapeddy wrote:
> 
>> I've knocked together the C-struct syntax extension, and it's actually quite 
>> simple and effective.
>> 
>> https://github.com/avsm/ocaml-cstruct/blob/master/lib_test/pcap.ml
>> 
>> is a pcap parser that cracks open pcap/ethernet/ipv4/tcp and displays the 
>> payloads. The 'cstruct' declaration defines all the accessors as you might 
>> expect, and the underlying buffers are all reference counted (as in MPL) and 
>> so can be passed around. The base-pointer can also be retrieved, so you can 
>> do comparisons of the underlying Io_page (e.g. to get a Xen grant onto it).
>> 
>> If noone has any objections, I'll add unit tests for the write path, and 
>> merge this into Mirage in my os-bigarray branch.  I dont think we actually 
>> need bitstring at all anymore.
>> 
>> Mort: reckon you can easily port the DNS code to this? It should speed up 
>> the responses an awful lot.
>> 
>> -anil
>