Re: pa_cstruct first cut available

[Thomas Gazagnaire <thomas.gazagnaire@xxxxxxxxx>]

> No manual free: the Bigarray buffers are internally reference counted, so as 
> soon as all of the views onto a buffer are no longer used, the Gc will free 
> the underlying resource.  The runtime generates a 'caml_ba_proxy' which is a 
> shared reference count on the underlying data.

nice

> One thing I might need to extend the bigarray stubs for is that you can't 
> register a *custom* finaliser. Each bigarray can either be CAML_BA_MANAGED 
> (i.e. the runtime allocated the bigarray and should free it), or 
> CAML_BA_MAPPED_FILE (mmap, so needs an munmap), or CAML_BA_EXTERNAL 
> (something else, take no action).
> 
> Right now, the runtime takes no action for CAML_BA_EXTERNAL, but we need to 
> be able to register a custom finaliser, so that we can do something 
> interesting with the underlying resource in Xen (mostly putting it back into 
> the Io_page pool for re-use).  This would also be useful in UNIX to similarly 
> pool pages rather than going to expensive malloc and free all the time.
> 
> Are you familiar with that code? Something upstreamable for ocaml-4.0 (or 
> indeed, if there's another way to do what I want) would be helpful so we 
> don't need to maintain a custom patch...

yes, I've hacked few things there for numcaml one year ago, so I can try to 
remember how it is working :-) Regarding the upstreamable patch, I guess that's 
just adding few line in caml_ba_finalize. But I really can't say if Xavier will 
accept it ... we can still try to publish the patch on Mantis and I'll ask 
Fabrice to convince Xavier ...

> Regarding your section question, the get_* functions just make a copy into 
> the local heap (but in most cases, it's a cheap unboxed integer in the minor 
> heap), and get_buffer calls are references and not copies.  It's all pretty 
> cheap and comparable to Bitstring, which allocates a fair few tuples.

ok, seems fine to me!

--
Thomas

>> Neat!
>> 
>> So you need to manually free the objects you don't use anymore ?
>> Aslo, do the get_* functions makes copy into the ocaml heap or there is a 
>> clever trick there as well ?
>> 
>> Thomas
>> 
>> On May 10, 2012, at 5:42 PM, Anil Madhavapeddy wrote:
>> 
>>> I've knocked together the C-struct syntax extension, and it's actually 
>>> quite simple and effective.
>>> 
>>> https://github.com/avsm/ocaml-cstruct/blob/master/lib_test/pcap.ml
>>> 
>>> is a pcap parser that cracks open pcap/ethernet/ipv4/tcp and displays the 
>>> payloads. The 'cstruct' declaration defines all the accessors as you might 
>>> expect, and the underlying buffers are all reference counted (as in MPL) 
>>> and so can be passed around. The base-pointer can also be retrieved, so you 
>>> can do comparisons of the underlying Io_page (e.g. to get a Xen grant onto 
>>> it).
>>> 
>>> If noone has any objections, I'll add unit tests for the write path, and 
>>> merge this into Mirage in my os-bigarray branch.  I dont think we actually 
>>> need bitstring at all anymore.
>>> 
>>> Mort: reckon you can easily port the DNS code to this? It should speed up 
>>> the responses an awful lot.
>>> 
>>> -anil
>> 
>