Re: Alternative Hashtbl and lwt yield-like functionality questions

[Haris Rotsos <Charalampos.Rotsos@xxxxxxxxxxxx>]

On 19 December 2011 12:45, Thomas Gazagnaire
<thomas.gazagnaire@xxxxxxxxx> wrote:
>> thanks for the idea. Although, I don't think that an array is
>> appropriate for my problem as I want to have the 0(1) lookup
>> complexity that the Hashtbl provides.
>
> I don't get it. Why do you need to read the data you are producing while your 
> test is running ?

I am running a code that simulates the functionality of a
learning_switch. The switch will inspect mac layer header and discover
on which port it sees which mac address. So the Hashtbl needs to
insert information while it reads data but also use that information
to forward frames.

>
>> In any case, what I did was to
>> reduce the size of the experiment. It was though a surprise to me how
>> inefficient memory-wise is the Hashtbl in comparison to c structures.
>
> it is really depending on the data-structure you are using. If you have a 
> know in advance the size of your data, and you want to have constant time 
> read/write, you can use arrays. If your data is not bound, the only way it to 
> amortize your costs. I'm curious to know which kind of C structure does a 
> better job here.

I am comparing against a similar implementation of the code that uses
the boost hashtbl. While the test runs, the c++ code create a very
small footprint, while the ocaml code starts bloating. As an example
for the previous experiment a mirage vm will run out of memory after
20 seconds of insertion, while the c++ code will consume only a few
megs. I guess this is the cost you pay when you want automatic memory
management.

>
> --
> Thomas
>
>> On 14 December 2011 14:04, Thomas Gazagnaire
>> <thomas.gazagnaire@xxxxxxxxx> wrote:
>>> How long does you test longs ? Do you really want to store your data in a 
>>> structured way ?
>>> What you can do to avoid too much allocation during your test is to 
>>> allocate an array each second (say of size 70K), and once filled, to store 
>>> it in a mutable list and allocate a new array to continue your test. When 
>>> you test is finished, you can analyze your mutable list and build your 
>>> association table using the data-structure you want.
>>
>>
>>
>> --
>> Charalampos Rotsos
>> PhD student
>> The University of Cambridge
>> Computer Laboratory
>> William Gates Building
>> JJ Thomson Avenue
>> Cambridge
>> CB3 0FD
>>
>> Phone: +44-(0) 1223 767032
>> Email: cr409@xxxxxxxxxxxx
>
>



-- 
Charalampos Rotsos
PhD student
The University of Cambridge
Computer Laboratory
William Gates Building
JJ Thomson Avenue
Cambridge
CB3 0FD

Phone: +44-(0) 1223 767032
Email: cr409@xxxxxxxxxxxx