[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: big data structure problem in ocaml

> ********************************
> module OrdKey = struct
>   type t = string
>   let compare = Pervasives.compare
> end

If you keys are always be strings, I would suggest using the optimized 

> module KVMap = Map.Make(OrdKey)
> let kv = ref KVMap.empty
> ……
> (* each time when a new kv pair is set, the following is invoked *)
> kv := KVMap.add !key_in_process line !kv
> **********************************

You are not copying kv here, you are just setting the box where you store the 
kv tree with an updated tree, so you can't do much.

> Does anyone have some hint? Or there are example code for similar problems in 
> the ocaml world?

What are you testing exactly ? Do you care about fast reads or fast writes ? Do 
you clean-up your cache regularly or is it an ever-growing one (which could 
explain the performance issues).

Some hints: usually a cheap way to build fixed-size cache is to use 
hash-indexed arrays: if you have few hash collision and a rough idea of the 
number of elements you want to store, that's pretty efficient. An example here:





Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.