[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

big data structure problem in ocaml

  • To: <cl-mirage@xxxxxxxxxxxxxxx>
  • From: "Yiming Zhang" <sdiris@xxxxxxxxx>
  • Date: Mon, 28 Jan 2013 17:40:36 -0000
  • List-id: MirageOS development <cl-mirage.lists.cam.ac.uk>
  • Thread-index: Ac39fpON6JMrbuC3Q9CB5e1UgOqEYw==

Hi all,


I migrated MemStore (https://github.com/interwolf/MemStore, an ocaml version memcached in its first stage) onto Mirage and it works fine (on 4.00.1+mirage-xen) for small data size. However, when I store large data, e.g., 1M kv pairs, it becomes slower and slower. I believe it is because I use a large data structure to store the kv, and when a new kv pair is set the whole data structure is copied (as shown below).


module OrdKey = struct

  type t = string

  let compare = Pervasives.compare


module KVMap = Map.Make(OrdKey)

let kv = ref KVMap.empty


(* each time when a new kv pair is set, the following is invoked *)

kv := KVMap.add !key_in_process line !kv



I have no idea how to copy just the new kv, instead of the whole kv storage, when a new kv is set. Does anyone have some hint? Or there are example code for similar problems in the ocaml world


Thank you all!





Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.