Re: [MirageOS-devel] Out of memory

[David Scott <scott.dj@xxxxxxxxx>]

Hi Thomas,

On Thu, Dec 17, 2015 at 4:59 PM, Thomas Leonard <talex5@xxxxxxxxx> wrote:

I've been trying to track down some occasional out-of-memory errors
with my Mirage/Xen unikernel. My simplified code does this:

let start ... =
  ...
  let rec loop () =
   let open Bigarray in
   let x = Array1.create Char c_layout 1024 in
   (* Gc.minor (); *)
   ignore x;
   loop () in
  loop ()

(note: Cstruct.create uses Array1.create internally)

When run, memory usage (as measured by mini-os's page allocator) rises
from about a third at the start to 100%, and then it dies:

[init]
mm: 2491/7642 (32.6 %)
mm: 2493/7642 (32.6 %)
[...]
mm: 7638/7642 (99.9 %)
mm: 7639/7642 (100.0 %)
mm: 7640/7642 (100.0 %)
mm: 7641/7642 (100.0 %)
mm: 7642/7642 (100.0 %)
Cannot handle page request order 0!
Fatal error: exception Out of memory
Raised at file "src/core/lwt.ml", line 901, characters 22-23
Called from file "lib/main.ml", line 58, characters 10-20
Called from file "main.ml", line 33, characters 6-58
Re-raised at file "main.ml", line 36, characters 10-12
Mirage exiting with status 2
Do_exit called!

If the Gc.minor () line is uncommented then it works.

The C code in caml_ba_alloc does:

  data = "">   if (data == NULL && size != 0) caml_raise_out_of_memory();

I guess we should do a GC.full_major here and retry, right? Should
this be changed upstream in OCaml itself?

Interestingly we've caught this (and done a -- perhaps OTT -- Gc.compact) here:

https://github.com/mirage/io-page/blob/1a489eb5ac2bdabbc08e142622ad061b83df0e2e/lib/io_page.ml#L36Â

I guess we never considered that code would make heavy use of big arrays in other contexts.

It would be nice if the OCaml code did a Gc before failing, just like it would do if an allocation from the minor heap failed because it was full.

Cheers,

Dave

--
Dr Thomas Leonard    http://roscidus.com/blog/
GPG: DA98 25AE CAD0 8975 7CDAÂ BD8E 0713 3F96 CA74 D8BA

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel

--

Dave Scott

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel