From lists+mirage@jimpryor.net Tue Dec 06 18:25:31 2011 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RXzhr-0001c6-6C (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 06 Dec 2011 18:25:31 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.6 from SpamAssassin-3.3.2-1195874 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [66.111.4.26 listed in list.dnswl.dnsbl.ja.net] * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from out2.smtp.messagingengine.com ([66.111.4.26]:36640) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp (csa=ok) id 1RXzhp-0007Vp-G7 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 06 Dec 2011 18:25:31 +0000 Received: from compute1.internal (compute1.nyi.mail.srv.osa [10.202.2.41]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 834712122B for ; Tue, 6 Dec 2011 13:25:29 -0500 (EST) Received: from web3.nyi.mail.srv.osa ([10.202.2.213]) by compute1.internal (MEProxy); Tue, 06 Dec 2011 13:25:29 -0500 Received: by web3.nyi.mail.srv.osa (Postfix, from userid 99) id 6114C4008E; Tue, 6 Dec 2011 13:25:29 -0500 (EST) Message-Id: <1323195929.21305.140661008212681@webmail.messagingengine.com> X-Sasl-Enc: 4UX/lwvxEzsNRtENnZ9HX0WTiCMzX0SCJ5pOclbp4Yy0 1323195929 From: lists+mirage@jimpryor.net To: cl-mirage@lists.cam.ac.uk MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii" X-Mailer: MessagingEngine.com Webmail Interface In-Reply-To: References: Subject: Trying to build on FreeBSD Date: Tue, 06 Dec 2011 13:25:29 -0500 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2011 18:25:31 -0000 X-Keywords: X-UID: 139 Status: O Content-Length: 14402 Lines: 303 Hi all, this project looks very exciting. I was trying to build it for FreeBSD, and can only get so far. Maybe someone more familiar with with this project and with OCamlbuild can make some suggestions? Here are the changes I've made to the git repo, in an attempt to get things working on FreeBSD: ------------------- patch ------------------- diff --git a/lib/myocamlbuild.ml b/lib/myocamlbuild.ml index f5d02e8..7d92d92 100644 --- a/lib/myocamlbuild.ml +++ b/lib/myocamlbuild.ml @@ -84,18 +84,20 @@ end (* OS detection *) module OS = struct - type u = Linux | Darwin + type u = Linux | Darwin | FreeBSD type t = Unix of u | Xen | Node let host = match String.lowercase (Util.run_and_read "uname -s") with | "linux" -> Unix Linux | "darwin" -> Unix Darwin + | "freebsd" -> Unix FreeBSD | os -> Printf.eprintf "`%s` is not a supported host OS\n" os; exit (-1) let unix_ext = match host with | Unix Linux -> "linux" | Unix Darwin -> "macosx" + | Unix FreeBSD -> "freebsd" | _ -> Printf.eprintf "unix_ext called on a non-UNIX host OS\n"; exit (-1) end diff --git a/regress/fs/readdir.sh b/regress/fs/readdir.sh index b3aeec5..d36be30 100755 --- a/regress/fs/readdir.sh +++ b/regress/fs/readdir.sh @@ -28,6 +28,17 @@ case `uname -s` in sudo umount tmpmount ;; + FreeBSD ) + dd if=/dev/zero of=${OUTPUT} bs=1 seek=128M count=0 + /sbin/newfs_msdos -L MirageFAT -F 16 -v ${OUTPUT} + rm -rf tmpmount && mkdir tmpmount + MD=$(sudo mdconfig ${OUTPUT}) + mount /dev/$MD tmpmount + sudo cp -r ${INPUT} tmpmount + sudo umount tmpmount && rmdir tmpmount + sudo mdconfig -d -u $MD + ;; + * ) echo Unknown OS detected exit 1 diff --git a/scripts/mir-fat-create b/scripts/mir-fat-create index b6acaf2..b604ef7 100755 --- a/scripts/mir-fat-create +++ b/scripts/mir-fat-create @@ -38,6 +38,17 @@ case `uname -s` in sudo umount tmpmount && rmdir tmpmount ;; + FreeBSD ) + dd if=/dev/zero of=${OUTPUT} bs=1 seek=128M count=0 + /sbin/newfs_msdos -L MirageFAT -F 16 -v ${OUTPUT} + rm -rf tmpmount && mkdir tmpmount + MD=$(sudo mdconfig ${OUTPUT}) + mount /dev/$MD tmpmount + sudo cp -r ${INPUT}/* tmpmount + sudo umount tmpmount && rmdir tmpmount + sudo mdconfig -d -u $MD + ;; + * ) echo Unknown OS detected exit 1 diff --git a/scripts/myocamlbuild.ml b/scripts/myocamlbuild.ml index 90b7156..63e72cb 100644 --- a/scripts/myocamlbuild.ml +++ b/scripts/myocamlbuild.ml @@ -56,19 +56,20 @@ end (** Host OS detection *) module OS = struct - type unix = Linux | Darwin + type unix = Linux | Darwin | FreeBSD type arch = X86_32 | X86_64 let host = match String.lowercase (Util.run_and_read "uname -s") with | "linux" -> Linux | "darwin" -> Darwin + | "freebsd" -> FreeBSD | os -> eprintf "`%s` is not a supported host OS\n" os; exit (-1) let arch = match String.lowercase (Util.run_and_read "uname -m") with | "x86_32" | "i686" -> X86_32 - | "i386" -> (match host with Linux -> X86_32 | Darwin -> X86_64) + | "i386" -> (match host with Linux | FreeBSD -> X86_32 | Darwin -> X86_64) | "x86_64" -> X86_64 | arch -> eprintf "`%s` is not a supported arch\n" arch; exit (-1) @@ -89,8 +90,8 @@ module Mir = struct let unixmain mode = lib / mode / "lib" / "main.o" in let mode = sprintf "unix-%s" (env "%(mode)") in let dl_libs = match host with - |Linux -> [A"-lm"; A"-lasmrun"; A"-lcamlstr"; A"-ldl"] - |Darwin -> [A"-lm"; A"-lasmrun"; A"-lcamlstr"] in + |Linux -> [A"-lm"; A"-lasmrun"; A"-lcamlstr"; A"-ldl"] + |Darwin |FreeBSD -> [A"-lm"; A"-lasmrun"; A"-lcamlstr"] in let tags = tags++"cc"++"c" in Cmd (S (A cc :: [ T(tags++"link"); A ocamlc_libdir; A"-o"; Px out; A (unixmain mode); P arg; A (unixrun mode); ] @ dl_libs)) --------------- end patch ------------------- And I get up to this point in the build process: /usr/local/bin/ocamlopt.opt -a std/pervasives.cmx std/array.cmx std/char.cmx std/list.cmx std/string.cmx std/sys.cmx std/buffer.cmx std/int32.cmx std/marshal.cmx std/obj.cmx std/printf.cmx std/base64.cmx std/stack.cmx std/sort.cmx std/set.cmx std/queue.cmx std/printexc.cmx std/nativeint.cmx std/map.cmx std/camlinternalLazy.cmx std/lazy.cmx std/stream.cmx std/lexing.cmx std/int64.cmx std/hashtbl.cmx std/gc.cmx std/digest.cmx std/complex.cmx std/callback.cmx std/weak.cmx std/scanf.cmx std/random.cmx std/format.cmx std/parsing.cmx std/camlinternalOO.cmx std/camlinternalMod.cmx std/std_exit.cmx std/log.cmx std/oo.cmx std/lwt_sequence.cmx std/lwt.cmx std/lwt_util.cmx std/lwt_mutex.cmx std/lwt_stream.cmx std/lwt_mvar.cmx std/lwt_list.cmx std/lwt_pool.cmx std/lwt_condition.cmx std/lwt_pqueue.cmx std/lwt_switch.cmx std/bitstring_types.cmx std/bitstring_config.cmx std/bitstring.cmx std/bitstring_stream.cmx std/regexp.cmx std/oS.cmx std/net.cmx std/block.cmx std/dns.cmx std/http.cmx std/utf8.cmx std/ulexing.cmx std/utf16.cmx std/dyntype.cmx std/cow.cmx std/openflow.cmx std/oUnit.cmx std/fs.cmx -o std/stdlib.cmxa cc -c -Wall -g -O3 -I /usr/local/lib/ocaml -fPIC -o os/runtime_unix/main.o os/runtime_unix/main.c ../../../tools/ocp-pack/_build/split.native std/base64.annot ../../../tools/ocp-pack/_build/split.native std/string.annot ../../../tools/ocp-pack/_build/split.native std/stack.annot ../../../tools/ocp-pack/_build/split.native std/sort.annot ../../../tools/ocp-pack/_build/split.native std/set.annot ../../../tools/ocp-pack/_build/split.native std/queue.annot ../../../tools/ocp-pack/_build/split.native std/printexc.annot ../../../tools/ocp-pack/_build/split.native std/pervasives.annot ../../../tools/ocp-pack/_build/split.native std/nativeint.annot ../../../tools/ocp-pack/_build/split.native std/marshal.annot ../../../tools/ocp-pack/_build/split.native std/map.annot ../../../tools/ocp-pack/_build/split.native std/lazy.annot ../../../tools/ocp-pack/_build/split.native std/stream.annot ../../../tools/ocp-pack/_build/split.native std/list.annot ../../../tools/ocp-pack/_build/split.native std/lexing.annot ../../../tools/ocp-pack/_build/split.native std/int64.annot ../../../tools/ocp-pack/_build/split.native std/int32.annot ../../../tools/ocp-pack/_build/split.native std/hashtbl.annot ../../../tools/ocp-pack/_build/split.native std/gc.annot ../../../tools/ocp-pack/_build/split.native std/digest.annot ../../../tools/ocp-pack/_build/split.native std/complex.annot ../../../tools/ocp-pack/_build/split.native std/char.annot ../../../tools/ocp-pack/_build/split.native std/camlinternalLazy.annot ../../../tools/ocp-pack/_build/split.native std/callback.annot ../../../tools/ocp-pack/_build/split.native std/buffer.annot ../../../tools/ocp-pack/_build/split.native std/array.annot ../../../tools/ocp-pack/_build/split.native std/weak.annot ../../../tools/ocp-pack/_build/split.native std/sys.annot ../../../tools/ocp-pack/_build/split.native std/scanf.annot ../../../tools/ocp-pack/_build/split.native std/random.annot ../../../tools/ocp-pack/_build/split.native std/obj.annot ../../../tools/ocp-pack/_build/split.native std/format.annot ../../../tools/ocp-pack/_build/split.native std/printf.annot ../../../tools/ocp-pack/_build/split.native std/parsing.annot ../../../tools/ocp-pack/_build/split.native std/camlinternalOO.annot ../../../tools/ocp-pack/_build/split.native std/camlinternalMod.annot ../../../tools/ocp-pack/_build/split.native std/std_exit.annot ../../../tools/ocp-pack/_build/split.native std/log.annot ../../../tools/ocp-pack/_build/split.native std/oo.annot ../../../tools/ocp-pack/_build/split.native std/lwt.annot ../../../tools/ocp-pack/_build/split.native std/lwt_util.annot ../../../tools/ocp-pack/_build/split.native std/lwt_stream.annot ../../../tools/ocp-pack/_build/split.native std/lwt_sequence.annot ../../../tools/ocp-pack/_build/split.native std/lwt_mvar.annot ../../../tools/ocp-pack/_build/split.native std/lwt_mutex.annot ../../../tools/ocp-pack/_build/split.native std/lwt_list.annot ../../../tools/ocp-pack/_build/split.native std/lwt_pool.annot ../../../tools/ocp-pack/_build/split.native std/lwt_condition.annot ../../../tools/ocp-pack/_build/split.native std/lwt_pqueue.annot ../../../tools/ocp-pack/_build/split.native std/lwt_switch.annot ../../../tools/ocp-pack/_build/split.native std/bitstring.annot ../../../tools/ocp-pack/_build/split.native std/bitstring_types.annot ../../../tools/ocp-pack/_build/split.native std/bitstring_config.annot ../../../tools/ocp-pack/_build/split.native std/bitstring_stream.annot ../../../tools/ocp-pack/_build/split.native std/regexp.annot ../../../tools/ocp-pack/_build/split.native std/oS.annot ../../../tools/ocp-pack/_build/split.native std/net.annot ../../../tools/ocp-pack/_build/split.native std/block.annot ../../../tools/ocp-pack/_build/split.native std/dns.annot ../../../tools/ocp-pack/_build/split.native std/http.annot ../../../tools/ocp-pack/_build/split.native std/utf8.annot ../../../tools/ocp-pack/_build/split.native std/utf16.annot ../../../tools/ocp-pack/_build/split.native std/ulexing.annot ../../../tools/ocp-pack/_build/split.native std/dyntype.annot ../../../tools/ocp-pack/_build/split.native std/cow.annot ../../../tools/ocp-pack/_build/split.native std/openflow.annot ../../../tools/ocp-pack/_build/split.native std/oUnit.annot ../../../tools/ocp-pack/_build/split.native std/fs.annot Solver failed: Ocamlbuild knows of no rules that apply to a target named std/unix.itarget. This can happen if you ask Ocamlbuild to build a target with the wrong extension (e.g. .opt instead of .native) or if the source files live in directories that have not been specified as include directories. Backtrace: - Failed to build the target unix.otarget - Failed to build all of these: - Building std/unix.otarget: - Building std/unix.itarget - Building unix.otarget: - Building os/runtime_unix/libunixrun.a: - Failed to build all of these: - Building os/runtime_unix/libunixrun.cmx: - Failed to build all of these: - Building os/runtime_unix/libunixrun.ml: - Failed to build all of these: - Building os/runtime_unix/libunixrun.mly - Building os/runtime_unix/libunixrun.mll - Building os/runtime_unix/libunixrun.smlpack - Building os/runtime_unix/libunixrun.mlpack - Building os/runtime_unix/libunixrun.mllib - Failed to build all of these: - Building std/tap_stubs_os.o: - Failed to build all of these: - Building std/tap_stubs_os.c: - Building std/tap_stubs_freebsd.c - Building std/tap_stubs_os.ml: - Failed to build all of these: - Building std/tap_stubs_os.mly - Building std/tap_stubs_os.mll - Building std/tap_stubs_os.smlpack - Building std/tap_stubs_os.mlpack - Building std/tap_stubs_os.S - Building std/tap_stubs_os.c: - Building std/tap_stubs_freebsd.c - Building tap_stubs_os.o: - Failed to build all of these: - Building tap_stubs_os.c: - Building tap_stubs_freebsd.c - Building tap_stubs_os.ml: - Failed to build all of these: - Building tap_stubs_os.mly - Building tap_stubs_os.mll - Building tap_stubs_os.smlpack - Building tap_stubs_os.mlpack - Building tap_stubs_os.S - Building tap_stubs_os.c: - Building tap_stubs_freebsd.c - Building os/runtime_unix/tap_stubs_os.o: - Failed to build all of these: - Building os/runtime_unix/tap_stubs_os.c: - Building os/runtime_unix/tap_stubs_freebsd.c - Building os/runtime_unix/tap_stubs_os.ml: - Failed to build all of these: - Building os/runtime_unix/tap_stubs_os.mly - Building os/runtime_unix/tap_stubs_os.mll - Building os/runtime_unix/tap_stubs_os.smlpack - Building os/runtime_unix/tap_stubs_os.mlpack - Building os/runtime_unix/tap_stubs_os.S - Building os/runtime_unix/tap_stubs_os.c: - Building os/runtime_unix/tap_stubs_freebsd.c - Building os/runtime_unix/libunixrun.cclib gmake[1]: *** [unix-direct] Error 6 gmake[1]: Leaving directory `.../mirage/git.build/lib' gmake: *** [all] Error 2 -- Jim Pryor jim@jimpryor.net From avsm@dark.recoil.org Tue Dec 06 22:52:21 2011 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RY3s5-0005Hx-NA (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 06 Dec 2011 22:52:21 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1195874 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:44996 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp (csa=unknown) id 1RY3s5-0003AX-QZ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 06 Dec 2011 22:52:21 +0000 Received: (qmail 15365 invoked by uid 10000); 6 Dec 2011 22:52:21 -0000 Date: Tue, 6 Dec 2011 22:52:20 +0000 From: Anil Madhavapeddy To: lists+mirage@jimpryor.net Subject: Re: Trying to build on FreeBSD Message-ID: <20111206225220.GA8201@dark.recoil.org> References: <1323195929.21305.140661008212681@webmail.messagingengine.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1323195929.21305.140661008212681@webmail.messagingengine.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2011 22:52:21 -0000 X-Keywords: X-UID: 140 Status: O Content-Length: 1169 Lines: 30 On Tue, Dec 06, 2011 at 01:25:29PM -0500, lists+mirage@jimpryor.net wrote: > Hi all, this project looks very exciting. I was trying to build it for > FreeBSD, and can only get so far. Maybe someone more familiar with with > this project and with OCamlbuild can make some suggestions? > > Here are the changes I've made to the git repo, in an attempt to get > things working on FreeBSD: Hey Jim, You're almost there (and I'm impressed you waded this far through the rather complex ocamlbuild plugin). Since we don't have a configure script yet, there is a rule in lib/myocamlbuild.ml ("cc: _linux.c -> _os.c platform file") that symlinks in the right tuntap bindings for the UNIX backend. This is in lib/os/runtime_unix/tap_stubs_{macosx,linux}.c If you put in an equivalent for tap_stubs_freebsd.c, you should be all set. It's probably pretty similar to the MacOS X one, except for the header files. Let me know how you get on and I'll be happy to merge it. We're currently hacking away at benchmarking and eliminating leaks on the low-level code, so there'll be a few improvements merged for the OpenFlow stack soon (courtesy of Haris and Balraj). cheers Anil From lists+mirage@jimpryor.net Wed Dec 07 00:27:27 2011 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RY5M7-0006N8-2m (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 07 Dec 2011 00:27:27 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.6 from SpamAssassin-3.3.2-1195874 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [66.111.4.26 listed in list.dnswl.dnsbl.ja.net] * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from out2.smtp.messagingengine.com ([66.111.4.26]:47145) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp (csa=ok) id 1RY5M5-000059-sE (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 07 Dec 2011 00:27:27 +0000 Received: from compute4.internal (compute4.nyi.mail.srv.osa [10.202.2.44]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 4CDF42058F for ; Tue, 6 Dec 2011 19:27:25 -0500 (EST) Received: from web3.nyi.mail.srv.osa ([10.202.2.213]) by compute4.internal (MEProxy); Tue, 06 Dec 2011 19:27:25 -0500 Received: by web3.nyi.mail.srv.osa (Postfix, from userid 99) id 29A3740085; Tue, 6 Dec 2011 19:27:25 -0500 (EST) Message-Id: <1323217645.18051.140661008370313@webmail.messagingengine.com> X-Sasl-Enc: 2dwtWGj/BOKD52rDFlnQ1jm5bxqAC/0WMhmDNpQEGUgU 1323217645 From: lists+mirage@jimpryor.net To: cl-mirage@lists.cam.ac.uk MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: multipart/mixed; boundary="_----------=_1323217645180510" X-Mailer: MessagingEngine.com Webmail Interface Subject: Re: Trying to build on FreeBSD Date: Tue, 06 Dec 2011 19:27:25 -0500 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2011 00:27:27 -0000 X-Keywords: X-UID: 141 Status: O Content-Length: 12322 Lines: 234 This is a multi-part message in MIME format. --_----------=_1323217645180510 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii" Date: Tue, 6 Dec 2011 19:27:25 -0500 X-Mailer: MessagingEngine.com Webmail Interface Hi Anil, thanks for the quick reply. I had to do what you suggested to create a lib/os/runtime_unix/ev_freebsd.c file, as well. I just copied the Mac file for that; I'm not sure whether FreeBSD should choose SELECT or EPOLL. But at least I got everything to build. I'll attach my patches from git HEAD to this email; I'm not sure though if this list will accept attachments. I may be including unnecessary headers in the tap_stubs_freebsd.c file, but as I said, at least this builds. I had to change the shebang lines for the bash scripts, because on BSD systems bash is underneath /usr/local. I changed them to `/usr/bin/env bash ...` instead. That _should_ work, however for reasons I don't understand then `gmake install` fails. I had to change the shebang line on ./assemble.sh to /usr/local/bin/bash. I haven't yet been able to run a test though. This page says to do this: $ cd mirage.git/tests/basic/sleep $ mir-unix-direct sleep.bin $ ./_build/sleep.bin But that seems obsolete... the sleep files are now underneath regress/basic, not tests/basic. And I don't have any mir-unix-direct on my system, either in the source directory or in the install folder. Should I use `mir-build`, which is installed in the install folder? When I do so, I see some processing go on, but then I fail with this error: .../mirage/git.build/regress/basic $ mir-build sleep.bin Finished, 1 target (0 cached) in 00:00:00. Solver failed: Ocamlbuild knows of no rules that apply to a target named sleep.bin. This can happen if you ask Ocamlbuild to build a target with the wrong extension (e.g. .opt instead of .native) or if the source files live in directories that have not been specified as include directories. Backtrace: - Failed to build the target sleep.bin - Building sleep.bin Compilation unsuccessful after building 0 targets (0 cached) in 00:00:00. -- Jim Pryor jim@jimpryor.net -- Jim Pryor jim@jimpryor.net --_----------=_1323217645180510 MIME-Version: 1.0 Content-Disposition: attachment; filename="freebsd.patch" Content-Id: <7acfcaa0ccb319500a7977233d2c8a71f943d6e9@messagingengine.com> Content-Transfer-Encoding: base64 Content-Type: application/octet-stream; name="freebsd.patch" X-Mailer: MIME::Lite 3.027 (F2.77; A2.06; B3.08; Q3.08) Date: Tue, 6 Dec 2011 19:27:25 -0500 ZGlmZiAtLWdpdCBhL2Fzc2VtYmxlLnNoIGIvYXNzZW1ibGUuc2gKaW5kZXgg ODE3NDMwMy4uMjA5YjhmMyAxMDA3NTUKLS0tIGEvYXNzZW1ibGUuc2gKKysr IGIvYXNzZW1ibGUuc2gKQEAgLTEsNCArMSw0IEBACi0jIS9iaW4vYmFzaCAt ZQorIyEvdXNyL2Jpbi9lbnYgYmFzaCAtZQogIwogIyBBc3NlbWJsZSBhIGJ1 bmNoIG9mIHN0YW5kYXJkIGxpYnJhcnkga2VybmVscyBmb3IgdGhlCiAjIHZh cmlvdXMgT1MgY29uZmlndXJhdGlvbnM6CmRpZmYgLS1naXQgYS9saWIvbXlv Y2FtbGJ1aWxkLm1sIGIvbGliL215b2NhbWxidWlsZC5tbAppbmRleCBmNWQw MmU4Li43ZDkyZDkyIDEwMDY0NAotLS0gYS9saWIvbXlvY2FtbGJ1aWxkLm1s CisrKyBiL2xpYi9teW9jYW1sYnVpbGQubWwKQEAgLTg0LDE4ICs4NCwyMCBA QCBlbmQKICgqIE9TIGRldGVjdGlvbiAqKQogbW9kdWxlIE9TID0gc3RydWN0 CiAKLSAgdHlwZSB1ID0gTGludXggfCBEYXJ3aW4KKyAgdHlwZSB1ID0gTGlu dXggfCBEYXJ3aW4gfCBGcmVlQlNECiAgIHR5cGUgdCA9IFVuaXggb2YgdSB8 IFhlbiB8IE5vZGUKIAogICBsZXQgaG9zdCA9CiAgICAgbWF0Y2ggU3RyaW5n Lmxvd2VyY2FzZSAoVXRpbC5ydW5fYW5kX3JlYWQgInVuYW1lIC1zIikgd2l0 aAogICAgIHwgImxpbnV4IiAgLT4gVW5peCBMaW51eAogICAgIHwgImRhcndp biIgLT4gVW5peCBEYXJ3aW4KKyAgICB8ICJmcmVlYnNkIiAtPiBVbml4IEZy ZWVCU0QKICAgICB8IG9zIC0+IFByaW50Zi5lcHJpbnRmICJgJXNgIGlzIG5v dCBhIHN1cHBvcnRlZCBob3N0IE9TXG4iIG9zOyBleGl0ICgtMSkKIAogICBs ZXQgdW5peF9leHQgPSBtYXRjaCBob3N0IHdpdGgKICAgICB8IFVuaXggTGlu dXggIC0+ICJsaW51eCIKICAgICB8IFVuaXggRGFyd2luIC0+ICJtYWNvc3gi CisgICAgfCBVbml4IEZyZWVCU0QgLT4gImZyZWVic2QiCiAgICAgfCBfIC0+ IFByaW50Zi5lcHJpbnRmICJ1bml4X2V4dCBjYWxsZWQgb24gYSBub24tVU5J WCBob3N0IE9TXG4iOyBleGl0ICgtMSkKIAogZW5kCmRpZmYgLS1naXQgYS9s aWIvb3MvcnVudGltZV91bml4L2V2X2ZyZWVic2QuYyBiL2xpYi9vcy9ydW50 aW1lX3VuaXgvZXZfZnJlZWJzZC5jCm5ldyBmaWxlIG1vZGUgMTAwNjQ0Cmlu ZGV4IDAwMDAwMDAuLmU3Yjg0ZDEKLS0tIC9kZXYvbnVsbAorKysgYi9saWIv b3MvcnVudGltZV91bml4L2V2X2ZyZWVic2QuYwpAQCAtMCwwICsxLDQgQEAK KyNkZWZpbmUgRVZfU1RBTkRBTE9ORSAxCisjZGVmaW5lIEVWX1VTRV9OQU5P U0xFRVAgMQorI2RlZmluZSBFVl9VU0VfU0VMRUNUIDEKKyNpbmNsdWRlICJl di5jIgpkaWZmIC0tZ2l0IGEvbGliL29zL3J1bnRpbWVfdW5peC90YXBfc3R1 YnNfZnJlZWJzZC5jIGIvbGliL29zL3J1bnRpbWVfdW5peC90YXBfc3R1YnNf ZnJlZWJzZC5jCm5ldyBmaWxlIG1vZGUgMTAwNjQ0CmluZGV4IDAwMDAwMDAu LjAyYmYzMjIKLS0tIC9kZXYvbnVsbAorKysgYi9saWIvb3MvcnVudGltZV91 bml4L3RhcF9zdHVic19mcmVlYnNkLmMKQEAgLTAsMCArMSw3NCBAQAorLyoK KyAqIENvcHlyaWdodCAoYykgMjAxMCBBbmlsIE1hZGhhdmFwZWRkeSA8YW5p bEByZWNvaWwub3JnPgorICoKKyAqIFBlcm1pc3Npb24gdG8gdXNlLCBjb3B5 LCBtb2RpZnksIGFuZCBkaXN0cmlidXRlIHRoaXMgc29mdHdhcmUgZm9yIGFu eQorICogcHVycG9zZSB3aXRoIG9yIHdpdGhvdXQgZmVlIGlzIGhlcmVieSBn cmFudGVkLCBwcm92aWRlZCB0aGF0IHRoZSBhYm92ZQorICogY29weXJpZ2h0 IG5vdGljZSBhbmQgdGhpcyBwZXJtaXNzaW9uIG5vdGljZSBhcHBlYXIgaW4g YWxsIGNvcGllcy4KKyAqCisgKiBUSEUgU09GVFdBUkUgSVMgUFJPVklERUQg IkFTIElTIiBBTkQgVEhFIEFVVEhPUiBESVNDTEFJTVMgQUxMIFdBUlJBTlRJ RVMKKyAqIFdJVEggUkVHQVJEIFRPIFRISVMgU09GVFdBUkUgSU5DTFVESU5H IEFMTCBJTVBMSUVEIFdBUlJBTlRJRVMgT0YKKyAqIE1FUkNIQU5UQUJJTElU WSBBTkQgRklUTkVTUy4gSU4gTk8gRVZFTlQgU0hBTEwgVEhFIEFVVEhPUiBC RSBMSUFCTEUgRk9SCisgKiBBTlkgU1BFQ0lBTCwgRElSRUNULCBJTkRJUkVD VCwgT1IgQ09OU0VRVUVOVElBTCBEQU1BR0VTIE9SIEFOWSBEQU1BR0VTCisg KiBXSEFUU09FVkVSIFJFU1VMVElORyBGUk9NIExPU1MgT0YgVVNFLCBEQVRB IE9SIFBST0ZJVFMsIFdIRVRIRVIgSU4gQU4KKyAqIEFDVElPTiBPRiBDT05U UkFDVCwgTkVHTElHRU5DRSBPUiBPVEhFUiBUT1JUSU9VUyBBQ1RJT04sIEFS SVNJTkcgT1VUIE9GCisgKiBPUiBJTiBDT05ORUNUSU9OIFdJVEggVEhFIFVT RSBPUiBQRVJGT1JNQU5DRSBPRiBUSElTIFNPRlRXQVJFLgorICovCisKKyNp bmNsdWRlIDx1bmlzdGQuaD4KKyNpbmNsdWRlIDxzdGRsaWIuaD4KKyNpbmNs dWRlIDxzdGRpbnQuaD4KKyNpbmNsdWRlIDxzdGRpby5oPgorI2luY2x1ZGUg PHN0cmluZy5oPgorI2luY2x1ZGUgPGZjbnRsLmg+CisjaW5jbHVkZSA8ZXJy bm8uaD4KKyNpbmNsdWRlIDxlcnIuaD4KKworI2luY2x1ZGUgPGNhbWwvbWx2 YWx1ZXMuaD4KKyNpbmNsdWRlIDxjYW1sL21lbW9yeS5oPgorI2luY2x1ZGUg PGNhbWwvYWxsb2MuaD4KKyNpbmNsdWRlIDxjYW1sL2ZhaWwuaD4KKworI2lu Y2x1ZGUgPHN5cy90eXBlcy5oPgorI2luY2x1ZGUgPHN5cy9zb2NrZXQuaD4K KyNpbmNsdWRlIDxzeXMvaW9jdGwuaD4KKyNpbmNsdWRlIDxuZXQvaWYuaD4K Kworc3RhdGljIHZvaWQKK3NldG5vbmJsb2NrKGludCBmZCkKK3sKKyAgaW50 IGZsYWdzOworICBmbGFncyA9IGZjbnRsKGZkLCBGX0dFVEZMKTsKKyAgaWYg KGZsYWdzIDwgMCkKKyAgICBlcnIoMSwgInNldG5vbmJsb2NrOiBmY250bCIp OworICBmbGFncyB8PSBPX05PTkJMT0NLOworICBpZiAoZmNudGwoZmQsIEZf U0VURkwsIGZsYWdzKSA8IDApCisgICAgZXJyKDEsICJzZXRub25ibG9jaywg Rl9TRVRGTCIpOworfQorCitDQU1McHJpbSB2YWx1ZQordGFwX29wZW5kZXYo dmFsdWUgdl9zdHIpCit7CisgIGNoYXIgZGV2W0lGTkFNU0laXTsKKyAgY2hh ciBidWZbMTAyNF07CisgIGludCBmZDsKKworICBzbnByaW50ZihkZXYsIHNp emVvZiBkZXYsICIvZGV2LyVzIiwgU3RyaW5nX3ZhbCh2X3N0cikpOworCisg IGludCBkZXZfaWQ7CisKKyAgLy9zbWFsbCBoYWNrIHRvIGNyZWF0ZSBtdWx0 aXBsZSBpbnRlcmZhY2VzCisgIHNzY2FuZihkZXYsICIvZGV2L3RhcCVkIiwg JmRldl9pZCk7CisgIGZwcmludGYoc3RkZXJyLCAiSSBzaG91bGQgYmUgb3Bl bmluZyAxMC4lZC4wLjFcbiIsIGRldl9pZCk7CisKKyAgZnByaW50ZihzdGRl cnIsICJvcGVuZGV2OiAlc1xuIiwgZGV2KTsKKyAgZmQgPSBvcGVuKGRldiwg T19SRFdSKTsKKyAgaWYgKGZkIDwgMCkKKyAgICBlcnIoMSwgInRhcF9vcGVu ZGV2Iik7CisgIHNldG5vbmJsb2NrKGZkKTsKKyAgLyogTWFyayBpbnRlcmZh Y2UgYXMgdXAgKi8KKyAgc25wcmludGYoYnVmLCBzaXplb2YgYnVmLCAiL3Ni aW4vaWZjb25maWcgJXMgMTAuJWQuMC4yIG5ldG1hc2sgMjU1LjI1NS4yNTUu MCB1cCIsIFN0cmluZ192YWwodl9zdHIpLCBkZXZfaWQpOworICBmcHJpbnRm KHN0ZGVyciwgIiVzXG4iLCBidWYpOworICBzeXN0ZW0oYnVmKTsKKyAgcmV0 dXJuIFZhbF9pbnQoZmQpOworfQorCmRpZmYgLS1naXQgYS9yZWdyZXNzL2Zz L3JlYWRkaXIuc2ggYi9yZWdyZXNzL2ZzL3JlYWRkaXIuc2gKaW5kZXggYjNh ZWVjNS4uZDM2YmUzMCAxMDA3NTUKLS0tIGEvcmVncmVzcy9mcy9yZWFkZGly LnNoCisrKyBiL3JlZ3Jlc3MvZnMvcmVhZGRpci5zaApAQCAtMjgsNiArMjgs MTcgQEAgY2FzZSBgdW5hbWUgLXNgIGluCiAgICAgc3VkbyB1bW91bnQgdG1w bW91bnQKICAgOzsKIAorICBGcmVlQlNEICkKKyAgICBkZCBpZj0vZGV2L3pl cm8gb2Y9JHtPVVRQVVR9IGJzPTEgc2Vlaz0xMjhNIGNvdW50PTAKKyAgICAv c2Jpbi9uZXdmc19tc2RvcyAtTCBNaXJhZ2VGQVQgLUYgMTYgLXYgJHtPVVRQ VVR9CisgICAgcm0gLXJmIHRtcG1vdW50ICYmIG1rZGlyIHRtcG1vdW50Cisg ICAgTUQ9JChzdWRvIG1kY29uZmlnICR7T1VUUFVUfSkgCisgICAgbW91bnQg L2Rldi8kTUQgdG1wbW91bnQKKyAgICBzdWRvIGNwIC1yICR7SU5QVVR9IHRt cG1vdW50CisgICAgc3VkbyB1bW91bnQgdG1wbW91bnQgJiYgcm1kaXIgdG1w bW91bnQKKyAgICBzdWRvIG1kY29uZmlnIC1kIC11ICRNRAorICA7OworCiAg ICogKQogICAgIGVjaG8gVW5rbm93biBPUyBkZXRlY3RlZAogICAgIGV4aXQg MQpkaWZmIC0tZ2l0IGEvcmVncmVzcy9uZXQvZGVlbnMuc2ggYi9yZWdyZXNz L25ldC9kZWVucy5zaAppbmRleCAwZWNmNjMzLi4wMDZkZGE3IDEwMDY0NAot LS0gYS9yZWdyZXNzL25ldC9kZWVucy5zaAorKysgYi9yZWdyZXNzL25ldC9k ZWVucy5zaApAQCAtMSw0ICsxLDQgQEAKLSMhL2Jpbi9iYXNoIC1leAorIyEv dXNyL2Jpbi9lbnYgYmFzaCAtZXgKIAogY2FzZSAkMSBpbgogICBwcmVydW4g KSAKZGlmZiAtLWdpdCBhL3JlZ3Jlc3MvbmV0L3Bpbmcuc2ggYi9yZWdyZXNz L25ldC9waW5nLnNoCmluZGV4IDhjMDQ2MGIuLmU4NWUzNDAgMTAwNjQ0Ci0t LSBhL3JlZ3Jlc3MvbmV0L3Bpbmcuc2gKKysrIGIvcmVncmVzcy9uZXQvcGlu Zy5zaApAQCAtMSw0ICsxLDQgQEAKLSMhL2Jpbi9iYXNoIC1leAorIyEvdXNy L2Jpbi9lbnYgYmFzaCAtZXgKIAogZWNobyBwaW5nLnNoIGNhbGxlZDogJCoK IApkaWZmIC0tZ2l0IGEvc2NyaXB0cy9taXItZmF0LWNyZWF0ZSBiL3Njcmlw dHMvbWlyLWZhdC1jcmVhdGUKaW5kZXggYjZhY2FmMi4uYjYwNGVmNyAxMDA3 NTUKLS0tIGEvc2NyaXB0cy9taXItZmF0LWNyZWF0ZQorKysgYi9zY3JpcHRz L21pci1mYXQtY3JlYXRlCkBAIC0zOCw2ICszOCwxNyBAQCBjYXNlIGB1bmFt ZSAtc2AgaW4KICAgICBzdWRvIHVtb3VudCB0bXBtb3VudCAmJiBybWRpciB0 bXBtb3VudAogICA7OwogCisgIEZyZWVCU0QgKQorICAgIGRkIGlmPS9kZXYv emVybyBvZj0ke09VVFBVVH0gYnM9MSBzZWVrPTEyOE0gY291bnQ9MAorICAg IC9zYmluL25ld2ZzX21zZG9zIC1MIE1pcmFnZUZBVCAtRiAxNiAtdiAke09V VFBVVH0KKyAgICBybSAtcmYgdG1wbW91bnQgJiYgbWtkaXIgdG1wbW91bnQK KyAgICBNRD0kKHN1ZG8gbWRjb25maWcgJHtPVVRQVVR9KSAKKyAgICBtb3Vu dCAvZGV2LyRNRCB0bXBtb3VudAorICAgIHN1ZG8gY3AgLXIgJHtJTlBVVH0v KiB0bXBtb3VudAorICAgIHN1ZG8gdW1vdW50IHRtcG1vdW50ICYmIHJtZGly IHRtcG1vdW50CisgICAgc3VkbyBtZGNvbmZpZyAtZCAtdSAkTUQKKyAgOzsK KwogICAqICkKICAgICBlY2hvIFVua25vd24gT1MgZGV0ZWN0ZWQKICAgICBl eGl0IDEKZGlmZiAtLWdpdCBhL3NjcmlwdHMvbXlvY2FtbGJ1aWxkLm1sIGIv c2NyaXB0cy9teW9jYW1sYnVpbGQubWwKaW5kZXggOTBiNzE1Ni4uYWJmYzlm YyAxMDA2NDQKLS0tIGEvc2NyaXB0cy9teW9jYW1sYnVpbGQubWwKKysrIGIv c2NyaXB0cy9teW9jYW1sYnVpbGQubWwKQEAgLTU2LDIwICs1NiwyMSBAQCBl bmQKICgqKiBIb3N0IE9TIGRldGVjdGlvbiAqKQogbW9kdWxlIE9TID0gc3Ry dWN0CiAKLSAgdHlwZSB1bml4ID0gTGludXggfCBEYXJ3aW4KKyAgdHlwZSB1 bml4ID0gTGludXggfCBEYXJ3aW4gfCBGcmVlQlNECiAgIHR5cGUgYXJjaCA9 IFg4Nl8zMiB8IFg4Nl82NAogCiAgIGxldCBob3N0ID0KICAgICBtYXRjaCBT dHJpbmcubG93ZXJjYXNlIChVdGlsLnJ1bl9hbmRfcmVhZCAidW5hbWUgLXMi KSB3aXRoCiAgICAgfCAibGludXgiICAtPiBMaW51eAogICAgIHwgImRhcndp biIgLT4gRGFyd2luCisgICAgfCAiZnJlZWJzZCIgLT4gRnJlZUJTRAogICAg IHwgb3MgLT4gZXByaW50ZiAiYCVzYCBpcyBub3QgYSBzdXBwb3J0ZWQgaG9z dCBPU1xuIiBvczsgZXhpdCAoLTEpCiAKICAgbGV0IGFyY2ggPQogICAgIG1h dGNoIFN0cmluZy5sb3dlcmNhc2UgKFV0aWwucnVuX2FuZF9yZWFkICJ1bmFt ZSAtbSIpIHdpdGgKICAgICB8ICJ4ODZfMzIiIHwgImk2ODYiICAtPiBYODZf MzIKLSAgICB8ICJpMzg2IiAtPiAobWF0Y2ggaG9zdCB3aXRoIExpbnV4IC0+ IFg4Nl8zMiB8IERhcndpbiAtPiBYODZfNjQpCi0gICAgfCAieDg2XzY0IiAt PiBYODZfNjQKKyAgICB8ICJpMzg2IiAtPiAobWF0Y2ggaG9zdCB3aXRoIExp bnV4IHwgRnJlZUJTRCAtPiBYODZfMzIgfCBEYXJ3aW4gLT4gWDg2XzY0KQor ICAgIHwgIng4Nl82NCIgfCAiYW1kNjQiIC0+IFg4Nl82NAogICAgIHwgYXJj aCAtPiBlcHJpbnRmICJgJXNgIGlzIG5vdCBhIHN1cHBvcnRlZCBhcmNoXG4i IGFyY2g7IGV4aXQgKC0xKQogCiAgIGxldCBqc19vZl9vY2FtbF9pbnN0YWxs ZWQgPQpAQCAtODksOCArOTAsOCBAQCBtb2R1bGUgTWlyID0gc3RydWN0CiAg ICAgbGV0IHVuaXhtYWluIG1vZGUgPSBsaWIgLyBtb2RlIC8gImxpYiIgLyAi bWFpbi5vIiBpbgogICAgIGxldCBtb2RlID0gc3ByaW50ZiAidW5peC0lcyIg KGVudiAiJShtb2RlKSIpIGluCiAgICAgbGV0IGRsX2xpYnMgPSBtYXRjaCBo b3N0IHdpdGgKLSAgICAgIHxMaW51eCAgLT4gW0EiLWxtIjsgQSItbGFzbXJ1 biI7IEEiLWxjYW1sc3RyIjsgQSItbGRsIl0KLSAgICAgIHxEYXJ3aW4gLT4g W0EiLWxtIjsgQSItbGFzbXJ1biI7IEEiLWxjYW1sc3RyIl0gaW4KKyAgICAg IHxMaW51eCAtPiBbQSItbG0iOyBBIi1sYXNtcnVuIjsgQSItbGNhbWxzdHIi OyBBIi1sZGwiXQorICAgICAgfERhcndpbiB8RnJlZUJTRCAtPiBbQSItbG0i OyBBIi1sYXNtcnVuIjsgQSItbGNhbWxzdHIiXSBpbgogICAgIGxldCB0YWdz ID0gdGFncysrImNjIisrImMiIGluCiAgICAgQ21kIChTIChBIGNjIDo6IFsg VCh0YWdzKysibGluayIpOyBBIG9jYW1sY19saWJkaXI7IEEiLW8iOyBQeCBv dXQ7IAogICAgICAgICAgICAgIEEgKHVuaXhtYWluIG1vZGUpOyBQIGFyZzsg QSAodW5peHJ1biBtb2RlKTsgXSBAIGRsX2xpYnMpKQpkaWZmIC0tZ2l0IGEv c2NyaXB0cy94Y3Auc2ggYi9zY3JpcHRzL3hjcC5zaAppbmRleCBlZmJkZjgy Li4yZmFkMTQyIDEwMDc1NQotLS0gYS9zY3JpcHRzL3hjcC5zaAorKysgYi9z Y3JpcHRzL3hjcC5zaApAQCAtMSw0ICsxLDQgQEAKLSMhL2Jpbi9iYXNoCisj IS91c3IvYmluL2VudiBiYXNoCiAjIEJ1aWxkIGEgbWlyYWdlIFZNIG9uIFhl biBDbG91ZCBQbGF0Zm9ybQogCiBzZXQgLWUK --_----------=_1323217645180510-- From lists+mirage@jimpryor.net Wed Dec 07 01:04:28 2011 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RY5vw-0006ng-PW (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 07 Dec 2011 01:04:28 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.6 from SpamAssassin-3.3.2-1195874 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [66.111.4.26 listed in list.dnswl.dnsbl.ja.net] * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from out2.smtp.messagingengine.com ([66.111.4.26]:50096) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp (csa=ok) id 1RY5vw-00066C-D1 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 07 Dec 2011 01:04:28 +0000 Received: from compute4.internal (compute4.nyi.mail.srv.osa [10.202.2.44]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id AA8DB20CB6 for ; Tue, 6 Dec 2011 20:04:27 -0500 (EST) Received: from web3.nyi.mail.srv.osa ([10.202.2.213]) by compute4.internal (MEProxy); Tue, 06 Dec 2011 20:04:27 -0500 Received: by web3.nyi.mail.srv.osa (Postfix, from userid 99) id 88A4A40085; Tue, 6 Dec 2011 20:04:27 -0500 (EST) Message-Id: <1323219867.25958.140661008380657@webmail.messagingengine.com> X-Sasl-Enc: UkSyCn/yxr96dko+wzEANgNd1cNTr2RRNlY7rtNoLVBU 1323219867 From: lists+mirage@jimpryor.net To: cl-mirage@lists.cam.ac.uk MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii" X-Mailer: MessagingEngine.com Webmail Interface References: <1323217645.18051.140661008370313@webmail.messagingengine.com> Subject: Re: Trying to build on FreeBSD In-Reply-To: <1323217645.18051.140661008370313@webmail.messagingengine.com> Date: Tue, 06 Dec 2011 20:04:27 -0500 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2011 01:04:28 -0000 X-Keywords: X-UID: 142 Status: O Content-Length: 1132 Lines: 55 > I haven't yet been able to run a test though. This page > says to do this: > > $ cd mirage.git/tests/basic/sleep > $ mir-unix-direct sleep.bin > $ ./_build/sleep.bin > > But that seems obsolete... the sleep files are now underneath > regress/basic, not tests/basic. And I don't have any mir-unix-direct on > my system, either in the source directory or in the install folder. > Should I use `mir-build`, which is installed in the install folder? When > I do so, I see some processing go on, but then I fail... This seems to work: .../mirage/git.build/regress/basic $ mir-build unix-direct/sleep.bin Finished, 0 targets (0 cached) in 00:00:00. Finished, 18 targets (0 cached) in 00:00:00. .../mirage/git.build/regress/basic $ ./_build/unix-direct/sleep.bin Main: startup start: four four 1 start: three three 1 start: two two 1 start: one one 1 Devices: [Unix.Blkif] provider start Main: entering runloop four 2 one 2 two 2 three 2 four 3 one 3 two 3 three 3 four 4 one 4 four 5 done: two one 5 three 4 done: four done: one done: three Main: end -- Jim Pryor jim@jimpryor.net From avsm@dark.recoil.org Sun Dec 11 18:09:22 2011 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RZnpy-0002s3-Ky (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 11 Dec 2011 18:09:22 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1212747 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:45175 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp (csa=unknown) id 1RZnpx-0005U7-QH (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 11 Dec 2011 18:09:22 +0000 Received: (qmail 2340 invoked by uid 10000); 11 Dec 2011 18:09:20 -0000 Date: Sun, 11 Dec 2011 18:09:20 +0000 From: Anil Madhavapeddy To: lists+mirage@jimpryor.net Subject: Re: Trying to build on FreeBSD Message-ID: <20111211180920.GB29788@dark.recoil.org> References: <1323217645.18051.140661008370313@webmail.messagingengine.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1323217645.18051.140661008370313@webmail.messagingengine.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2011 18:09:22 -0000 X-Keywords: X-UID: 143 Status: O Content-Length: 2630 Lines: 60 On Tue, Dec 06, 2011 at 07:27:25PM -0500, lists+mirage@jimpryor.net wrote: > Hi Anil, thanks for the quick reply. I had to do what you suggested to > create a lib/os/runtime_unix/ev_freebsd.c file, as well. I just copied > the Mac file for that; I'm not sure whether FreeBSD should choose SELECT > or EPOLL. But at least I got everything to build. I'll attach my patches > from git HEAD to this email; I'm not sure though if this list will > accept attachments. FreeBSD should actually use kqueue, but I'm less convinced by the utility of libev these days; the bindings are quite complex and there's at least one SEGV bug lurking in there. > I had to change the shebang lines for the bash scripts, because on BSD > systems bash is underneath /usr/local. I changed them to `/usr/bin/env > bash ...` > instead. That _should_ work, however for reasons I don't understand then > `gmake install` fails. I had to change the shebang line on ./assemble.sh > to /usr/local/bin/bash. I'll fix these up to work on BSD make directly; the Makefiles are just wrappers now and don't do much except call ocamlbuild. > > I haven't yet been able to run a test though. This page > says to do this: > > $ cd mirage.git/tests/basic/sleep > $ mir-unix-direct sleep.bin > $ ./_build/sleep.bin Ah yes, that is out-of-date indeed: we did a big tutorial and neglected the wiki examples, but I've fixed them up now. The old build scheme was to have a separate command per-backend, but now the ocamlbuild integration has advanced enough that we just need to specify the backend as a virtual pathname prefix. So you do: $ mir-build unix-direct/sleep.bin or $ mir-build xen/sleep.xen # note the different extension I'm just refreshing my FreeBSD-current machine to see if I can get a Mirage Xen domU compiling on there. Shouldn't be any reason why not. We've been doing quite a few benchmarks on performance recently, and also some other work on I/O that have led to some ideas about improving the structure of the Mirage network stack considerably. Expect a new branch that strips things down and simplifies things considerably over the next month or so. In particular, it's probably time to split out some of the higher-level libraries into separate repos (regexp, http, etc) as other people want to use them outside Mirage too. I'd like Mirage-core to have everything up to TCP for networking for now, and focus on keeping it simple and adding robust profiling and visualisation support (sorely needed at the moment). -- Anil Madhavapeddy http://anil.recoil.org From h.rotsos@gmail.com Wed Dec 14 12:49:31 2011 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RaoH5-0005II-H2 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 14 Dec 2011 12:49:31 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1213599 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (h.rotsos[at]gmail.com) * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-qy0-f179.google.com ([209.85.216.179]:52109) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp (csa=unknown) id 1RaoH4-0004Sp-Fm (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 14 Dec 2011 12:49:31 +0000 Received: by qcol42 with SMTP id l42so504115qco.38 for ; Wed, 14 Dec 2011 04:49:30 -0800 (PST) MIME-Version: 1.0 Received: by 10.224.197.202 with SMTP id el10mr9381594qab.39.1323866970216; Wed, 14 Dec 2011 04:49:30 -0800 (PST) Sender: h.rotsos@gmail.com Received: by 10.229.124.78 with HTTP; Wed, 14 Dec 2011 04:49:30 -0800 (PST) Date: Wed, 14 Dec 2011 12:49:30 +0000 X-Google-Sender-Auth: 9NVlRRea8BUqyyaHym-x0lwkPkg Message-ID: Subject: Alternative Hashtbl and lwt yield-like functionality questions From: Haris Rotsos To: "cl-mirage@lists.cam.ac.uk Mailing List" Content-Type: text/plain; charset=UTF-8 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 12:49:31 -0000 X-Keywords: X-UID: 144 Status: O Content-Length: 1544 Lines: 41 Hi all, As the mail tittle suggests, I am sending this mail to ask some ocaml-oriented question. Firstly, I am writing a regression test for mirage in which I need to store a large number of entries in an Hash table. The test will create unique entries in the table with a rate of ~70K netries per second. My experiment so far shows that the performance of the Hashtbl insertions start to drop after a few seconds, as the memory management will start to consume cycles in order to allocate new buckets and possible rearrange the the distribution of the hashed values. The key I am using is a record of a string and a 64bit int. The string represents a mac address. Is any one aware of some other ocaml module that could satisfy the basic requirements of a hash table and would possible scale to large numbers of insertions? Secondly, I am currently using the lwt threading mechanism in order to handle all the asynvronous functionality of my code. I have a minor problem with the scheduler though as it tends to stick with a single thread during a test and doesn't schedule any other as long as it has data to process. Can somebody more experienced with lwt point me some calls or configuration parameters that would allow either a thread to yield explicitly, or implement a more fair scheduling policy. thanks in advance for any replies. -- Charalampos Rotsos PhD student The University of Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD Phone: +44-(0) 1223 767032 Email: cr409@cl.cam.ac.uk From raphlalou@gmail.com Wed Dec 14 13:52:44 2011 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RapGG-0006o1-4U (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 14 Dec 2011 13:52:44 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.0 from SpamAssassin-3.3.2-1213599 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (raphlalou[at]gmail.com) * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-gx0-f179.google.com ([209.85.161.179]:44987) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp (csa=unknown) id 1RapGF-0008Ai-rY (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 14 Dec 2011 13:52:44 +0000 Received: by ggnv5 with SMTP id v5so745898ggn.38 for ; Wed, 14 Dec 2011 05:52:43 -0800 (PST) MIME-Version: 1.0 Received: by 10.50.220.164 with SMTP id px4mr24357627igc.8.1323870762766; Wed, 14 Dec 2011 05:52:42 -0800 (PST) Received: by 10.42.142.6 with HTTP; Wed, 14 Dec 2011 05:52:42 -0800 (PST) In-Reply-To: References: Date: Wed, 14 Dec 2011 13:52:42 +0000 Message-ID: Subject: Re: Alternative Hashtbl and lwt yield-like functionality questions From: Raphael Proust To: Haris Rotsos Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "cl-mirage@lists.cam.ac.uk Mailing List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 13:52:44 -0000 X-Keywords: X-UID: 145 Status: O Content-Length: 1943 Lines: 46 On Wed, Dec 14, 2011 at 12:49 PM, Haris Rotsos wrote: > Hi all, > > As the mail tittle suggests, I am sending this mail to ask some > ocaml-oriented question. > > Firstly, I am writing a regression test for mirage in which I need to > store a large number of entries in an Hash table. The test will create > unique entries in the table with a rate of =C2=A0 ~70K netries per second= . > My experiment so far shows that the performance of the Hashtbl > insertions start to drop after a few seconds, as the memory management > will start to consume cycles in order to allocate new buckets and > possible rearrange the the distribution of the hashed values. The key > I am using is a record of a string and a 64bit int. The string > represents a mac address. Is any one aware of some other ocaml module > that could satisfy the basic requirements of a hash table and would > possible scale to large numbers of insertions? Not an expert there. I know there are regular exchange on the mailing list about Hashtables and such. Maybe looking at the archive might help=E2=80=A6 > > Secondly, I am currently using the lwt threading mechanism in order to > handle all the asynvronous functionality of my code. I have a minor > problem with the scheduler though as it tends to stick with a single > thread during a test and doesn't schedule any other as long as it has > data to process. Can somebody more experienced with lwt point me some > calls or configuration parameters that would allow either a thread to > yield explicitly, or implement a more fair scheduling policy. Lwt_unix.yield yields. If you are running in mirage (as I suspect you do) you can use Os.Time.sleep with 0. as argument (in the standard Lwt (and all the versions I know of) yield is just "sleep 0."). Ideally you can just make the function available through the interface file time.mli since it is already defined in time.ml. --=20 _______ Raphael From thomas.gazagnaire@gmail.com Wed Dec 14 14:04:42 2011 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RapRq-000798-IO (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 14 Dec 2011 14:04:42 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.0 from SpamAssassin-3.3.2-1213599 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-ey0-f179.google.com ([209.85.215.179]:56853) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp (csa=unknown) id 1RapRp-0006oN-Xw (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 14 Dec 2011 14:04:42 +0000 Received: by eaad12 with SMTP id d12so780209eaa.38 for ; Wed, 14 Dec 2011 06:04:41 -0800 (PST) Received: by 10.204.130.217 with SMTP id u25mr917450bks.81.1323871481152; Wed, 14 Dec 2011 06:04:41 -0800 (PST) Received: from [192.168.0.12] (gou06-3-88-170-165-56.fbx.proxad.net. [88.170.165.56]) by mx.google.com with ESMTPS id fg16sm6121475bkb.16.2011.12.14.06.04.39 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 14 Dec 2011 06:04:40 -0800 (PST) Subject: Re: Alternative Hashtbl and lwt yield-like functionality questions Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: Date: Wed, 14 Dec 2011 15:04:37 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <15F4ED8C-3F1F-47B8-8DDC-23BDBEADF4EE@gmail.com> References: To: Haris Rotsos X-Mailer: Apple Mail (2.1084) Cc: "cl-mirage@lists.cam.ac.uk Mailing List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 14:04:42 -0000 X-Keywords: X-UID: 146 Status: O Content-Length: 1983 Lines: 49 > Firstly, I am writing a regression test for mirage in which I need to > store a large number of entries in an Hash table. The test will create > unique entries in the table with a rate of ~70K netries per second. > My experiment so far shows that the performance of the Hashtbl > insertions start to drop after a few seconds, as the memory management > will start to consume cycles in order to allocate new buckets and > possible rearrange the the distribution of the hashed values. The key > I am using is a record of a string and a 64bit int. The string > represents a mac address. Is any one aware of some other ocaml module > that could satisfy the basic requirements of a hash table and would > possible scale to large numbers of insertions? How long does you test longs ? Do you really want to store your data in = a structured way ? What you can do to avoid too much allocation during your test is to = allocate an array each second (say of size 70K), and once filled, to = store it in a mutable list and allocate a new array to continue your = test. When you test is finished, you can analyze your mutable list and = build your association table using the data-structure you want. -- Thomas > Secondly, I am currently using the lwt threading mechanism in order to > handle all the asynvronous functionality of my code. I have a minor > problem with the scheduler though as it tends to stick with a single > thread during a test and doesn't schedule any other as long as it has > data to process. Can somebody more experienced with lwt point me some > calls or configuration parameters that would allow either a thread to > yield explicitly, or implement a more fair scheduling policy. >=20 > thanks in advance for any replies. >=20 >=20 > --=20 > Charalampos Rotsos > PhD student > The University of Cambridge > Computer Laboratory > William Gates Building > JJ Thomson Avenue > Cambridge > CB3 0FD >=20 > Phone: +44-(0) 1223 767032 > Email: cr409@cl.cam.ac.uk >=20 From h.rotsos@gmail.com Mon Dec 19 11:28:09 2011 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RcbO5-00022O-To (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 19 Dec 2011 11:28:09 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1220357 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (h.rotsos[at]gmail.com) * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-qy0-f179.google.com ([209.85.216.179]:36372) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp (csa=unknown) id 1RcbO5-0001YJ-Xg (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 19 Dec 2011 11:28:09 +0000 Received: by qcol42 with SMTP id l42so3179885qco.38 for ; Mon, 19 Dec 2011 03:28:08 -0800 (PST) MIME-Version: 1.0 Received: by 10.229.107.33 with SMTP id z33mr5272937qco.7.1324294088771; Mon, 19 Dec 2011 03:28:08 -0800 (PST) Sender: h.rotsos@gmail.com Received: by 10.229.124.78 with HTTP; Mon, 19 Dec 2011 03:28:08 -0800 (PST) In-Reply-To: References: Date: Mon, 19 Dec 2011 11:28:08 +0000 X-Google-Sender-Auth: UmTfn5XiFUo8c6WxPYZetZ8x_wY Message-ID: Subject: Re: Alternative Hashtbl and lwt yield-like functionality questions From: Haris Rotsos To: Raphael Proust Content-Type: text/plain; charset=UTF-8 Cc: "cl-mirage@lists.cam.ac.uk Mailing List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 11:28:10 -0000 X-Keywords: X-UID: 147 Status: O Content-Length: 747 Lines: 29 yeap this one worked. by the way the delay to switch thread is seems t be minimum and no impact on the performance of the code. :D On 14 December 2011 13:52, Raphael Proust wrote: > > Lwt_unix.yield yields. > > If you are running in mirage (as I suspect you do) you can use > Os.Time.sleep with 0. as argument (in the standard Lwt (and all the > versions I know of) yield is just "sleep 0."). > > Ideally you can just make the function available through the interface > file time.mli since it is already defined in time.ml. -- Charalampos Rotsos PhD student The University of Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD Phone: +44-(0) 1223 767032 Email: cr409@cl.cam.ac.uk From h.rotsos@gmail.com Mon Dec 19 11:30:48 2011 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RcbQd-0002A6-Up (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 19 Dec 2011 11:30:47 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1220357 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (h.rotsos[at]gmail.com) * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-qy0-f179.google.com ([209.85.216.179]:60180) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp (csa=unknown) id 1RcbQd-0002zO-Xn (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 19 Dec 2011 11:30:47 +0000 Received: by qcol42 with SMTP id l42so3181051qco.38 for ; Mon, 19 Dec 2011 03:30:46 -0800 (PST) MIME-Version: 1.0 Received: by 10.229.77.17 with SMTP id e17mr5268574qck.47.1324294246858; Mon, 19 Dec 2011 03:30:46 -0800 (PST) Sender: h.rotsos@gmail.com Received: by 10.229.124.78 with HTTP; Mon, 19 Dec 2011 03:30:46 -0800 (PST) In-Reply-To: <15F4ED8C-3F1F-47B8-8DDC-23BDBEADF4EE@gmail.com> References: <15F4ED8C-3F1F-47B8-8DDC-23BDBEADF4EE@gmail.com> Date: Mon, 19 Dec 2011 11:30:46 +0000 X-Google-Sender-Auth: 2u6ty6jERIxdRYptoay-j1SGbhI Message-ID: Subject: Re: Alternative Hashtbl and lwt yield-like functionality questions From: Haris Rotsos To: Thomas Gazagnaire Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "cl-mirage@lists.cam.ac.uk Mailing List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 11:30:48 -0000 X-Keywords: X-UID: 148 Status: O Content-Length: 1063 Lines: 31 thanks for the idea. Although, I don't think that an array is appropriate for my problem as I want to have the 0(1) lookup complexity that the Hashtbl provides. In any case, what I did was to reduce the size of the experiment. It was though a surprise to me how inefficient memory-wise is the Hashtbl in comparison to c structures. On 14 December 2011 14:04, Thomas Gazagnaire wrote: > How long does you test longs ? Do you really want to store your data in a= structured way ? > What you can do to avoid too much allocation during your test is to alloc= ate an array each second (say of size 70K), and once filled, to store it in= a mutable list and allocate a new array to continue your test. When you te= st is finished, you can analyze your mutable list and build your associatio= n table using the data-structure you want. --=20 Charalampos Rotsos PhD student The University of Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD Phone: +44-(0) 1223 767032 Email: cr409@cl.cam.ac.uk From thomas.gazagnaire@gmail.com Mon Dec 19 12:45:17 2011 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Rccaj-0004dC-Lc (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 19 Dec 2011 12:45:17 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.0 from SpamAssassin-3.3.2-1220357 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-ee0-f51.google.com ([74.125.83.51]:34467) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp (csa=unknown) id 1Rccaj-0005fn-Dg (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 19 Dec 2011 12:45:17 +0000 Received: by eekb57 with SMTP id b57so5072859eek.38 for ; Mon, 19 Dec 2011 04:45:17 -0800 (PST) Received: by 10.213.113.212 with SMTP id b20mr169121ebq.44.1324298716910; Mon, 19 Dec 2011 04:45:16 -0800 (PST) Received: from [192.168.0.12] (gou06-3-88-170-165-56.fbx.proxad.net. [88.170.165.56]) by mx.google.com with ESMTPS id t59sm26411638eeh.10.2011.12.19.04.45.15 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 19 Dec 2011 04:45:15 -0800 (PST) Subject: Re: Alternative Hashtbl and lwt yield-like functionality questions Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: Date: Mon, 19 Dec 2011 13:45:13 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <4625A802-8929-43AB-BECC-E04F4CA7EFE5@gmail.com> References: <15F4ED8C-3F1F-47B8-8DDC-23BDBEADF4EE@gmail.com> To: Haris Rotsos X-Mailer: Apple Mail (2.1084) Cc: "cl-mirage@lists.cam.ac.uk Mailing List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 12:45:17 -0000 X-Keywords: X-UID: 149 Status: O Content-Length: 1557 Lines: 46 > thanks for the idea. Although, I don't think that an array is > appropriate for my problem as I want to have the 0(1) lookup > complexity that the Hashtbl provides. I don't get it. Why do you need to read the data you are producing while = your test is running ? > In any case, what I did was to > reduce the size of the experiment. It was though a surprise to me how > inefficient memory-wise is the Hashtbl in comparison to c structures. it is really depending on the data-structure you are using. If you have = a know in advance the size of your data, and you want to have constant = time read/write, you can use arrays. If your data is not bound, the only = way it to amortize your costs. I'm curious to know which kind of C = structure does a better job here. -- Thomas > On 14 December 2011 14:04, Thomas Gazagnaire > wrote: >> How long does you test longs ? Do you really want to store your data = in a structured way ? >> What you can do to avoid too much allocation during your test is to = allocate an array each second (say of size 70K), and once filled, to = store it in a mutable list and allocate a new array to continue your = test. When you test is finished, you can analyze your mutable list and = build your association table using the data-structure you want. >=20 >=20 >=20 > --=20 > Charalampos Rotsos > PhD student > The University of Cambridge > Computer Laboratory > William Gates Building > JJ Thomson Avenue > Cambridge > CB3 0FD >=20 > Phone: +44-(0) 1223 767032 > Email: cr409@cl.cam.ac.uk From h.rotsos@gmail.com Mon Dec 19 13:05:17 2011 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Rccu5-00054w-GD (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 19 Dec 2011 13:05:17 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1220357 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (h.rotsos[at]gmail.com) * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-qy0-f179.google.com ([209.85.216.179]:56078) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp (csa=unknown) id 1Rccu4-0006aK-Si (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 19 Dec 2011 13:05:17 +0000 Received: by qcol42 with SMTP id l42so3230080qco.38 for ; Mon, 19 Dec 2011 05:05:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.229.107.33 with SMTP id z33mr5463236qco.7.1324299916032; Mon, 19 Dec 2011 05:05:16 -0800 (PST) Sender: h.rotsos@gmail.com Received: by 10.229.124.78 with HTTP; Mon, 19 Dec 2011 05:05:15 -0800 (PST) In-Reply-To: <4625A802-8929-43AB-BECC-E04F4CA7EFE5@gmail.com> References: <15F4ED8C-3F1F-47B8-8DDC-23BDBEADF4EE@gmail.com> <4625A802-8929-43AB-BECC-E04F4CA7EFE5@gmail.com> Date: Mon, 19 Dec 2011 13:05:15 +0000 X-Google-Sender-Auth: FKu4pX2h3qOHgAmtQQdBQeVGya4 Message-ID: Subject: Re: Alternative Hashtbl and lwt yield-like functionality questions From: Haris Rotsos To: Thomas Gazagnaire Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "cl-mirage@lists.cam.ac.uk Mailing List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 13:05:17 -0000 X-Keywords: X-UID: 150 Status: O Content-Length: 2577 Lines: 80 On 19 December 2011 12:45, Thomas Gazagnaire wrote: >> thanks for the idea. Although, I don't think that an array is >> appropriate for my problem as I want to have the 0(1) lookup >> complexity that the Hashtbl provides. > > I don't get it. Why do you need to read the data you are producing while = your test is running ? I am running a code that simulates the functionality of a learning_switch. The switch will inspect mac layer header and discover on which port it sees which mac address. So the Hashtbl needs to insert information while it reads data but also use that information to forward frames. > >> In any case, what I did was to >> reduce the size of the experiment. It was though a surprise to me how >> inefficient memory-wise is the Hashtbl in comparison to c structures. > > it is really depending on the data-structure you are using. If you have a= know in advance the size of your data, and you want to have constant time = read/write, you can use arrays. If your data is not bound, the only way it = to amortize your costs. I'm curious to know which kind of C structure does = a better job here. I am comparing against a similar implementation of the code that uses the boost hashtbl. While the test runs, the c++ code create a very small footprint, while the ocaml code starts bloating. As an example for the previous experiment a mirage vm will run out of memory after 20 seconds of insertion, while the c++ code will consume only a few megs. I guess this is the cost you pay when you want automatic memory management. > > -- > Thomas > >> On 14 December 2011 14:04, Thomas Gazagnaire >> wrote: >>> How long does you test longs ? Do you really want to store your data in= a structured way ? >>> What you can do to avoid too much allocation during your test is to all= ocate an array each second (say of size 70K), and once filled, to store it = in a mutable list and allocate a new array to continue your test. When you = test is finished, you can analyze your mutable list and build your associat= ion table using the data-structure you want. >> >> >> >> -- >> Charalampos Rotsos >> PhD student >> The University of Cambridge >> Computer Laboratory >> William Gates Building >> JJ Thomson Avenue >> Cambridge >> CB3 0FD >> >> Phone: +44-(0) 1223 767032 >> Email: cr409@cl.cam.ac.uk > > --=20 Charalampos Rotsos PhD student The University of Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD Phone: +44-(0) 1223 767032 Email: cr409@cl.cam.ac.uk From thomas.gazagnaire@gmail.com Mon Dec 19 13:17:09 2011 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Rcd5Z-0005RP-3l (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 19 Dec 2011 13:17:09 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.0 from SpamAssassin-3.3.2-1220357 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-ee0-f51.google.com ([74.125.83.51]:34836) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp (csa=unknown) id 1Rcd5Y-0007oj-XZ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 19 Dec 2011 13:17:09 +0000 Received: by eekb57 with SMTP id b57so5098099eek.38 for ; Mon, 19 Dec 2011 05:17:08 -0800 (PST) Received: by 10.213.29.141 with SMTP id q13mr2621171ebc.13.1324300628064; Mon, 19 Dec 2011 05:17:08 -0800 (PST) Received: from [192.168.0.12] (gou06-3-88-170-165-56.fbx.proxad.net. [88.170.165.56]) by mx.google.com with ESMTPS id q28sm44497484eea.6.2011.12.19.05.17.06 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 19 Dec 2011 05:17:07 -0800 (PST) Subject: Re: Alternative Hashtbl and lwt yield-like functionality questions Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: Date: Mon, 19 Dec 2011 14:17:04 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <82A31F2C-E18F-4282-A9C7-230B3B11432A@gmail.com> References: <15F4ED8C-3F1F-47B8-8DDC-23BDBEADF4EE@gmail.com> <4625A802-8929-43AB-BECC-E04F4CA7EFE5@gmail.com> To: Haris Rotsos X-Mailer: Apple Mail (2.1084) Cc: "cl-mirage@lists.cam.ac.uk Mailing List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 13:17:09 -0000 X-Keywords: X-UID: 151 Status: O Content-Length: 2723 Lines: 82 >> I don't get it. Why do you need to read the data you are producing = while your test is running ? >=20 > I am running a code that simulates the functionality of a > learning_switch. The switch will inspect mac layer header and discover > on which port it sees which mac address. So the Hashtbl needs to > insert information while it reads data but also use that information > to forward frames. Ok I see. Thx for the explanation. >>> In any case, what I did was to >>> reduce the size of the experiment. It was though a surprise to me = how >>> inefficient memory-wise is the Hashtbl in comparison to c = structures. >>=20 >> it is really depending on the data-structure you are using. If you = have a know in advance the size of your data, and you want to have = constant time read/write, you can use arrays. If your data is not bound, = the only way it to amortize your costs. I'm curious to know which kind = of C structure does a better job here. >=20 > I am comparing against a similar implementation of the code that uses > the boost hashtbl. While the test runs, the c++ code create a very > small footprint, while the ocaml code starts bloating. As an example > for the previous experiment a mirage vm will run out of memory after > 20 seconds of insertion, while the c++ code will consume only a few > megs. I guess this is the cost you pay when you want automatic memory > management. The default hashing algorithm is known to generate a lot of collisions = on some data-structure. Maybe you've hit such a case. It's strange to = have so much allocation. Anyway, I'm interested to look at your code if = it is available somewhere :-) -- Thomas >>> On 14 December 2011 14:04, Thomas Gazagnaire >>> wrote: >>>> How long does you test longs ? Do you really want to store your = data in a structured way ? >>>> What you can do to avoid too much allocation during your test is to = allocate an array each second (say of size 70K), and once filled, to = store it in a mutable list and allocate a new array to continue your = test. When you test is finished, you can analyze your mutable list and = build your association table using the data-structure you want. >>>=20 >>>=20 >>>=20 >>> -- >>> Charalampos Rotsos >>> PhD student >>> The University of Cambridge >>> Computer Laboratory >>> William Gates Building >>> JJ Thomson Avenue >>> Cambridge >>> CB3 0FD >>>=20 >>> Phone: +44-(0) 1223 767032 >>> Email: cr409@cl.cam.ac.uk >>=20 >>=20 >=20 >=20 >=20 > --=20 > Charalampos Rotsos > PhD student > The University of Cambridge > Computer Laboratory > William Gates Building > JJ Thomson Avenue > Cambridge > CB3 0FD >=20 > Phone: +44-(0) 1223 767032 > Email: cr409@cl.cam.ac.uk From anil@recoil.org Mon Dec 19 14:29:41 2011 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RceDl-0007Uq-Dx (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 19 Dec 2011 14:29:41 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1220357 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:33558 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp (csa=unknown) id 1RceDk-0001VC-ZR (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 19 Dec 2011 14:29:41 +0000 Received: (qmail 15380 invoked by uid 634); 19 Dec 2011 14:29:40 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from freak-0.srg.cl.cam.ac.uk (HELO [192.168.2.8]) (128.232.32.220) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Mon, 19 Dec 2011 14:29:40 +0000 Subject: Re: Alternative Hashtbl and lwt yield-like functionality questions Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Anil Madhavapeddy In-Reply-To: Date: Mon, 19 Dec 2011 14:29:39 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <1AD533D2-50A9-4E23-A708-8518CBA4BF67@recoil.org> References: <15F4ED8C-3F1F-47B8-8DDC-23BDBEADF4EE@gmail.com> <4625A802-8929-43AB-BECC-E04F4CA7EFE5@gmail.com> To: Haris Rotsos X-Mailer: Apple Mail (2.1251.1) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk Mailing List" , Thomas Gazagnaire X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 14:29:41 -0000 X-Keywords: X-UID: 152 Status: O Content-Length: 767 Lines: 18 On 19 Dec 2011, at 13:05, Haris Rotsos wrote: > I am comparing against a similar implementation of the code that uses > the boost hashtbl. While the test runs, the c++ code create a very > small footprint, while the ocaml code starts bloating. As an example > for the previous experiment a mirage vm will run out of memory after > 20 seconds of insertion, while the c++ code will consume only a few > megs. I guess this is the cost you pay when you want automatic memory > management. We should never actually *run out* of memory in the GC case; it should = trigger a collection and continue (albeit a bit more slowly). However, = if you are actually holding references to the intermediate values then = it will indeed run out of memory... as would Boost. Anil= From Chris.Greenhalgh@nottingham.ac.uk Tue Dec 20 12:18:50 2011 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Rcyeg-0006pj-OC (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 20 Dec 2011 12:18:50 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1220657 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from ixe-mta-19-tx.emailfiltering.com ([194.116.198.150]:53152 helo=ixe-mta-19.emailfiltering.com) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp (csa=unknown) id 1Rcyea-0006Od-X0 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 20 Dec 2011 12:18:50 +0000 Received: from smtp2.nottingham.ac.uk ([128.243.44.5]) by ixe-mta-19.emailfiltering.com with emfmta (version 4.8.5.64) by TLS id 2076029652 for cl-mirage@lists.cam.ac.uk; 345c355e9843240a; Tue, 20 Dec 2011 12:18:24 +0000 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp2.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1Rcyb8-0002JD-4f for cl-mirage@lists.cam.ac.uk; Tue, 20 Dec 2011 12:15:10 +0000 Received: from EXCHANGE1.ad.nottingham.ac.uk ([fe80::7962:f868:e6ee:6267]) by UIWEXHUB02.ad.nottingham.ac.uk ([2002:80f3:f84::80f3:f84]) with mapi; Tue, 20 Dec 2011 12:14:51 +0000 From: Christopher Greenhalgh To: "cl-mirage@lists.cam.ac.uk" Date: Tue, 20 Dec 2011 12:14:51 +0000 Subject: requests for clarification Thread-Topic: requests for clarification Thread-Index: Acy/EPmNzLhXOxi0QISRynXAgMp6MQ== Message-ID: <4C20415C7327A849961F3B84214137255D56E751CD@EXCHANGE1.ad.nottingham.ac.uk> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_4C20415C7327A849961F3B84214137255D56E751CDEXCHANGE1adno_" MIME-Version: 1.0 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 12:18:50 -0000 X-Keywords: X-UID: 153 Status: O Content-Length: 10272 Lines: 209 --_000_4C20415C7327A849961F3B84214137255D56E751CDEXCHANGE1adno_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I thought I'd use some of the vacation to try to get up to speed a bit with= mirage, and have a few questions and a couple of tutorial comments so far.= .. What is the status of/plans for the orm stuff? (https://github.com/mirage/o= rm - seems to be outside the current 'release' and tutorial scope) (plan A = for learning mirage was to try making a web application or two) Can I check my understanding on some of the networking stuff... - the unix-socket version uses sockets directly and doesn't try to= support the Ethif interface, whereas the unix-direct version does; this is= the common low level interface (on both Xen and Unix) on which the ocaml i= p stack is implemented (common to both targets)? - The Flow and esp. Channel and Manager interfaces are common abst= ractions across all platforms? I have to say I am pretty unclear exactly wh= at the role and function of the Manager is; I guess there is one per applic= ation, even if multiple interfaces, and there is some reference to "swap to= shared memory" but this seems to be a promissory note) Can someone comment on the wisdom of using the unix socket vs direct versio= n? (thinking of performance as well as stability) Is anyone working on the Node (or browser) version of the networking? (whic= h appears to just be an unimplemented shell) Some of the mirage build was broken (for me on Centos 6, anyway) by the rec= ent change to some of the .sh file headers (e.g. assemble.sh) from #!/bin/b= ash to #!/usr/bin/env bash -e: /usr/bin/env: bash -e: No such file or directory When I run any of the net examples as unix-direct (again on Centos 6), (hav= ing created tap0 explicitly using tunctl, which wasn't mentioned in the tut= orial), it appears (from what it prints and from subsequent ifconfig output= ) to execute each time: /sbin/ifconfig tap0 10.0.0.2 netmask 255.255.255.0 up But the mirage process is itself using ip 10.0.0.2, so I have to change the= host interface ip back to 10.0.0.1 (e.g.) before I can communicate with th= e mirage process on ip 10.0.0.2 Cheers Chris http://www.mrl.nott.ac.uk/~cmg/ = This message and any attachment are intended solely for the addressee a= nd may contain confidential information. If you have received this mess= age in error, please send it back to me, and immediately delete it. P= lease do not use, copy or disclose the information contained in this me= ssage or in any attachment. Any views or opinions expressed by the aut= hor of this email do not necessarily reflect the views of the Universit= y of Nottingham.=0D=0A=0D=0AThis message has been checked for viruses b= ut the contents of an attachment=0D=0Amay still contain software viruse= s which could damage your computer system:=0D=0Ayou are advised to perf= orm your own checks. Email communications with the=0D=0AUniversity of N= ottingham may be monitored as permitted by UK legislation.= --_000_4C20415C7327A849961F3B84214137255D56E751CDEXCHANGE1adno_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I thought I̵= 7;d use some of the vacation to try to get up to speed a bit with mirage, a= nd have a few questions and a couple of tutorial comments so far…

 

W= hat is the status of/plans for the orm stuff? (https://github.com/mirage/orm - seems to be outside the c= urrent ‘release’ and tutorial scope) (plan A for learning mirag= e was to try making a web application or two)

 

Can I check my understandin= g on some of the networking stuff…

-          <= /span>the unix-socket version uses sockets directly and doesnR= 17;t try to support the Ethif interface, whereas the unix-direct version do= es; this is the common low level interface (on both Xen and Unix) on which = the ocaml ip stack is implemented (common to both targets)?

<= p class=3DMsoListParagraph style=3D'text-indent:-18.0pt;mso-list:l0 level1 = lfo1'>-       &nb= sp;  The Flow and esp. Channel and Manager int= erfaces are common abstractions across all platforms? I have to say I am pr= etty unclear exactly what the role and function of the Manager is; I guess = there is one per application, even if multiple interfaces, and there is som= e reference to “swap to shared memory” but this seems to be a p= romissory note)

 

Can someone comment on the wisdom of using the unix socke= t vs direct version? (thinking of performance as well as stability)

 

Is an= yone working on the Node (or browser) version of the networking? (which app= ears to just be an unimplemented shell)

=  

Some of the mirage build was brok= en (for me on Centos 6, anyway) by the recent change to some of the .sh fil= e headers (e.g. assemble.sh) from #!/bin/bash to #!/usr/bin= /env bash -e:

/usr/bin/= env: bash -e: No such file or directory

 

When I run any of the net exa= mples as unix-direct (again on Centos 6), (having created tap0 explicitly u= sing tunctl, which wasn’t mentioned in the tutorial), it appears (fro= m what it prints and from subsequent ifconfig output) to execute each time:=

/sbin/ifconfig tap0 10.0.0.2 netmask 25= 5.255.255.0 up

But the mirage process is= itself using ip 10.0.0.2, so I have to change the host interface ip back t= o 10.0.0.1 (e.g.) before I can communicate with the mirage process on ip 10= .0.0.2

 

Cheers

Chris

http://www.mrl= .nott.ac.uk/~cmg/

 <= /p>

 

=
=0D=0A

=0D=0AThis message and any attachment are intended solely= for the addressee and may=20=0D=0Acontain confidential information. If= you have received this message in error,=20=0D=0Aplease send it back t= o me, and immediately delete it. Please do not use,=20=0D=0Acopy or d= isclose the information contained in this message or in any attachment.= =20=0D=0AAny views or opinions expressed by the author of this email d= o not necessarily=20=0D=0Areflect the views of the University of Nottin= gham.=0D=0A

=0D=0A

=0D=0AThis message has been checked for viruses= but the contents of an attachment=0D=0Amay still contain software viru= ses which could damage your computer system:=0D=0Ayou are advised to pe= rform your own checks. Email communications with the=0D=0AUniversity of= Nottingham may be monitored as permitted by UK legislation.=0D=0A

= = --_000_4C20415C7327A849961F3B84214137255D56E751CDEXCHANGE1adno_-- From Chris.Greenhalgh@nottingham.ac.uk Wed Dec 21 15:46:58 2011 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RdONe-0005Nh-9W (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 21 Dec 2011 15:46:58 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1221146 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from thb-mta-19-tx.emailfiltering.com ([194.116.199.150]:49495 helo=thb-mta-19.emailfiltering.com) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp (csa=unknown) id 1RdONY-0006M6-FG (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 21 Dec 2011 15:46:58 +0000 Received: from smtp2.nottingham.ac.uk ([128.243.44.5]) by thb-mta-19.emailfiltering.com with emfmta (version 4.8.5.64) by TLS id 2240699570 for cl-mirage@lists.cam.ac.uk; 138b150f6fc63dad; Wed, 21 Dec 2011 15:46:32 +0000 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp2.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1RdOIM-0003RI-R6 for cl-mirage@lists.cam.ac.uk; Wed, 21 Dec 2011 15:41:30 +0000 Received: from EXCHANGE1.ad.nottingham.ac.uk ([fe80::7962:f868:e6ee:6267]) by UIWEXHUB02.ad.nottingham.ac.uk ([2002:80f3:f84::80f3:f84]) with mapi; Wed, 21 Dec 2011 15:41:07 +0000 From: Christopher Greenhalgh To: "cl-mirage@lists.cam.ac.uk" Date: Wed, 21 Dec 2011 15:41:07 +0000 Subject: http post unimplemented Thread-Topic: http post unimplemented Thread-Index: Acy/9vT3eH+cMfjdRdeEJsmM7U33EA== Message-ID: <4C20415C7327A849961F3B84214137255D56E751EB@EXCHANGE1.ad.nottingham.ac.uk> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_4C20415C7327A849961F3B84214137255D56E751EBEXCHANGE1adno_" MIME-Version: 1.0 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 15:46:58 -0000 X-Keywords: X-UID: 154 Status: O Content-Length: 4238 Lines: 95 --_000_4C20415C7327A849961F3B84214137255D56E751EBEXCHANGE1adno_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Is there any particular reason why the http POST body handling is commented= out/marked TODO in https://github.com/avsm/mirage/blob/master/lib/http/req= uest.ml? Cheers Chris = This message and any attachment are intended solely for the addressee a= nd may contain confidential information. If you have received this mess= age in error, please send it back to me, and immediately delete it. P= lease do not use, copy or disclose the information contained in this me= ssage or in any attachment. Any views or opinions expressed by the aut= hor of this email do not necessarily reflect the views of the Universit= y of Nottingham.=0D=0A=0D=0AThis message has been checked for viruses b= ut the contents of an attachment=0D=0Amay still contain software viruse= s which could damage your computer system:=0D=0Ayou are advised to perf= orm your own checks. Email communications with the=0D=0AUniversity of N= ottingham may be monitored as permitted by UK legislation.= --_000_4C20415C7327A849961F3B84214137255D56E751EBEXCHANGE1adno_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Is there any par= ticular reason why the http POST body handling is commented out/marked TODO= in https://github.com/avsm/mirage/blob/master/lib/http/request.ml?

 

C= heers

Chris

 

=
=0D=0A

=0D=0AThis message and any attachment are intended solely= for the addressee and may=20=0D=0Acontain confidential information. If= you have received this message in error,=20=0D=0Aplease send it back t= o me, and immediately delete it. Please do not use,=20=0D=0Acopy or d= isclose the information contained in this message or in any attachment.= =20=0D=0AAny views or opinions expressed by the author of this email d= o not necessarily=20=0D=0Areflect the views of the University of Nottin= gham.=0D=0A

=0D=0A

=0D=0AThis message has been checked for viruses= but the contents of an attachment=0D=0Amay still contain software viru= ses which could damage your computer system:=0D=0Ayou are advised to pe= rform your own checks. Email communications with the=0D=0AUniversity of= Nottingham may be monitored as permitted by UK legislation.=0D=0A

= = --_000_4C20415C7327A849961F3B84214137255D56E751EBEXCHANGE1adno_-- From avsm@dark.recoil.org Wed Dec 21 23:28:38 2011 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RdVaQ-00011U-Ii (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 21 Dec 2011 23:28:38 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1221146 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:13640 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp (csa=unknown) id 1RdVaP-0008Ea-s7 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 21 Dec 2011 23:28:38 +0000 Received: (qmail 23022 invoked by uid 10000); 21 Dec 2011 23:28:37 -0000 Date: Wed, 21 Dec 2011 23:28:37 +0000 From: Anil Madhavapeddy To: Christopher Greenhalgh Subject: Re: requests for clarification Message-ID: <20111221232837.GK15450@dark.recoil.org> References: <4C20415C7327A849961F3B84214137255D56E751CD@EXCHANGE1.ad.nottingham.ac.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C20415C7327A849961F3B84214137255D56E751CD@EXCHANGE1.ad.nottingham.ac.uk> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 23:28:38 -0000 X-Keywords: X-UID: 155 Status: O Content-Length: 6009 Lines: 120 On Tue, Dec 20, 2011 at 12:14:51PM +0000, Christopher Greenhalgh wrote: > I thought I'd use some of the vacation to try to get up to speed a bit > with mirage, and have a few questions and a couple of tutorial comments > so far... Splendid! It's worth an initial note on where we are, since there's been quite a bit of background hacking recently towards various papers, and not all of it is merged yet. We've patched together a fairly complete protocol stack, but found a few deficiencies in the process: - Bitstrings are very efficient to read in a zero-copy fashion, but writing constructs many small strings that are copied several times. This draft paper on Reconfigurable I/O [1] has a C interface that I'm adapting at the moment for use in Mirage. Balraj and Haris have also done extensive work on TCP/OpenFlow, so we have a reasonably stable base now at least! - Profiling and characterising *why* something is slow under Xen is rather difficult currently, but should be easy to fix by implementing the gprof stubs in our microkernel, and adjusting the build system. - Several of the libraries should be able to be used independently of Mirage, most notably the xenstore implementation. Dave has pulled the code out (github/djs55/ocaml-xenstore), and I'm going to fix the Mirage build to use a git remote subrepo. Ideally, we should have a skeleton library repository that can compile for normal UNIX Lwt as well as Mirage. This is only practical for libraries which dont interact much with the system, but it's better than the current c&p situation. - The most existing thing left to integrate is FRP-style I/O [2]. This makes all kernel structures bidirectional. For example, rather than just having an ARP cache, anything that queries the cache will be 'tied' to it, and subsequent updates (e.g. an ARP packet or timeout) will result in a recomputation of the upstream flow. This is the essence of cloud programming, where there are lots of environmental changes (e.g. live relocation), and a Mirage kernel which handles them explicitly is an interesting experiment. So what this means is that the repository is going to undergo some short-term volatility through January :) I'll likely unhook most of the high-level protocol libraries from the build, and make the benchmark library work with Openflow/Ethernet first, and then up to TCP. Once we are happy that we can profile and stress test those layers, I'll bring the DNS/HTTP/etc libraries back in the new world. However, all this work will happen in a branch, so you can continue to mess around in the master branch to get a feel for the system. [1] http://anil.recoil.org/papers/drafts/2012-resolve-draft1.pdf [2] http://ambassadortothecomputers.blogspot.com/2010/05/how-froc-works.html > What is the status of/plans for the orm stuff? > (https://github.com/mirage/orm - seems to be outside the current > 'release' and tutorial scope) (plan A for learning mirage was to try > making a web application or two) The ORM was a bit of an early experiment. It certainly works, but the semantics of deletion are rather tricky. It's going to be a few months before it can be hooked back in. However, the easiest way to build a small web app is to compile in the data into the binary directly. > Can I check my understanding on some of the networking stuff... > > - the unix-socket version uses sockets directly and doesn't try > to support the Ethif interface, whereas the unix-direct version does; > this is the common low level interface (on both Xen and Unix) on which > the ocaml ip stack is implemented (common to both targets)? Yup. > > - The Flow and esp. Channel and Manager interfaces are common > abstractions across all platforms? I have to say I am pretty unclear > exactly what the role and function of the Manager is; I guess there is > one per application, even if multiple interfaces, and there is some > reference to "swap to shared memory" but this seems to be a promissory > note) The Manager was intended to be the system wide service (so yes, it could select libvchan sharedmem instead of TCP). However, that whole interface is going to change into a higher-level 'open a URI flow' instead. You would just open 'tcp://foo.bar:500' or 'xio://proc3' (for Mirage-internal calls). For now, just use the Manager from the examples but don't get too attached to it :) > Can someone comment on the wisdom of using the unix socket vs direct > version? (thinking of performance as well as stability) Sockets are there to make developing the higher level protocols easier, without worrying about TCP bugs. > Is anyone working on the Node (or browser) version of the networking? > (which appears to just be an unimplemented shell) Not really at the moment. It's a cool hack that keeps the architecture 'honest' (i.e. no C bindings), but doesn't really have a compelling purpose beyond that. Raphael did the most recent work, but it probably needs another week or so to fix up the I/O bindings to node.js (which uses a Buffer abstraction instead of Javascript strings) > > Some of the mirage build was broken (for me on Centos 6, anyway) by the > recent change to some of the .sh file headers (e.g. assemble.sh) from > #!/bin/bash to #!/usr/bin/env bash -e: /usr/bin/env: bash -e: No such > file or directory > > When I run any of the net examples as unix-direct (again on Centos 6), > (having created tap0 explicitly using tunctl, which wasn't mentioned in > the tutorial), it appears (from what it prints and from subsequent > ifconfig output) to execute each time: /sbin/ifconfig tap0 10.0.0.2 > netmask 255.255.255.0 up But the mirage process is itself using ip > 10.0.0.2, so I have to change the host interface ip back to 10.0.0.1 > (e.g.) before I can communicate with the mirage process on ip 10.0.0.2 Ah, these might be due to Haris' hacks to get OpenFlow to work. I'm paging back in to Mirage hacking now, so will tidy this stuff up too. -a From avsm@dark.recoil.org Thu Dec 22 00:00:31 2011 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RdW5H-0001Ew-RE (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 22 Dec 2011 00:00:31 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1221146 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:43644 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp (csa=unknown) id 1RdW5H-0001MH-qG (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 22 Dec 2011 00:00:31 +0000 Received: (qmail 24178 invoked by uid 10000); 22 Dec 2011 00:00:31 -0000 Date: Thu, 22 Dec 2011 00:00:31 +0000 From: Anil Madhavapeddy To: Christopher Greenhalgh Subject: Re: http post unimplemented Message-ID: <20111222000031.GO15450@dark.recoil.org> References: <4C20415C7327A849961F3B84214137255D56E751EB@EXCHANGE1.ad.nottingham.ac.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C20415C7327A849961F3B84214137255D56E751EB@EXCHANGE1.ad.nottingham.ac.uk> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 00:00:31 -0000 X-Keywords: X-UID: 156 Status: O Content-Length: 1290 Lines: 24 Due to the I/O streaming interface being in flux; the normal HTTP requests are copied into a string, whereas this didnt quite cut it for POSTs. Could be re-enabled if the POSTs are small... -a On Wed, Dec 21, 2011 at 03:41:07PM +0000, Christopher Greenhalgh wrote: > Is there any particular reason why the http POST body handling is > commented out/marked TODO in > https://github.com/avsm/mirage/blob/master/lib/http/request.ml? > > Cheers > Chris > > This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. > > This message has been checked for viruses but the contents of an attachment > may still contain software viruses which could damage your computer system: > you are advised to perform your own checks. Email communications with the > University of Nottingham may be monitored as permitted by UK legislation. -- Anil Madhavapeddy http://anil.recoil.org From Richard.Mortier@nottingham.ac.uk Thu Dec 22 09:38:08 2011 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Rdf6G-0001wX-Ew (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 22 Dec 2011 09:38:08 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1221646 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from ixe-mta-19-tx.emailfiltering.com ([194.116.198.150]:33020 helo=ixe-mta-19.emailfiltering.com) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp (csa=unknown) id 1Rdf6A-0004T9-rQ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 22 Dec 2011 09:38:08 +0000 Received: from smtp2.nottingham.ac.uk ([128.243.44.5]) by ixe-mta-19.emailfiltering.com with emfmta (version 4.8.5.64) by TLS id 2078276560 for cl-mirage@lists.cam.ac.uk; b56ed7f14e8a1ae9; Thu, 22 Dec 2011 09:37:42 +0000 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp2.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1Rdf4m-0004Wj-NE; Thu, 22 Dec 2011 09:36:36 +0000 Received: from EXCHANGE1.ad.nottingham.ac.uk ([fe80::7962:f868:e6ee:6267]) by UIWEXHUB02.ad.nottingham.ac.uk ([2002:80f3:f84::80f3:f84]) with mapi; Thu, 22 Dec 2011 09:36:10 +0000 From: Richard Mortier To: Anil Madhavapeddy Date: Thu, 22 Dec 2011 09:36:09 +0000 Subject: Re: requests for clarification Thread-Topic: requests for clarification Thread-Index: AczAjSMkpuH3YFXARvWAwloo8r/kNg== Message-ID: <16834300-E1E0-42C4-A851-10D2BE30E51C@nottingham.ac.uk> References: <4C20415C7327A849961F3B84214137255D56E751CD@EXCHANGE1.ad.nottingham.ac.uk> <20111221232837.GK15450@dark.recoil.org> In-Reply-To: <20111221232837.GK15450@dark.recoil.org> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Christopher Greenhalgh , "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 09:38:08 -0000 X-Keywords: X-UID: 157 Status: O Content-Length: 1931 Lines: 42 On 21 Dec 2011, at 23:28, Anil Madhavapeddy wrote: >> What is the status of/plans for the orm stuff? >> (https://github.com/mirage/orm - seems to be outside the current >> 'release' and tutorial scope) (plan A for learning mirage was to try >> making a web application or two) >=20 > The ORM was a bit of an early experiment. It certainly works, but the > semantics of deletion are rather tricky. It's going to be a few months > before it can be hooked back in. However, the easiest way to build a smal= l > web app is to compile in the data into the binary directly. imo it might ultimately be nice to try having a near-identical interface be= tween network and storage - kinda like sockets vs fds but type-safe. i gue= ss in such a case the orm would become more of a dynamically reconfigurable= un/marshalling stub generator service (cf. from nemesis days) operating on some a= ppropriate reader/writer interfaces. perhaps with barriers to stop frp fro= bbing stuff on disk - an interesting concept for the immutable datastore pe= rhaps? --=20 Cheers, R. = This message and any attachment are intended solely for the addressee a= nd may contain confidential information. If you have received this mess= age in error, please send it back to me, and immediately delete it. P= lease do not use, copy or disclose the information contained in this me= ssage or in any attachment. Any views or opinions expressed by the aut= hor of this email do not necessarily reflect the views of the Universit= y of Nottingham.=0D=0A=0D=0AThis message has been checked for viruses b= ut the contents of an attachment=0D=0Amay still contain software viruse= s which could damage your computer system:=0D=0Ayou are advised to perf= orm your own checks. Email communications with the=0D=0AUniversity of N= ottingham may be monitored as permitted by UK legislation.= From Chris.Greenhalgh@nottingham.ac.uk Thu Dec 22 13:08:47 2011 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RdiO7-0006mm-85 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 22 Dec 2011 13:08:47 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1221646 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from ixe-mta-19-tx.emailfiltering.com ([194.116.198.150]:57342 helo=ixe-mta-19.emailfiltering.com) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp (csa=unknown) id 1RdiO1-0001Tr-qz (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 22 Dec 2011 13:08:47 +0000 Received: from smtp1.nottingham.ac.uk ([128.243.44.4]) by ixe-mta-19.emailfiltering.com with emfmta (version 4.8.5.64) by TLS id 2078608280 for cl-mirage@lists.cam.ac.uk; 7c6e6126a9278755; Thu, 22 Dec 2011 13:08:21 +0000 Received: from uiwexhub01.ad.nottingham.ac.uk ([128.243.15.133]) by smtp1.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1RdiNg-0001us-MH for cl-mirage@lists.cam.ac.uk; Thu, 22 Dec 2011 13:08:20 +0000 Received: from EXCHANGE1.ad.nottingham.ac.uk ([fe80::7962:f868:e6ee:6267]) by UIWEXHUB01.ad.nottingham.ac.uk ([2002:80f3:f85::80f3:f85]) with mapi; Thu, 22 Dec 2011 13:08:20 +0000 From: Christopher Greenhalgh To: Anil Madhavapeddy Date: Thu, 22 Dec 2011 13:08:18 +0000 Subject: RE: requests for clarification Thread-Topic: requests for clarification Thread-Index: AczAOERM9DaMZcI+QcaGrj55k9Ky1gAXP3Bw Message-ID: <4C20415C7327A849961F3B84214137255D56E751F0@EXCHANGE1.ad.nottingham.ac.uk> References: <4C20415C7327A849961F3B84214137255D56E751CD@EXCHANGE1.ad.nottingham.ac.uk> <20111221232837.GK15450@dark.recoil.org> In-Reply-To: <20111221232837.GK15450@dark.recoil.org> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 13:08:47 -0000 X-Keywords: X-UID: 158 Status: O Content-Length: 7775 Lines: 161 Thanks.=20 So in terms of application persistence (with writing), it looks like the on= ly 'portable' option is Blkif, and the only Read/write abstraction over tha= t at the moment is the FAT filesystem? (the direct version of simpleKV runs over a Blkif, with its own format, but= is readonly; the socket version of simpleKV uses the UNIX native fd interf= ace directly over a host directory, rather than the unix version of the Blk= if which runs over a single file) So if I want something else I write my own persistence over Blkif? Or are t= here some other chunks of code sitting around that would be a better starti= ng point? Cheers Chris -----Original Message----- From: Anil Madhavapeddy [mailto:anil@recoil.org]=20 Sent: 21 December 2011 23:29 To: Christopher Greenhalgh Cc: cl-mirage@lists.cam.ac.uk Subject: Re: requests for clarification On Tue, Dec 20, 2011 at 12:14:51PM +0000, Christopher Greenhalgh wrote: > I thought I'd use some of the vacation to try to get up to speed a bit=20 > with mirage, and have a few questions and a couple of tutorial=20 > comments so far... Splendid! It's worth an initial note on where we are, since there's been qu= ite a bit of background hacking recently towards various papers, and not al= l of it is merged yet. We've patched together a fairly complete protocol stack, but found a few de= ficiencies in the process: - Bitstrings are very efficient to read in a zero-copy fashion, but writing constructs many small strings that are copied several times. This draft paper on Reconfigurable I/O [1] has a C interface that I'm adapting at th= e moment for use in Mirage. Balraj and Haris have also done extensive work on TCP/OpenFlow, so we have a reasonably stable base now at least! - Profiling and characterising *why* something is slow under Xen is rather difficult currently, but should be easy to fix by implementing the gprof stubs in our microkernel, and adjusting the build system.=20 - Several of the libraries should be able to be used independently of Mirage, most notably the xenstore implementation. Dave has pulled the code out (github/djs55/ocaml-xenstore), and I'm going to fix the Mirage build to use a git remote subrepo. Ideally, we should have a skeleton library repository that can compile for normal UNIX Lwt as well as Mirage. This is only practical for libraries which dont interact much with the system, but it's better than the current c&p situation. - The most existing thing left to integrate is FRP-style I/O [2]. This makes all kernel structures bidirectional. For example, rather than just having an ARP cache, anything that queries the cache will be 'tied' to it, and subsequent updates (e.g. an ARP packet or timeout) will result in a recomputation of the upstream flow. This is the essence of cloud programming, where there are lots of environmental changes (e.g. live relocation), and a Mirage kernel which handles them explicitly is an interesting experiment. So what this means is that the repository is going to undergo some short-te= rm volatility through January :) I'll likely unhook most of the high-level= protocol libraries from the build, and make the benchmark library work wit= h Openflow/Ethernet first, and then up to TCP. Once we are happy that we ca= n profile and stress test those layers, I'll bring the DNS/HTTP/etc librari= es back in the new world. However, all this work will happen in a branch, so you can continue to mess= around in the master branch to get a feel for the system. [1] http://anil.recoil.org/papers/drafts/2012-resolve-draft1.pdf [2] http://ambassadortothecomputers.blogspot.com/2010/05/how-froc-works.htm= l > What is the status of/plans for the orm stuff? > (https://github.com/mirage/orm - seems to be outside the current=20 > 'release' and tutorial scope) (plan A for learning mirage was to try=20 > making a web application or two) The ORM was a bit of an early experiment. It certainly works, but the seman= tics of deletion are rather tricky. It's going to be a few months before it= can be hooked back in. However, the easiest way to build a small web app i= s to compile in the data into the binary directly. > Can I check my understanding on some of the networking stuff... >=20 > - the unix-socket version uses sockets directly and doesn't try > to support the Ethif interface, whereas the unix-direct version does;=20 > this is the common low level interface (on both Xen and Unix) on which=20 > the ocaml ip stack is implemented (common to both targets)? Yup. >=20 > - The Flow and esp. Channel and Manager interfaces are common > abstractions across all platforms? I have to say I am pretty unclear=20 > exactly what the role and function of the Manager is; I guess there is=20 > one per application, even if multiple interfaces, and there is some=20 > reference to "swap to shared memory" but this seems to be a promissory > note) The Manager was intended to be the system wide service (so yes, it could se= lect libvchan sharedmem instead of TCP). However, that whole interface is g= oing to change into a higher-level 'open a URI flow' instead. You would jus= t open 'tcp://foo.bar:500' or 'xio://proc3' (for Mirage-internal calls). Fo= r now, just use the Manager from the examples but don't get too attached to= it :) > Can someone comment on the wisdom of using the unix socket vs direct=20 > version? (thinking of performance as well as stability) Sockets are there to make developing the higher level protocols easier, wit= hout worrying about TCP bugs. > Is anyone working on the Node (or browser) version of the networking? > (which appears to just be an unimplemented shell) Not really at the moment. It's a cool hack that keeps the architecture 'hon= est' (i.e. no C bindings), but doesn't really have a compelling purpose bey= ond that. Raphael did the most recent work, but it probably needs another w= eek or so to fix up the I/O bindings to node.js (which uses a Buffer abstra= ction instead of Javascript strings) >=20 > Some of the mirage build was broken (for me on Centos 6, anyway) by=20 > the recent change to some of the .sh file headers (e.g. assemble.sh)=20 > from #!/bin/bash to #!/usr/bin/env bash -e: /usr/bin/env: bash -e: No=20 > such file or directory >=20 > When I run any of the net examples as unix-direct (again on Centos 6),=20 > (having created tap0 explicitly using tunctl, which wasn't mentioned=20 > in the tutorial), it appears (from what it prints and from subsequent=20 > ifconfig output) to execute each time: /sbin/ifconfig tap0 10.0.0.2=20 > netmask 255.255.255.0 up But the mirage process is itself using ip=20 > 10.0.0.2, so I have to change the host interface ip back to 10.0.0.1 > (e.g.) before I can communicate with the mirage process on ip 10.0.0.2 Ah, these might be due to Haris' hacks to get OpenFlow to work. I'm paging = back in to Mirage hacking now, so will tidy this stuff up too.=20 -a = This message and any attachment are intended solely for the addressee a= nd may contain confidential information. If you have received this mess= age in error, please send it back to me, and immediately delete it. P= lease do not use, copy or disclose the information contained in this me= ssage or in any attachment. Any views or opinions expressed by the aut= hor of this email do not necessarily reflect the views of the Universit= y of Nottingham.=0D=0A=0D=0AThis message has been checked for viruses b= ut the contents of an attachment=0D=0Amay still contain software viruse= s which could damage your computer system:=0D=0Ayou are advised to perf= orm your own checks. Email communications with the=0D=0AUniversity of N= ottingham may be monitored as permitted by UK legislation.= From avsm@dark.recoil.org Sun Dec 25 15:40:45 2011 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ReqBp-0002nX-3d (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 25 Dec 2011 15:40:45 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1222937 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:41252 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp (csa=unknown) id 1ReqBn-0008NA-rL (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 25 Dec 2011 15:40:45 +0000 Received: (qmail 2638 invoked by uid 10000); 25 Dec 2011 15:40:43 -0000 Date: Sun, 25 Dec 2011 15:40:43 +0000 From: Anil Madhavapeddy To: Richard Mortier Subject: Re: requests for clarification Message-ID: <20111225154042.GA26304@dark.recoil.org> References: <4C20415C7327A849961F3B84214137255D56E751CD@EXCHANGE1.ad.nottingham.ac.uk> <20111221232837.GK15450@dark.recoil.org> <16834300-E1E0-42C4-A851-10D2BE30E51C@nottingham.ac.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <16834300-E1E0-42C4-A851-10D2BE30E51C@nottingham.ac.uk> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Christopher Greenhalgh , "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Dec 2011 15:40:45 -0000 X-Keywords: X-UID: 159 Status: O Content-Length: 1312 Lines: 25 On Thu, Dec 22, 2011 at 09:36:09AM +0000, Richard Mortier wrote: > > imo it might ultimately be nice to try having a near-identical interface > between network and storage - kinda like sockets vs fds but type-safe. > i guess in such a case the orm would become more of a dynamically > reconfigurable un/marshalling stub generator service (cf. > from > nemesis days) operating on some appropriate reader/writer interfaces. > perhaps with barriers to stop frp frobbing stuff on disk - an > interesting concept for the immutable datastore perhaps? The ORM can support introducing barriers automatically into autogen I/O code, but the main challenge is pinning down how to delete values. There's no garbage collector on disk, so where does the 'gc root' come from? Overall, I think OCaml has advanced quite a bit in the last few years from when we did the ORM. It now has first-class modules and GADTs (in trunk) that should make it easier to define an ORM without the need for so much code-generation. Id also like to integrate FRP into the way we persist values, so that one could (for example) have threads be activated when a variable on disk changes. Thus, editing a config file would actually side-effect and cause stuff to trigger. Anil From avsm@dark.recoil.org Sun Dec 25 15:51:27 2011 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ReqMB-0002ro-MH (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 25 Dec 2011 15:51:27 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1222937 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:1863 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp (csa=unknown) id 1ReqMA-00036K-sB (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 25 Dec 2011 15:51:27 +0000 Received: (qmail 27512 invoked by uid 10000); 25 Dec 2011 15:51:26 -0000 Date: Sun, 25 Dec 2011 15:51:26 +0000 From: Anil Madhavapeddy To: Christopher Greenhalgh Subject: Re: requests for clarification Message-ID: <20111225155126.GC26304@dark.recoil.org> References: <4C20415C7327A849961F3B84214137255D56E751CD@EXCHANGE1.ad.nottingham.ac.uk> <20111221232837.GK15450@dark.recoil.org> <4C20415C7327A849961F3B84214137255D56E751F0@EXCHANGE1.ad.nottingham.ac.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C20415C7327A849961F3B84214137255D56E751F0@EXCHANGE1.ad.nottingham.ac.uk> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Dec 2011 15:51:27 -0000 X-Keywords: X-UID: 160 Status: O Content-Length: 8487 Lines: 150 On Thu, Dec 22, 2011 at 01:08:18PM +0000, Christopher Greenhalgh wrote: > Thanks. > > So in terms of application persistence (with writing), it looks like the > only 'portable' option is Blkif, and the only Read/write abstraction > over that at the moment is the FAT filesystem? Thats right... the FAT filesystem 'should work' but its mainly been tested for read-only but read-write might still be buggy. I'll let Dave comment on that... > (the direct version of simpleKV runs over a Blkif, with its own format, > but is readonly; the socket version of simpleKV uses the UNIX native fd > interface directly over a host directory, rather than the unix version > of the Blkif which runs over a single file) > > So if I want something else I write my own persistence over Blkif? Or > are there some other chunks of code sitting around that would be a > better starting point? You could also extend the KV_RO into a KV_RW version that passes through the write call to the socket. However, at this stage it might be easier to just persist via a call to a remote service (like an Amazon S3 call or something). Depends how much hacking you feel like doing on Mirage really! -anil > > Cheers > Chris > > > > -----Original Message----- > From: Anil Madhavapeddy [mailto:anil@recoil.org] > Sent: 21 December 2011 23:29 > To: Christopher Greenhalgh > Cc: cl-mirage@lists.cam.ac.uk > Subject: Re: requests for clarification > > On Tue, Dec 20, 2011 at 12:14:51PM +0000, Christopher Greenhalgh wrote: > > I thought I'd use some of the vacation to try to get up to speed a bit > > with mirage, and have a few questions and a couple of tutorial > > comments so far... > > Splendid! It's worth an initial note on where we are, since there's been > quite a bit of background hacking recently towards various papers, and > not all of it is merged yet. > > We've patched together a fairly complete protocol stack, but found a few > deficiencies in the process: > > - Bitstrings are very efficient to read in a zero-copy fashion, but writing > constructs many small strings that are copied several times. This draft > paper on Reconfigurable I/O [1] has a C interface that I'm adapting at the > moment for use in Mirage. Balraj and Haris have also done extensive work > on TCP/OpenFlow, so we have a reasonably stable base now at least! > > - Profiling and characterising *why* something is slow under Xen is > rather difficult currently, but should be easy to fix by implementing the > gprof stubs in our microkernel, and adjusting the build system. > > - Several of the libraries should be able to be used independently of > Mirage, most notably the xenstore implementation. Dave has pulled the > code out (github/djs55/ocaml-xenstore), and I'm going to fix the Mirage > build to use a git remote subrepo. Ideally, we should have a skeleton > library repository that can compile for normal UNIX Lwt as well as > Mirage. This is only practical for libraries which dont interact much > with the system, but it's better than the current c&p situation. > > - The most existing thing left to integrate is FRP-style I/O [2]. This > makes all kernel structures bidirectional. For example, rather than just > having an ARP cache, anything that queries the cache will be 'tied' to > it, and subsequent updates (e.g. an ARP packet or timeout) will result > in a recomputation of the upstream flow. This is the essence of cloud > programming, where there are lots of environmental changes (e.g. live > relocation), and a Mirage kernel which handles them explicitly is an > interesting experiment. > > So what this means is that the repository is going to undergo some short-term volatility through January :) I'll likely unhook most of the high-level protocol libraries from the build, and make the benchmark library work with Openflow/Ethernet first, and then up to TCP. Once we are happy that we can profile and stress test those layers, I'll bring the DNS/HTTP/etc libraries back in the new world. > > However, all this work will happen in a branch, so you can continue to mess around in the master branch to get a feel for the system. > > [1] http://anil.recoil.org/papers/drafts/2012-resolve-draft1.pdf > [2] http://ambassadortothecomputers.blogspot.com/2010/05/how-froc-works.html > > > What is the status of/plans for the orm stuff? > > (https://github.com/mirage/orm - seems to be outside the current > > 'release' and tutorial scope) (plan A for learning mirage was to try > > making a web application or two) > > The ORM was a bit of an early experiment. It certainly works, but the semantics of deletion are rather tricky. It's going to be a few months before it can be hooked back in. However, the easiest way to build a small web app is to compile in the data into the binary directly. > > > Can I check my understanding on some of the networking stuff... > > > > - the unix-socket version uses sockets directly and doesn't try > > to support the Ethif interface, whereas the unix-direct version does; > > this is the common low level interface (on both Xen and Unix) on which > > the ocaml ip stack is implemented (common to both targets)? > > Yup. > > > > > - The Flow and esp. Channel and Manager interfaces are common > > abstractions across all platforms? I have to say I am pretty unclear > > exactly what the role and function of the Manager is; I guess there is > > one per application, even if multiple interfaces, and there is some > > reference to "swap to shared memory" but this seems to be a promissory > > note) > > The Manager was intended to be the system wide service (so yes, it could select libvchan sharedmem instead of TCP). However, that whole interface is going to change into a higher-level 'open a URI flow' instead. You would just open 'tcp://foo.bar:500' or 'xio://proc3' (for Mirage-internal calls). For now, just use the Manager from the examples but don't get too attached to it :) > > > Can someone comment on the wisdom of using the unix socket vs direct > > version? (thinking of performance as well as stability) > > Sockets are there to make developing the higher level protocols easier, without worrying about TCP bugs. > > > Is anyone working on the Node (or browser) version of the networking? > > (which appears to just be an unimplemented shell) > > Not really at the moment. It's a cool hack that keeps the architecture 'honest' (i.e. no C bindings), but doesn't really have a compelling purpose beyond that. Raphael did the most recent work, but it probably needs another week or so to fix up the I/O bindings to node.js (which uses a Buffer abstraction instead of Javascript strings) > > > > > Some of the mirage build was broken (for me on Centos 6, anyway) by > > the recent change to some of the .sh file headers (e.g. assemble.sh) > > from #!/bin/bash to #!/usr/bin/env bash -e: /usr/bin/env: bash -e: No > > such file or directory > > > > When I run any of the net examples as unix-direct (again on Centos 6), > > (having created tap0 explicitly using tunctl, which wasn't mentioned > > in the tutorial), it appears (from what it prints and from subsequent > > ifconfig output) to execute each time: /sbin/ifconfig tap0 10.0.0.2 > > netmask 255.255.255.0 up But the mirage process is itself using ip > > 10.0.0.2, so I have to change the host interface ip back to 10.0.0.1 > > (e.g.) before I can communicate with the mirage process on ip 10.0.0.2 > > Ah, these might be due to Haris' hacks to get OpenFlow to work. I'm paging back in to Mirage hacking now, so will tidy this stuff up too. > > -a > This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. > > This message has been checked for viruses but the contents of an attachment > may still contain software viruses which could damage your computer system: > you are advised to perform your own checks. Email communications with the > University of Nottingham may be monitored as permitted by UK legislation. -- Anil Madhavapeddy http://anil.recoil.org From Richard.Mortier@nottingham.ac.uk Fri Dec 30 10:40:42 2011 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RgZtB-00058y-VI (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 30 Dec 2011 10:40:41 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1225474 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from ixe-mta-19-tx.emailfiltering.com ([194.116.198.150]:55089 helo=ixe-mta-19.emailfiltering.com) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp (csa=unknown) id 1RgZt6-0003AV-EA (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 30 Dec 2011 10:40:41 +0000 Received: from smtp1.nottingham.ac.uk ([128.243.44.4]) by ixe-mta-19.emailfiltering.com with emfmta (version 4.8.5.64) by TLS id 2082860416 for cl-mirage@lists.cam.ac.uk; cfb0c1d2334a529b; Fri, 30 Dec 2011 10:40:15 +0000 Received: from uiwexhub01.ad.nottingham.ac.uk ([128.243.15.133]) by smtp1.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1RgZsl-0000OC-Jn; Fri, 30 Dec 2011 10:40:15 +0000 Received: from EXCHANGE1.ad.nottingham.ac.uk ([fe80::7962:f868:e6ee:6267]) by UIWEXHUB01.ad.nottingham.ac.uk ([2002:80f3:f85::80f3:f85]) with mapi; Fri, 30 Dec 2011 10:40:15 +0000 From: Richard Mortier To: Anil Madhavapeddy Date: Fri, 30 Dec 2011 10:40:14 +0000 Subject: Re: requests for clarification Thread-Topic: requests for clarification Thread-Index: AczG32pNm2qljRUiThi99AauutDM/w== Message-ID: References: <4C20415C7327A849961F3B84214137255D56E751CD@EXCHANGE1.ad.nottingham.ac.uk> <20111221232837.GK15450@dark.recoil.org> <16834300-E1E0-42C4-A851-10D2BE30E51C@nottingham.ac.uk> <20111225154042.GA26304@dark.recoil.org> In-Reply-To: <20111225154042.GA26304@dark.recoil.org> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Christopher Greenhalgh , "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 10:40:42 -0000 X-Keywords: X-UID: 161 Status: O Content-Length: 2844 Lines: 59 On 25 Dec 2011, at 15:40, Anil Madhavapeddy wrote: > On Thu, Dec 22, 2011 at 09:36:09AM +0000, Richard Mortier wrote: >>=20 >> imo it might ultimately be nice to try having a near-identical interface >> between network and storage - kinda like sockets vs fds but type-safe. >> i guess in such a case the orm would become more of a dynamically >> reconfigurable un/marshalling stub generator service (cf. >> from >> nemesis days) operating on some appropriate reader/writer interfaces. >> perhaps with barriers to stop frp frobbing stuff on disk - an >> interesting concept for the immutable datastore perhaps? >=20 > The ORM can support introducing barriers automatically into autogen I/O > code, but the main challenge is pinning down how to delete values. There'= s > no garbage collector on disk, so where does the 'gc root' come from? or perhaps the data structures on disk need to support different invariants= ? > Overall, I think OCaml has advanced quite a bit in the last few years fro= m > when we did the ORM. It now has first-class modules and GADTs (in trunk) > that should make it easier to define an ORM without the need for so much > code-generation. Id also like to integrate FRP into the way we persist > values, so that one could (for example) have threads be activated when a > variable on disk changes. Thus, editing a config file would actually > side-effect and cause stuff to trigger. yes- that is exactly the example i was thinking of - some straightforward w= ay to have the code that implements both archiving a versioned set of confi= g files (the "immutable append-only store"), and that links the on-disk rep= resentation with the live in-memory (or indeed, distributed over-network) v= ersion(s). =20 i do like the idea of having a config file edited outwith its owning applic= ation wake up the application to detect and respond to the change- which fe= els more like a network interaction than a normal storage interaction. --=20 Cheers, R. = This message and any attachment are intended solely for the addressee a= nd may contain confidential information. If you have received this mess= age in error, please send it back to me, and immediately delete it. P= lease do not use, copy or disclose the information contained in this me= ssage or in any attachment. Any views or opinions expressed by the aut= hor of this email do not necessarily reflect the views of the Universit= y of Nottingham.=0D=0A=0D=0AThis message has been checked for viruses b= ut the contents of an attachment=0D=0Amay still contain software viruse= s which could damage your computer system:=0D=0Ayou are advised to perf= orm your own checks. Email communications with the=0D=0AUniversity of N= ottingham may be monitored as permitted by UK legislation.= From avsm@dark.recoil.org Fri Dec 30 16:43:12 2011 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1RgfY0-0000cm-Aa (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 30 Dec 2011 16:43:12 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1225474 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:11750 helo=dark.recoil.org) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp (csa=unknown) id 1RgfXz-0001VY-F6 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 30 Dec 2011 16:43:12 +0000 Received: (qmail 24207 invoked by uid 10000); 30 Dec 2011 16:43:11 -0000 Date: Fri, 30 Dec 2011 16:43:11 +0000 From: Anil Madhavapeddy To: Richard Mortier Subject: Re: requests for clarification Message-ID: <20111230164311.GB14393@dark.recoil.org> References: <4C20415C7327A849961F3B84214137255D56E751CD@EXCHANGE1.ad.nottingham.ac.uk> <20111221232837.GK15450@dark.recoil.org> <16834300-E1E0-42C4-A851-10D2BE30E51C@nottingham.ac.uk> <20111225154042.GA26304@dark.recoil.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Christopher Greenhalgh , "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 16:43:12 -0000 X-Keywords: X-UID: 162 Status: O Content-Length: 1132 Lines: 29 On Fri, Dec 30, 2011 at 10:40:14AM +0000, Richard Mortier wrote: > > yes- that is exactly the example i was thinking of - some > straightforward way to have the code that implements both archiving a > versioned set of config files (the "immutable append-only store"), and > that links the on-disk representation with the live in-memory (or > indeed, distributed over-network) version(s). > > i do like the idea of having a config file edited outwith its owning > application wake up the application to detect and respond to the change- > which feels more like a network interaction than a normal storage > interaction. Yeah, and the main issue with using the ORM as it exists is that it uses SQLite, which isn't great for large immutable data structures. Interestingly though, there are some cool new storage systems being written in OCaml for NoSQL use. See: http://arakoon.org/ and more directly a storage backend: http://blog.incubaid.com/2011/12/23/announcing-baardskeerder/ It even uses Lwt! :-) The basic type-conv/dyntype infrastructure that is used by the ORM can be easily translated to a non-SQL backend too. -a