From anil@recoil.org Mon Sep 03 07:26:10 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1T8Q6s-00046k-TM (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 03 Sep 2012 07:26:10 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1375947 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:38380 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1T8Q6r-0005Es-S8 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 03 Sep 2012 07:26:10 +0100 Received: (qmail 30062 invoked by uid 634); 3 Sep 2012 06:26:09 -0000 Received: from c-69-181-69-199.hsd1.ca.comcast.net (HELO [192.168.1.105]) (69.181.69.199) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Mon, 03 Sep 2012 07:26:09 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: experiencing problems... From: Anil Madhavapeddy In-Reply-To: <3D2C25BE-204C-415A-908C-C4054885ECD8@nottingham.ac.uk> Date: Sun, 2 Sep 2012 23:26:05 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <3D2C25BE-204C-415A-908C-C4054885ECD8@nottingham.ac.uk> To: Richard Mortier X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 06:26:11 -0000 X-Keywords: X-UID: 525 Status: O Content-Length: 3537 Lines: 93 Ok, so I've checked through OPAM and promoted a number of the dev = repositories as stable versions on upstream OPAM (uri, re, cstruct, = cow). $ opam init default http://mirage.github.com/opam-0.4 $ opam install cstruct re uri oasis ...will work for now for those. The remaining ones in 'development' state are DNS and Cohttp. David = Sheets and I've done a very large fixup of Cohttp so that it can work = across Lwt-SSL/Async, and have made the Github bindings work really well = with them (we need this for the CUFP OCaml tutorial). For your immediate purposes, you just need to do an OPAM init and = install those packages, and do the benchmarking of cstruct vs bitstring. = The correct DNS repo is the mirage/ocaml-dns one (all of our repos are = moving towards the mirage/ organisation, so no confusion there). I'm hoping to merge the cohttp changes tomorrow when I get the Mirage = backend working, and then cut a release of that and the Github bindings = also. Thomas: do you have an ETA on a 'beta' version of OPAM that is stable? = We need to have a version that can be put on the CUFP VMs also, and it = seems pretty stable now. Haris: I'll look at the Openflow merge tomorrow just as soon as Cohttp = is done (that took some time to get right today). -anil On 31 Aug 2012, at 03:52, Richard Mortier = wrote: > trying (finally!) to get setup to do the performance test of = dns-cstruct vs dns-bitstring. seem to be having some problems which are = probably all solved somewhere. (and i also seem to be having problems = getting my irc tunnels setup to connect from the office else i'd ask = this on irc...) >=20 > * http://tutorial.openmirage.org/ currently returns a connection = refused >=20 > * opam init default ... per the website fails complaining that a = number of urls can't be found, eg., = http://mirage.github.com/opam/0o0755. dug out anil's last mail pointing = to the opam-0.4 repo and that seems to work though. >=20 > * minor niggle- just did a base install, and then opam switch-ed to = xen and then unix-direct versions to get them installed. seemed to use = wget to download urls.txt a number of times, without overwriting, so i = now have urls.txt.[1-9] in addition to urls.txt in = /home/rmm/.opam/repo/default/ -- is that intentional? >=20 > * (opam install dns) dependency on oasis, re, uri not captured >=20 > * (unix) opam install re fails as docs want to be built into = /usr/local/share/doc/ >=20 > lost track of opam development slightly - for the last two items, = should i just clone the repo at = (or = ?) and fix things? >=20 >=20 >=20 > --=20 > Cheers, >=20 > R. >=20 >=20 >=20 >=20 > This message and any attachment are intended solely for the addressee = and may contain confidential information. If you have received this = message in error, please send it back to me, and immediately delete it. = Please do not use, copy or disclose the information contained in this = message or in any attachment. Any views or opinions expressed by the = author of this email do not necessarily reflect the views of the = University of Nottingham. >=20 > This message has been checked for viruses but the contents of an = attachment > may still contain software viruses which could damage your computer = system: > you are advised to perform your own checks. Email communications with = the > University of Nottingham may be monitored as permitted by UK = legislation. From scott.dj@gmail.com Wed Sep 05 14:20:08 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1T9FWa-0007My-1u (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 05 Sep 2012 14:20:08 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.9 from SpamAssassin-3.3.2-1380492 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.220.179 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-vc0-f179.google.com ([209.85.220.179]:59800) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1T9FWW-0000tN-SI (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 05 Sep 2012 14:20:07 +0100 Received: by vcqp16 with SMTP id p16so691950vcq.38 for ; Wed, 05 Sep 2012 06:20:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.221.0.78 with SMTP id nl14mr2467192vcb.21.1346851203987; Wed, 05 Sep 2012 06:20:03 -0700 (PDT) Received: by 10.58.213.168 with HTTP; Wed, 5 Sep 2012 06:20:03 -0700 (PDT) Date: Wed, 5 Sep 2012 14:20:03 +0100 Message-ID: Subject: opam gotcha From: David Scott To: Mirage List Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 13:20:08 -0000 X-Keywords: X-UID: 526 Status: O Content-Length: 1476 Lines: 61 Hi, I just got bitten by this: 1. do some xen work: $ opam switch 3.12.1+mirage-xen To update your environment variables, you can now run: $ eval `opam config -env` djs@debian:~/ocaml-cstruct$ opam config -env PATH=/home/djs/.opam/3.12.1+mirage-xen/bin:/home/djs/mir-inst/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games; export PATH; OCAML_TOPLEVEL_PATH=/home/djs/.opam/3.12.1+mirage-xen/lib/toplevel; export OCAML_TOPLEVEL_PATH; MIRAGE_OS=xen; export MIRAGE_OS; MIRAGE_NET=direct; export MIRAGE_NET; Note the MIRAGE_* environment variables are exported 2. do some system work: $ opam switch system 3.12.1 /usr/lib/ocaml 3.12.1 To update your environment variables, you can now run: $ eval `opam config -env` djs@debian:~/ocaml-cstruct$ opam config -env PATH=/home/djs/.opam/system/bin:/home/djs/mir-inst/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games; export PATH; OCAML_TOPLEVEL_PATH=/home/djs/.opam/system/lib/toplevel; export OCAML_TOPLEVEL_PATH; CAML_LD_LIBRARY_PATH=/home/djs/.opam/system/lib/stublibs:/usr/lib/ocaml/stublibs; export CAML_LD_LIBRARY_PATH; Note the MIRAGE_* environment variables are still in my environment from the last time. 3. attempt to build cstruct, the Makefile has: OS ?= unix ifneq "$(MIRAGE_OS)" "" OS := $(MIRAGE_OS) endif ... so it does a xen build by mistake. I can manually unset these variables myself, now that I know I should. Is there a more automatic way? Cheers, -- Dave Scott From thomas.gazagnaire@gmail.com Wed Sep 05 14:28:37 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1T9Fen-0007fc-SI (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 05 Sep 2012 14:28:37 +0100 X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-wg0-f51.google.com ([74.125.82.51]:42683) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1T9Fcr-0008Qz-sH (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 05 Sep 2012 14:28:37 +0100 Received: by wgbed3 with SMTP id ed3so435798wgb.20 for ; Wed, 05 Sep 2012 06:26:37 -0700 (PDT) Received: by 10.216.99.199 with SMTP id x49mr14289076wef.171.1346851597635; Wed, 05 Sep 2012 06:26:37 -0700 (PDT) Received: from [192.168.51.14] ([82.235.36.67]) by mx.google.com with ESMTPS id bc2sm4703343wib.0.2012.09.05.06.26.31 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 05 Sep 2012 06:26:37 -0700 (PDT) Subject: Re: opam gotcha Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: Date: Wed, 5 Sep 2012 15:26:28 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: David Scott X-Mailer: Apple Mail (2.1084) Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 13:28:37 -0000 X-Keywords: X-UID: 527 Status: O Content-Length: 1927 Lines: 82 Hi Dave, Anil opened an issue on github for this: = https://github.com/OCamlPro/opam/issues/102 I'm not totally sure how I could fix this properly, but I'll try to = commit a fix for this today. -- Thomas On Sep 5, 2012, at 3:20 PM, David Scott wrote: > Hi, >=20 > I just got bitten by this: >=20 > 1. do some xen work: >=20 > $ opam switch 3.12.1+mirage-xen >=20 > To update your environment variables, you can now run: >=20 > $ eval `opam config -env` >=20 > djs@debian:~/ocaml-cstruct$ opam config -env > = PATH=3D/home/djs/.opam/3.12.1+mirage-xen/bin:/home/djs/mir-inst/bin:/usr/l= ocal/bin:/usr/bin:/bin:/usr/local/games:/usr/games; > export PATH; > OCAML_TOPLEVEL_PATH=3D/home/djs/.opam/3.12.1+mirage-xen/lib/toplevel; > export OCAML_TOPLEVEL_PATH; > MIRAGE_OS=3Dxen; export MIRAGE_OS; > MIRAGE_NET=3Ddirect; export MIRAGE_NET; >=20 > Note the MIRAGE_* environment variables are exported >=20 > 2. do some system work: >=20 > $ opam switch system > 3.12.1 > /usr/lib/ocaml > 3.12.1 >=20 > To update your environment variables, you can now run: >=20 > $ eval `opam config -env` >=20 > djs@debian:~/ocaml-cstruct$ opam config -env > = PATH=3D/home/djs/.opam/system/bin:/home/djs/mir-inst/bin:/usr/local/bin:/u= sr/bin:/bin:/usr/local/games:/usr/games; > export PATH; > OCAML_TOPLEVEL_PATH=3D/home/djs/.opam/system/lib/toplevel; export > OCAML_TOPLEVEL_PATH; > = CAML_LD_LIBRARY_PATH=3D/home/djs/.opam/system/lib/stublibs:/usr/lib/ocaml/= stublibs; > export CAML_LD_LIBRARY_PATH; >=20 > Note the MIRAGE_* environment variables are still in my environment > from the last time. >=20 > 3. attempt to build cstruct, the Makefile has: >=20 > OS ?=3D unix >=20 > ifneq "$(MIRAGE_OS)" "" > OS :=3D $(MIRAGE_OS) > endif >=20 > ... so it does a xen build by mistake. >=20 > I can manually unset these variables myself, now that I know I should. > Is there a more automatic way? >=20 > Cheers, > --=20 > Dave Scott >=20 From scott.dj@gmail.com Wed Sep 05 14:42:31 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1T9FsF-0000JW-JB (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 05 Sep 2012 14:42:31 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.9 from SpamAssassin-3.3.2-1380492 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.220.179 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-vc0-f179.google.com ([209.85.220.179]:35687) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1T9FsE-0008IX-mw (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 05 Sep 2012 14:42:31 +0100 Received: by vcqp16 with SMTP id p16so727146vcq.38 for ; Wed, 05 Sep 2012 06:42:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.119.198 with SMTP id a6mr2587373vcr.23.1346852549542; Wed, 05 Sep 2012 06:42:29 -0700 (PDT) Received: by 10.58.213.168 with HTTP; Wed, 5 Sep 2012 06:42:29 -0700 (PDT) In-Reply-To: References: Date: Wed, 5 Sep 2012 14:42:29 +0100 Message-ID: Subject: Re: opam gotcha From: David Scott To: Thomas Gazagnaire Content-Type: text/plain; charset=ISO-8859-1 Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 13:42:31 -0000 X-Keywords: X-UID: 528 Status: O Content-Length: 2217 Lines: 91 Hi, Thanks, I guess I should go straight to the issue tracker myself in future :-) The problem isn't urgent for me -- now that I understand what's happening I can easily work around it. Thanks, Dave On Wed, Sep 5, 2012 at 2:26 PM, Thomas Gazagnaire wrote: > Hi Dave, > > Anil opened an issue on github for this: https://github.com/OCamlPro/opam/issues/102 > > I'm not totally sure how I could fix this properly, but I'll try to commit a fix for this today. > > -- > Thomas > > On Sep 5, 2012, at 3:20 PM, David Scott wrote: > >> Hi, >> >> I just got bitten by this: >> >> 1. do some xen work: >> >> $ opam switch 3.12.1+mirage-xen >> >> To update your environment variables, you can now run: >> >> $ eval `opam config -env` >> >> djs@debian:~/ocaml-cstruct$ opam config -env >> PATH=/home/djs/.opam/3.12.1+mirage-xen/bin:/home/djs/mir-inst/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games; >> export PATH; >> OCAML_TOPLEVEL_PATH=/home/djs/.opam/3.12.1+mirage-xen/lib/toplevel; >> export OCAML_TOPLEVEL_PATH; >> MIRAGE_OS=xen; export MIRAGE_OS; >> MIRAGE_NET=direct; export MIRAGE_NET; >> >> Note the MIRAGE_* environment variables are exported >> >> 2. do some system work: >> >> $ opam switch system >> 3.12.1 >> /usr/lib/ocaml >> 3.12.1 >> >> To update your environment variables, you can now run: >> >> $ eval `opam config -env` >> >> djs@debian:~/ocaml-cstruct$ opam config -env >> PATH=/home/djs/.opam/system/bin:/home/djs/mir-inst/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games; >> export PATH; >> OCAML_TOPLEVEL_PATH=/home/djs/.opam/system/lib/toplevel; export >> OCAML_TOPLEVEL_PATH; >> CAML_LD_LIBRARY_PATH=/home/djs/.opam/system/lib/stublibs:/usr/lib/ocaml/stublibs; >> export CAML_LD_LIBRARY_PATH; >> >> Note the MIRAGE_* environment variables are still in my environment >> from the last time. >> >> 3. attempt to build cstruct, the Makefile has: >> >> OS ?= unix >> >> ifneq "$(MIRAGE_OS)" "" >> OS := $(MIRAGE_OS) >> endif >> >> ... so it does a xen build by mistake. >> >> I can manually unset these variables myself, now that I know I should. >> Is there a more automatic way? >> >> Cheers, >> -- >> Dave Scott >> > -- Dave Scott From anil@recoil.org Wed Sep 05 18:50:13 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1T9Jjx-0005E8-R7 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 05 Sep 2012 18:50:13 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1380492 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:16786 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1T9Jjw-0004vI-ZC (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 05 Sep 2012 18:50:13 +0100 Received: (qmail 29233 invoked by uid 634); 5 Sep 2012 17:50:12 -0000 Received: from 192-5-67-11.sri.com (HELO [10.255.241.243]) (192.5.67.11) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 05 Sep 2012 18:50:12 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: opam gotcha From: Anil Madhavapeddy In-Reply-To: Date: Wed, 5 Sep 2012 10:50:08 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <78AA66E6-E7D7-40ED-9DDA-BBB5F129CCAC@recoil.org> References: To: David Scott X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List , Thomas Gazagnaire X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 17:50:13 -0000 X-Keywords: X-UID: 529 Status: O Content-Length: 2992 Lines: 120 Sorry, I should have dropped a note to the list when I noticed this! = I've just aliased a shell script to unset MIRAGE_OS and MIRAGE_NET before calling = opam-switch. Thomas, perhaps one way to fix this is by having an opam-switch register = post-switch commands, and invoking those before any future switch? This one is = difficult to work around in build scripts, since the other (upstream) compiler = specifications know=20 nothing about our stuff... -anil On 5 Sep 2012, at 06:42, David Scott wrote: > Hi, >=20 > Thanks, I guess I should go straight to the issue tracker myself in = future :-) >=20 > The problem isn't urgent for me -- now that I understand what's > happening I can easily work around it. >=20 > Thanks, > Dave >=20 > On Wed, Sep 5, 2012 at 2:26 PM, Thomas Gazagnaire > wrote: >> Hi Dave, >>=20 >> Anil opened an issue on github for this: = https://github.com/OCamlPro/opam/issues/102 >>=20 >> I'm not totally sure how I could fix this properly, but I'll try to = commit a fix for this today. >>=20 >> -- >> Thomas >>=20 >> On Sep 5, 2012, at 3:20 PM, David Scott wrote: >>=20 >>> Hi, >>>=20 >>> I just got bitten by this: >>>=20 >>> 1. do some xen work: >>>=20 >>> $ opam switch 3.12.1+mirage-xen >>>=20 >>> To update your environment variables, you can now run: >>>=20 >>> $ eval `opam config -env` >>>=20 >>> djs@debian:~/ocaml-cstruct$ opam config -env >>> = PATH=3D/home/djs/.opam/3.12.1+mirage-xen/bin:/home/djs/mir-inst/bin:/usr/l= ocal/bin:/usr/bin:/bin:/usr/local/games:/usr/games; >>> export PATH; >>> OCAML_TOPLEVEL_PATH=3D/home/djs/.opam/3.12.1+mirage-xen/lib/toplevel; >>> export OCAML_TOPLEVEL_PATH; >>> MIRAGE_OS=3Dxen; export MIRAGE_OS; >>> MIRAGE_NET=3Ddirect; export MIRAGE_NET; >>>=20 >>> Note the MIRAGE_* environment variables are exported >>>=20 >>> 2. do some system work: >>>=20 >>> $ opam switch system >>> 3.12.1 >>> /usr/lib/ocaml >>> 3.12.1 >>>=20 >>> To update your environment variables, you can now run: >>>=20 >>> $ eval `opam config -env` >>>=20 >>> djs@debian:~/ocaml-cstruct$ opam config -env >>> = PATH=3D/home/djs/.opam/system/bin:/home/djs/mir-inst/bin:/usr/local/bin:/u= sr/bin:/bin:/usr/local/games:/usr/games; >>> export PATH; >>> OCAML_TOPLEVEL_PATH=3D/home/djs/.opam/system/lib/toplevel; export >>> OCAML_TOPLEVEL_PATH; >>> = CAML_LD_LIBRARY_PATH=3D/home/djs/.opam/system/lib/stublibs:/usr/lib/ocaml/= stublibs; >>> export CAML_LD_LIBRARY_PATH; >>>=20 >>> Note the MIRAGE_* environment variables are still in my environment >>> from the last time. >>>=20 >>> 3. attempt to build cstruct, the Makefile has: >>>=20 >>> OS ?=3D unix >>>=20 >>> ifneq "$(MIRAGE_OS)" "" >>> OS :=3D $(MIRAGE_OS) >>> endif >>>=20 >>> ... so it does a xen build by mistake. >>>=20 >>> I can manually unset these variables myself, now that I know I = should. >>> Is there a more automatic way? >>>=20 >>> Cheers, >>> -- >>> Dave Scott >>>=20 >>=20 >=20 >=20 >=20 > --=20 > Dave Scott >=20 From anil@recoil.org Wed Sep 05 23:25:48 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1T9O2e-0000K8-3W (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 05 Sep 2012 23:25:48 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1380492 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:24946 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1T9O2d-0000uK-Y2 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 05 Sep 2012 23:25:48 +0100 Received: (qmail 5819 invoked by uid 634); 5 Sep 2012 22:25:47 -0000 Received: from 192-5-67-11.sri.com (HELO [10.255.241.243]) (192.5.67.11) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 05 Sep 2012 23:25:47 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Updating the latest OPAM, small thing to note Message-Id: <43FB06A1-14E4-44E9-A192-DEA326A28571@recoil.org> Date: Wed, 5 Sep 2012 15:25:42 -0700 To: Mirage List Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 22:25:48 -0000 X-Keywords: X-UID: 530 Status: O Content-Length: 333 Lines: 9 If you are using the very latest OPAM and run into a compilation error, = then just run 'make distclean' before starting a new compilation. Some of the support packages have changed to support fancy git-style = man-pages if you run 'opam install --help', and the Makefile = dependencies aren't quite enough to detect that. -anil= From scott.dj@gmail.com Fri Sep 07 14:02:51 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1T9yCx-00044V-Sw (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 07 Sep 2012 14:02:51 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.9 from SpamAssassin-3.3.2-1381532 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.51 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-vb0-f51.google.com ([209.85.212.51]:43152) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1T9yCw-0006uP-oD (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 07 Sep 2012 14:02:51 +0100 Received: by vbbfn1 with SMTP id fn1so3034336vbb.38 for ; Fri, 07 Sep 2012 06:02:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.175.130 with SMTP id ca2mr5699251vdc.112.1347022965866; Fri, 07 Sep 2012 06:02:45 -0700 (PDT) Received: by 10.58.213.168 with HTTP; Fri, 7 Sep 2012 06:02:45 -0700 (PDT) Date: Fri, 7 Sep 2012 14:02:45 +0100 Message-ID: Subject: mirage-www From: David Scott To: Mirage List Content-Type: multipart/mixed; boundary=bcaec5014c1d627f7b04c91c39fa X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 13:02:52 -0000 X-Keywords: X-UID: 531 Status: O Content-Length: 16662 Lines: 297 --bcaec5014c1d627f7b04c91c39fa Content-Type: text/plain; charset=ISO-8859-1 Hi, I've built mirage-www for xen and modified the static IP to fit my environment. (I tried DHCP first but this didn't work -- if I get the time I'll try to debug). I can ping the server fine, and it's certainly receiving a lot of traffic on my (probably fairly busy) local network. When I try to fetch a URL the TCP connection hangs. On the console I get: Dispatch: dynamic URL / ... irrelevant spam TCP retransmission on timer seq = -889321980 I've attached a small tcpdump of the conversation. I started with ping and then tried HTTP. According to tcpdump/wireshark it went like this: $ tcpdump -r mirage.pcap -n reading from file mirage.pcap, link-type EN10MB (Ethernet) 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id 9300, seq 1, length 64 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id 9300, seq 1, length 64 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id 9300, seq 2, length 64 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id 9300, seq 2, length 64 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id 9300, seq 3, length 64 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id 9300, seq 3, length 64 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id 9300, seq 4, length 64 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id 9300, seq 4, length 64 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id 9300, seq 5, length 64 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id 9300, seq 5, length 64 -- so far so good, this is just my initial pings. Switching to 'wget http://10.80.239.140/' 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [S], seq 2284582709, win 5840, options [mss 1460,sackOK,TS val 909789846 ecr 0,nop,wscale 6], length 0 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [S.], seq 3536243828, ack 2284582710, win 65535, options [mss 1380,wscale 2,eol], length 0 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack 1, win 92, length 0 -- TCP connection established 13:47:14.242417 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [P.], seq 1:112, ack 1, win 92, length 111 -- HTTP GET / sent 13:47:14.242869 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], seq 1:18, ack 112, win 65535, length 17 -- HTTP/1.1 200 OK replied 13:47:14.242880 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack 18, win 92, length 0 13:47:14.243369 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], seq 18:68, ack 112, win 65535, length 50 -- "content-length..." replied 13:47:14.243556 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], seq 68:1448, ack 112, win 65535, length 1380 ... after this chunks of the blog post are transmitted 13:47:14.243562 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack 18, win 92, length 0 13:47:14.243566 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], seq 1448:2828, ack 112, win 65535, length 1380 13:47:14.243571 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack 18, win 92, length 0 13:47:14.243575 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], seq 2828:4164, ack 112, win 65535, length 1336 13:47:14.243579 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack 18, win 92, length 0 13:47:14.243602 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], seq 4164:5544, ack 112, win 65535, length 1380 13:47:14.243608 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack 18, win 92, length 0 13:47:14.243776 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], seq 18:68, ack 112, win 65535, length 50 -- fast retransmit of the "content-length" packet 13:47:18.242606 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], seq 18:68, ack 112, win 65535, length 50 -- and again 13:47:20.242782 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], seq 18:68, ack 112, win 65535, length 50 -- and again 13:47:24.242904 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], seq 18:68, ack 112, win 65535, length 50 -- and again 13:47:32.243159 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], seq 18:68, ack 112, win 65535, length 50 -- and again 13:47:48.243262 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], seq 18:68, ack 112, win 65535, length 50 I tried a ping flood, and all is ok at this level: djs@ely:~$ sudo ping -f 10.80.239.140 PING 10.80.239.140 (10.80.239.140) 56(84) bytes of data. .^C --- 10.80.239.140 ping statistics --- 9930 packets transmitted, 9929 received, 0% packet loss, time 2556ms rtt min/avg/max/mdev = 0.166/0.242/1.762/0.064 ms, ipg/ewma 0.257/0.252 ms It looks like a problem in TCP. Anyone got any hints where to look? Cheers, -- Dave Scott --bcaec5014c1d627f7b04c91c39fa Content-Type: application/octet-stream; name="mirage.pcap" Content-Disposition: attachment; filename="mirage.pcap" Content-Transfer-Encoding: base64 X-Attachment-Id: f_h6ta7ri50 1MOyoQIABAAAAAAAAAAAANwFAAABAAAAy+xJUI9JCABiAAAAYgAAAAAADAesAQAQGDJRoAgARQAA VAAAQABAATRdClACIApQ74wIAE8hJFQAAcvsSVB8SQgACAkKCwwNDg8QERITFBUWFxgZGhscHR4f ICEiIyQlJicoKSorLC0uLzAxMjM0NTY3y+xJUAxMCABiAAAAYgAAAAAQGDJRoFR10OGhvwgARQAA VO4hAAAkAaI7ClDvjApQAiAAAFchJFQAAcvsSVB8SQgACAkKCwwNDg8QERITFBUWFxgZGhscHR4f ICEiIyQlJicoKSorLC0uLzAxMjM0NTY3zOxJUKBFCABiAAAAYgAAAAAADAesAQAQGDJRoAgARQAA VAAAQABAATRdClACIApQ74wIADUkJFQAAszsSVCVRQgACAkKCwwNDg8QERITFBUWFxgZGhscHR4f ICEiIyQlJicoKSorLC0uLzAxMjM0NTY3zOxJUNZGCABiAAAAYgAAAAAQGDJRoFR10OGhvwgARQAA VJZcAAAkAfoAClDvjApQAiAAAD0kJFQAAszsSVCVRQgACAkKCwwNDg8QERITFBUWFxgZGhscHR4f ICEiIyQlJicoKSorLC0uLzAxMjM0NTY3zexJUGhCCABiAAAAYgAAAAAADAesAQAQGDJRoAgARQAA VAAAQABAATRdClACIApQ74wIAHAmJFQAA83sSVBZQggACAkKCwwNDg8QERITFBUWFxgZGhscHR4f ICEiIyQlJicoKSorLC0uLzAxMjM0NTY3zexJUKlDCABiAAAAYgAAAAAQGDJRoFR10OGhvwgARQAA VHosAAAkARYxClDvjApQAiAAAHgmJFQAA83sSVBZQggACAkKCwwNDg8QERITFBUWFxgZGhscHR4f ICEiIyQlJicoKSorLC0uLzAxMjM0NTY3zuxJUGZCCABiAAAAYgAAAAAADAesAQAQGDJRoAgARQAA VAAAQABAATRdClACIApQ74wIAHElJFQABM7sSVBXQggACAkKCwwNDg8QERITFBUWFxgZGhscHR4f ICEiIyQlJicoKSorLC0uLzAxMjM0NTY3zuxJUIxDCABiAAAAYgAAAAAQGDJRoFR10OGhvwgARQAA VFBRAAAkAUAMClDvjApQAiAAAHklJFQABM7sSVBXQggACAkKCwwNDg8QERITFBUWFxgZGhscHR4f ICEiIyQlJicoKSorLC0uLzAxMjM0NTY3z+xJUGZCCABiAAAAYgAAAAAADAesAQAQGDJRoAgARQAA VAAAQABAATRdClACIApQ74wIAHAkJFQABc/sSVBXQggACAkKCwwNDg8QERITFBUWFxgZGhscHR4f ICEiIyQlJicoKSorLC0uLzAxMjM0NTY3z+xJUOxECABiAAAAYgAAAAAQGDJRoFR10OGhvwgARQAA VBBkAAAkAX/5ClDvjApQAiAAAHgkJFQABc/sSVBXQggACAkKCwwNDg8QERITFBUWFxgZGhscHR4f ICEiIyQlJicoKSorLC0uLzAxMjM0NTY30uxJUECuAwBKAAAASgAAAAAADAesAQAQGDJRoAgARQAA PL2jQABABnbMClACIApQ74yRJgBQiCv3NQAAAACgAhbQmTwAAAIEBbQEAggKNjpKlgAAAAABAwMG 0uxJUL2yAwA+AAAAPgAAAAAQGDJRoFR10OGhvwgARQAAMPMdAAAkBp1eClDvjApQAiAAUJEm0sbM dIgr9zZwEv//zP4AAAIEBWQDAwIA0uxJUNOyAwA2AAAANgAAAAAADAesAQAQGDJRoAgARQAAKL2k QABABnbfClACIApQ74yRJgBQiCv3NtLGzHVQEABc+RYAANLsSVDxsgMApQAAAKUAAAAAAAwHrAEA EBgyUaAIAEUAAJe9pUAAQAZ2bwpQAiAKUO+MkSYAUIgr9zbSxsx1UBgAXAbWAABHRVQgLyBIVFRQ LzEuMA0KVXNlci1BZ2VudDogV2dldC8xLjEyIChsaW51eC1nbnUpDQpBY2NlcHQ6ICovKg0KSG9z dDogMTAuODAuMjM5LjE0MA0KQ29ubmVjdGlvbjogS2VlcC1BbGl2ZQ0KDQrS7ElQtbQDAEcAAABH AAAAABAYMlGgVHXQ4aG/CABFAAA5tIoAACQG2+gKUO+MClACIABQkSbSxsx1iCv3pVAY//85JQAA SFRUUC8xLjEgMjAwIE9LDQrS7ElQwLQDADYAAAA2AAAAAAAMB6wBABAYMlGgCABFAAAovaZAAEAG dt0KUAIgClDvjJEmAFCIK/el0sbMhlAQAFz4lgAA0uxJUKm2AwBoAAAAaAAAAAAQGDJRoFR10OGh vwgARQAAWmI+AAAkBi4UClDvjApQAiAAUJEm0sbMhogr96VQGP//RigAAGNvbnRlbnQtbGVuZ3Ro OiAxMTA3NA0KY29udGVudC10eXBlOiB0ZXh0L2h0bWwNCg0K0uxJUGS3AwCaBQAAmgUAAAAQGDJR oFR10OGhvwgARQAFjNf0AAAkBrMrClDvjApQAiAAUJEm0sbMuIgr96VQGP//QOgAADw/eG1sIHZl cnNpb249IjEuMCIgZW5jb2Rpbmc9IlVURi04Ij8+Cgo8aHRtbD4KCjxoZWFkPgogIDx0aXRsZT5v cGVubWlyYWdlIDo6IGhvbWU8L3RpdGxlPgogIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgdHlwZT0i dGV4dC9jc3MiIGhyZWY9Ii9zdHlsZXMvaW5kZXguY3NzIi8+CiAgPHNjcmlwdCB0eXBlPSJ0ZXh0 L2phdmFzY3JpcHQiPgogICAgdmFyIF9nYXEgPSBfZ2FxIHx8IFtdOwogICAgX2dhcS5wdXNoKFsn X3NldEFjY291bnQnLCAnVUEtMTk2MTAxNjgtMSddKTsKICAgIF9nYXEucHVzaChbJ190cmFja1Bh Z2V2aWV3J10pOwoKICAgIChmdW5jdGlvbigpIHsKICAgICAgdmFyIGdhID0gZG9jdW1lbnQuY3Jl YXRlRWxlbWVudCgnc2NyaXB0Jyk7IGdhLnR5cGUgPSAndGV4dC9qYXZhc2NyaXB0JzsgZ2EuYXN5 bmMgPSB0cnVlOwogICAgICBnYS5zcmMgPSAoJ2h0dHBzOicgPT0gZG9jdW1lbnQubG9jYXRpb24u cHJvdG9jb2wgPyAnaHR0cHM6Ly9zc2wnIDogJ2h0dHA6Ly93d3cnKSArICcuZ29vZ2xlLWFuYWx5 dGljcy5jb20vZ2EuanMnOwogICAgICB2YXIgcyA9IGRvY3VtZW50LmdldEVsZW1lbnRzQnlUYWdO YW1lKCdzY3JpcHQnKVswXTsgcy5wYXJlbnROb2RlLmluc2VydEJlZm9yZShnYSwgcyk7CiAgICB9 KSgpOwogIDwvc2NyaXB0PiAKICAKPC9oZWFkPgoKPGJvZHk+CiAgPGRpdiBpZD0id3JhcHBlciI+ CiAgICA8ZGl2IGlkPSJoZWFkZXIiPgogICAgICA8ZGl2IGlkPSJoZWFkZXJfbG9nbyI+CiAgICAg ICAgPGEgaWQ9ImxvZ28iIGhyZWY9Ii8iPjxpbWcgc3JjPSIvZ3JhcGhpY3MvbWlyYWdlLWxvZ28u cG5nIiBhbHQ9IkxvZ28iLz48L2E+CiAgICAgIDwvZGl2PgogICAgICA8ZGl2IGlkPSJpbmZvX2Jh ciI+CiAgICAgICAgPGRpdiBpZD0ibGlua2JhciI+CiAgICAgICAgICA8dWw+CiAgICAgICAgICAg ICA8dWw+CiAgICAgICAgPGxpPjxhIGNsYXNzPSJjdXJyZW50X3BhZ2UiIGhyZWY9Ii8iPkhvbWU8 L2E+PC9saT4KICAgICAgCiAgICAgICAgPGxpPjxhIGhyZWY9Ii9ibG9nLyI+QmxvZzwvYT48L2xp PgogICAgICAKICAgICAgICA8bGk+PGEgaHJlZj0iL3dpa2kvIj5XaWtpPC9hPjwvbGk+CiAgICAg IAogICAgICAgIDxsaT48YSBocmVmPSJodHRwOi8vZ2l0aHViLmNvbS9hdnNtL21pcmFnZSI+Q29k ZTwvYT48L2xpPgogICAgICAKICAgICAgICA8bGk+PGEgaHJlZj0iL2Fib3V0LyI+QWJvdXQ8L2E+ PC9saT4KICAgICAgPC91bD4gCiAgICAgICAgICA8L3VsPgogICAgICAgIDwvZGl2PgogICAgICA8 L2Rpdj4KICAgIDwvZGl2PgogIAogICAgPGRpdiBpZD0iY29udGVudCI+CiAgICAgIAogICAgPGRp diBjbGFzcz0ibGVmdF9jb2x1bW4iPgogICAgICAgCiAgPGRpdiBjbGFzcz0ibGVmdF9jb2x1bW4i PgogINLsSVBqtwMANgAAADYAAAAAAAwHrAEAEBgyUaAIAEUAACi9p0AAQAZ23ApQAiAKUO+MkSYA UIgr96XSxsyGUBAAXPiWAADS7ElQbrcDAJoFAACaBQAAABAYMlGgVHXQ4aG/CABFAAWMu+MAACQG zzwKUO+MClACIABQkSbSxtIciCv3pVAY//8D3AAAICA8ZGl2IGNsYXNzPSJzdW1tYXJ5X2luZm9y bWF0aW9uIj4gIDxwPk1pcmFnZSBpcyBhbiA8YSBocmVmPSJodHRwOi8vZW4ud2lraXBlZGlhLm9y Zy93aWtpL0V4b2tlcm5lbCI+ZXhva2VybmVsPC9hPiBmb3IgY29uc3RydWN0aW5nIHNlY3VyZSwg aGlnaC1wZXJmb3JtYW5jZSBuZXR3b3JrIGFwcGxpY2F0aW9ucyBhY3Jvc3MgYSB2YXJpZXR5IG9m IGNsb3VkIGNvbXB1dGluZyBhbmQgbW9iaWxlIHBsYXRmb3Jtcy4gIENvZGUgY2FuIGJlIGRldmVs b3BlZCBvbiBhIG5vcm1hbCBPUyBzdWNoIGFzIExpbnV4IG9yIE1hY09TIFgsIGFuZCB0aGVuIGNv bXBpbGVkIGludG8gYSBmdWxseS1zdGFuZGFsb25lLCBzcGVjaWFsaXNlZCBtaWNyb2tlcm5lbCB0 aGF0IHJ1bnMgdW5kZXIgdGhlIDxhIGhyZWY9Imh0dHA6Ly94ZW4ub3JnLyI+WGVuPC9hPiBoeXBl cnZpc29yLiAgU2luY2UgWGVuIHBvd2VycyBtb3N0IHB1YmxpYyA8YSBocmVmPSJodHRwOi8vZW4u d2lraXBlZGlhLm9yZy9DbG91ZF9jb21wdXRpbmciPmNsb3VkIGNvbXB1dGluZzwvYT4gaW5mcmFz dHJ1Y3R1cmUgc3VjaCBhcyA8YSBocmVmPSJodHRwOi8vYXdzLmFtYXpvbi5jb20iPkFtYXpvbiBF QzI8L2E+LCB0aGlzIGxldHMgeW91ciBzZXJ2ZXJzIHJ1biBtb3JlIGNoZWFwbHksIHNlY3VyZWx5 IGFuZCBmaW5lciBjb250cm9sIHRoYW4gd2l0aCBhIGZ1bGwgc29mdHdhcmUgc3RhY2suPC9wPjxw Pk1pcmFnZSBpcyBiYXNlZCBhcm91bmQgdGhlIDxhIGhyZWY9Imh0dHA6Ly9jYW1sLmlucmlhLmZy LyI+T0NhbWw8L2E+IGxhbmd1YWdlLCB3aXRoIHN5bnRheCBleHRlbnNpb25zIGFuZCBsaWJyYXJp ZXMgd2hpY2ggcHJvdmlkZSBuZXR3b3JraW5nLCBzdG9yYWdlIGFuZCBjb25jdXJyZW5jeSBzdXBw b3J0IHRoYXQgYXJlIGVhc3kgdG8gdXNlIGR1cmluZyBkZXZlbG9wbWVudCwgYW5kIG1hcCBkaXJl Y3RseSBpbnRvIG9wZXJhdGluZyBzeXN0ZW0gY29uc3RydWN0cyB3aGVuIGJlaW5nIGNvbXBpbGVk IGZvciBwcm9kdWN0aW9uIGRlcGxveW1lbnQuIFRoZSBmcmFtZXdvcmsgaXMgZnVsbHkgZXZlbnQt ZHJpdmVuLCB3aXRoIG5vIHN1cHBvcnQgZm9yIHByZWVtcHRpdmUgdGhyZWFkaW5nLjwvcD48cD5N aXJhZ2UgaXMgc3RpbGwgaW4gPGI+cHJlLWFscGhhIHN0YWdlPC9iPiwgYnV0IHRoZSBpbmZyYXN0 cnVjdHVyZSB5b3Ugc2VlIGhlcmUgaXMgc2VsZi1ob3N0aW5nLiBDaGVjayBvdXQgdGhlIDxhIGhy ZWY9Ii93aWtpIj53aWtpPC9hPiBmb3IgYW4gPGEgaHJlZj0iL3dpa2kvaW5zdGFsbCI+aW5zdGFs bGF0aW9uPC9hPiBndWlkZSwgY29tcGlsZSB5b3VyIDxhIGhyZWY9Ii93aWtpL2hlbGxvLXdvcmxk Ij5oZWxsbyB3b3JsZCBtaWNyb2tlcm5lbDwvYT4sIGdldCBzdGFydGVk0uxJUHO3AwA2AAAANgAA AAAADAesAQAQGDJRoAgARQAAKL2oQABABnbbClACIApQ74yRJgBQiCv3pdLGzIZQEABc+JYAANLs SVB3twMAbgUAAG4FAAAAEBgyUaBUddDhob8IAEUABWD1PQAAJAaWDgpQ74wKUAIgAFCRJtLG14CI K/elUBj//5JAAAAgd2l0aCB0aGUgPGEgaHJlZj0iL3dpa2kveGVuLWJvb3QiPnB1YmxpYyBjbG91 ZDwvYT4sIG9yIHdhdGNoIHRoZSA8YSBocmVmPSIvd2lraS90YWxrcyI+dGFsazwvYT4gYW5kIHJl YWQgdGhlIDxhIGhyZWY9Ii93aWtpL3BhcGVycyI+cGFwZXJzPC9hPi48L3A+ICA8L2Rpdj4KICA8 L2Rpdj4KICA8ZGl2IGNsYXNzPSJyaWdodF9jb2x1bW4iPiAgPC9kaXY+CgogICAgPC9kaXY+IAog ICAgPGRpdiBjbGFzcz0icmlnaHRfY29sdW1uIj4KICAgICAgIAogIDxkaXYgY2xhc3M9ImxlZnRf Y29sdW1uIj4KICAgIDxkaXYgY2xhc3M9InN1bW1hcnlfaW5mb3JtYXRpb24iPiA8c3R5bGU+Ci5j b2RlaGlsaXRlIC5obGwgeyBiYWNrZ3JvdW5kLWNvbG9yOiAjNDA0MDQwIH0KLmhsbCB7IHBhZGRp bmc6IDJweDsgbWFyZ2luLXRvcDogMTBweDsgfQouY29kZWhpbGl0ZSAuYyB7IGNvbG9yOiAjOTk5 OTk5OyBmb250LXN0eWxlOiBpdGFsaWMgfSAvKiBDb21tZW50ICovCi5jb2RlaGlsaXRlIC5lcnIg eyBjb2xvcjogI2E2MTcxNzsgYmFja2dyb3VuZC1jb2xvcjogI2UzZDJkMiB9IC8qIEVycm9yICov Ci5jb2RlaGlsaXRlIC5nIHsgY29sb3I6ICNkMGQwZDAgfSAvKiBHZW5lcmljICovCi5jb2RlaGls aXRlIC5rIHsgY29sb3I6ICM2YWI4MjU7IGZvbnQtd2VpZ2h0OiBib2xkIH0gLyogS2V5d29yZCAq LwouY29kZWhpbGl0ZSAubCB7IGNvbG9yOiAjZDBkMGQwIH0gLyogTGl0ZXJhbCAqLwouY29kZWhp bGl0ZSAubiB7IGNvbG9yOiAjZDBkMGQwIH0gLyogTmFtZSAqLwouY29kZWhpbGl0ZSAubyB7IGNv bG9yOiAjZDBkMGQwIH0gLyogT3BlcmF0b3IgKi8KLmNvZGVoaWxpdGUgLnggeyBjb2xvcjogI2Qw ZDBkMCB9IC8qIE90aGVyICovCi5jb2RlaGlsaXRlIC5wIHsgY29sb3I6ICNkMGQwZDAgfSAvKiBQ dW5jdHVhdGlvbiAqLwouY29kZWhpbGl0ZSAuY20geyBjb2xvcjogIzk5OTk5OTsgZm9udC1zdHls ZTogaXRhbGljIH0gLyogQ29tbWVudC5NdWx0aWxpbmUgKi8KLmNvZGVoaWxpdGUgLmNwIHsgY29s b3I6ICNjZDI4Mjg7IGZvbnQtd2VpZ2h0OiBib2xkIH0gLyogQ29tbWVudC5QcmVwcm9jICovCi5j b2RlaGlsaXRlIC5jMSB7IGNvbG9yOiAjOTk5OTk5OyBmb250LXN0eWxlOiBpdGFsaWMgfSAvKiBD b21tZW50LlNpbmdsZSAqLwouY29kZWhpbGl0ZSAuY3MgeyBjb2xvcjogI2U1MDgwODsgZm9udC13 ZWlnaHQ6IGJvbGQ7IGJhY2tncm91bmQtY29sb3I6ICM1MjAwMDAgfSAvKiBDb21tZW50LlNwZWNp YWwgKi8KLmNvZGVoaWxpdGUgLmdkIHsgY29sb3I6ICNkMjIzMjMgfSAvKiBHZW5lcmljLkRlbGV0 ZWQgKi8KLmNvZGVoaWxpdGUgLmdlIHsgY29sb3I6ICNkMGQw0uxJUHu3AwA2AAAANgAAAAAADAes AQAQGDJRoAgARQAAKL2pQABABnbaClACIApQ74yRJgBQiCv3pdLGzIZQEABc+JYAANLsSVCStwMA mgUAAJoFAAAAEBgyUaBUddDhob8IAEUABYzuvQAAJAacYgpQ74wKUAIgAFCRJtLG3LiIK/elUBj/ /3YXAABkMDsgZm9udC1zdHlsZTogaXRhbGljIH0gLyogR2VuZXJpYy5FbXBoICovCi5jb2RlaGls aXRlIC5nciB7IGNvbG9yOiAjZDIyMzIzIH0gLyogR2VuZXJpYy5FcnJvciAqLwouY29kZWhpbGl0 ZSAuZ2ggeyBjb2xvcjogI2ZmZmZmZjsgZm9udC13ZWlnaHQ6IGJvbGQgfSAvKiBHZW5lcmljLkhl YWRpbmcgKi8KLmNvZGVoaWxpdGUgLmdpIHsgY29sb3I6ICM1ODk4MTkgfSAvKiBHZW5lcmljLklu c2VydGVkICovCi5jb2RlaGlsaXRlIC5nbyB7IGNvbG9yOiAjY2NjY2NjIH0gLyogR2VuZXJpYy5P dXRwdXQgKi8KLmNvZGVoaWxpdGUgLmdwIHsgY29sb3I6ICNhYWFhYWEgfSAvKiBHZW5lcmljLlBy b21wdCAqLwouY29kZWhpbGl0ZSAuZ3MgeyBjb2xvcjogI2QwZDBkMDsgZm9udC13ZWlnaHQ6IGJv bGQgfSAvKiBHZW5lcmljLlN0cm9uZyAqLwouY29kZWhpbGl0ZSAuZ3UgeyBjb2xvcjogI2ZmZmZm ZjsgdGV4dC1kZWNvcmF0aW9uOiB1bmRlcmxpbmUgfSAvKiBHZW5lcmljLlN1YmhlYWRpbmcgKi8K LmNvZGVoaWxpdGUgLmd0IHsgY29sb3I6ICNkMjIzMjMgfSAvKiBHZW5lcmljLlRyYWNlYmFjayAq LwouY29kZWhpbGl0ZSAua2MgeyBjb2xvcjogIzZhYjgyNTsgZm9udC13ZWlnaHQ6IGJvbGQgfSAv KiBLZXl3b3JkLkNvbnN0YW50ICovCi5jb2RlaGlsaXRlIC5rZCB7IGNvbG9yOiAjNmFiODI1OyBm b250LXdlaWdodDogYm9sZCB9IC8qIEtleXdvcmQuRGVjbGFyYXRpb24gKi8KLmNvZGVoaWxpdGUg LmtuIHsgY29sb3I6ICM2YWI4MjU7IGZvbnQtd2VpZ2h0OiBib2xkIH0gLyogS2V5d29yZC5OYW1l c3BhY2UgKi8KLmNvZGVoaWxpdGUgLmtwIHsgY29sb3I6ICM2YWI4MjUgfSAvKiBLZXl3b3JkLlBz ZXVkbyAqLwouY29kZWhpbGl0ZSAua3IgeyBjb2xvcjogIzZhYjgyNTsgZm9udC13ZWlnaHQ6IGJv bGQgfSAvKiBLZXl3b3JkLlJlc2VydmVkICovCi5jb2RlaGlsaXRlIC5rdCB7IGNvbG9yOiAjNmFi ODI1OyBmb250LXdlaWdodDogYm9sZCB9IC8qIEtleXdvcmQuVHlwZSAqLwouY29kZWhpbGl0ZSAu bGQgeyBjb2xvcjogI2QwZDBkMCB9IC8qIExpdGVyYWwuRGF0ZSAqLwouY29kZWhpbGl0ZSAubSB7 IGNvbG9yOiAjMzY3N2E5IH0gLyogTGl0ZXJhbC5OdW1iZXIgKi8KLmNvZGVoaWxpdGUgLnMgeyBj b2xvcjogI2VkOWQxMyB9IC8qIExpdGVyYWwuU3RyaW5nICovCi5jb2RlaGlsaXRlIC5uYSB7IGNv bG9yOiAjYmJiYmJiIH0gLyogTmFtZS5BdHRyaWJ1dGUgKi8KLmNvZGVoaWxpdGUgLm5iIHsgY29s b3I6ICMyNDkwOWQgfSAvKiBOYW1lLkJ1aWx0aW4gKi8KLmNvZGVoaWxpdGUgLm5jIHsgY29sb3I6 ICM0NDdmY2Y7IHRleHQtZGVjb3JhdGlvbjogdW5kZXJsaW5lIH0gLyogTmFtZS5DbGFzcyAqLwou Y29kZWhpbGl0ZSAubm8geyDS7ElQmLcDADYAAAA2AAAAAAAMB6wBABAYMlGgCABFAAAovapAAEAG dtkKUAIgClDvjJEmAFCIK/el0sbMhlAQAFz4lgAA0uxJUEC4AwBoAAAAaAAAAAAQGDJRoFR10OGh vwgARQAAWiDqAAAkBm9oClDvjApQAiAAUJEm0sbMhogr96VQGP//RigAAGNvbnRlbnQtbGVuZ3Ro OiAxMTA3NA0KY29udGVudC10eXBlOiB0ZXh0L2h0bWwNCg0K1uxJUK6zAwBoAAAAaAAAAAAQGDJR oFR10OGhvwgARQAAWge2AAAkBoicClDvjApQAiAAUJEm0sbMhogr96VQGP//RigAAGNvbnRlbnQt bGVuZ3RoOiAxMTA3NA0KY29udGVudC10eXBlOiB0ZXh0L2h0bWwNCg0K2OxJUF60AwBoAAAAaAAA AAAQGDJRoFR10OGhvwgARQAAWtp5AAAkBrXYClDvjApQAiAAUJEm0sbMhogr96VQGP//RigAAGNv bnRlbnQtbGVuZ3RoOiAxMTA3NA0KY29udGVudC10eXBlOiB0ZXh0L2h0bWwNCg0K3OxJUNi0AwBo AAAAaAAAAAAQGDJRoFR10OGhvwgARQAAWiiNAAAkBmfFClDvjApQAiAAUJEm0sbMhogr96VQGP// RigAAGNvbnRlbnQtbGVuZ3RoOiAxMTA3NA0KY29udGVudC10eXBlOiB0ZXh0L2h0bWwNCg0K5OxJ UNe1AwBoAAAAaAAAAAAQGDJRoFR10OGhvwgARQAAWmcPAAAkBilDClDvjApQAiAAUJEm0sbMhogr 96VQGP//RigAAGNvbnRlbnQtbGVuZ3RoOiAxMTA3NA0KY29udGVudC10eXBlOiB0ZXh0L2h0bWwN Cg0K9OxJUD62AwBoAAAAaAAAAAAQGDJRoFR10OGhvwgARQAAWj/PAAAkBlCDClDvjApQAiAAUJEm 0sbMhogr96VQGP//RigAAGNvbnRlbnQtbGVuZ3RoOiAxMTA3NA0KY29udGVudC10eXBlOiB0ZXh0 L2h0bWwNCg0K --bcaec5014c1d627f7b04c91c39fa-- From Richard.Mortier@nottingham.ac.uk Fri Sep 07 15:21:28 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1T9zR2-0005ps-4j (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 07 Sep 2012 15:21:28 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1381532 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from thb-mta-19-tx.emailfiltering.com ([194.116.199.150]:45681 helo=thb-mta-19.emailfiltering.com) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1T9zQw-00037l-ov (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 07 Sep 2012 15:21:28 +0100 Received: from smtp4.nottingham.ac.uk ([128.243.220.65]) by thb-mta-19.emailfiltering.com with emfmta (version 4.8.5.104) by TLS id 2551778290 ;227998ee2960b66e; Fri, 07 Sep 2012 15:21:02 +0100 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp4.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1T9zMQ-0006OW-2d for cl-mirage@lists.cam.ac.uk; Fri, 07 Sep 2012 15:16:42 +0100 Received: from EXCHANGE1.ad.nottingham.ac.uk ([fe80::60c2:2879:5a5a:29a6]) by UIWEXHUB02.ad.nottingham.ac.uk ([2002:80f3:f84::80f3:f84]) with mapi; Fri, 7 Sep 2012 15:16:20 +0100 From: Richard Mortier To: David Scott Date: Fri, 7 Sep 2012 15:16:19 +0100 Subject: Re: mirage-www Thread-Topic: mirage-www Thread-Index: Ac2NA1nrmaYkKjjHSoatTMfJDjuaog== Message-ID: References: In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 14:21:28 -0000 X-Keywords: X-UID: 532 Status: O Content-Length: 1282 Lines: 34 On 7 Sep 2012, at 14:02, David Scott wrote: > ...It looks like a problem in TCP. Anyone got any hints where to look? not specifically, but it looks like the server is acking up to byte 112; th= e 3 repeated acks up to 112 trigger the fast retx as they should. isn't the server supposed to ack 1 past (because the ack itself counts as a= byte)? ie., to 113. (certainly that would explain why the client keeps t= reating the ack to 112 as a request for retx.) --=20 Cheers, R. = This message and any attachment are intended solely for the addressee a= nd may contain confidential information. If you have received this mess= age in error, please send it back to me, and immediately delete it. P= lease do not use, copy or disclose the information contained in this me= ssage or in any attachment. Any views or opinions expressed by the aut= hor of this email do not necessarily reflect the views of the Universit= y of Nottingham.=0D=0A=0D=0AThis message has been checked for viruses b= ut the contents of an attachment=0D=0Amay still contain software viruse= s which could damage your computer system:=0D=0Ayou are advised to perf= orm your own checks. Email communications with the=0D=0AUniversity of N= ottingham may be monitored as permitted by UK legislation.= From crowcroft@gmail.com Fri Sep 07 16:04:08 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TA06K-0007BQ-EX (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 07 Sep 2012 16:04:08 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1381532 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.51 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (crowcroft[at]gmail.com) * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-wg0-f51.google.com ([74.125.82.51]:46448) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1TA06I-0001ZT-QJ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 07 Sep 2012 16:04:07 +0100 Received: by wgbed3 with SMTP id ed3so1892604wgb.20 for ; Fri, 07 Sep 2012 08:04:05 -0700 (PDT) MIME-Version: 1.0 Received: by 10.180.105.6 with SMTP id gi6mr12908814wib.4.1347030245810; Fri, 07 Sep 2012 08:04:05 -0700 (PDT) Sender: crowcroft@gmail.com Received: by 10.194.15.10 with HTTP; Fri, 7 Sep 2012 08:04:05 -0700 (PDT) Received: by 10.194.15.10 with HTTP; Fri, 7 Sep 2012 08:04:05 -0700 (PDT) In-Reply-To: References: Date: Fri, 7 Sep 2012 16:04:05 +0100 X-Google-Sender-Auth: 7gNKJFShxgVutdZmrV5BOON29rU Message-ID: Subject: Re: mirage-www From: Jon Crowcroft To: Richard Mortier Content-Type: multipart/alternative; boundary=f46d04428cc84da35b04c91debe7 Cc: David Scott , Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 15:04:08 -0000 X-Keywords: X-UID: 533 Status: O Content-Length: 3405 Lines: 93 --f46d04428cc84da35b04c91debe7 Content-Type: text/plain; charset=ISO-8859-1 No, sequence numbers apply to bytes of payload data On 7 Sep 2012 15:21, "Richard Mortier" wrote: > > On 7 Sep 2012, at 14:02, David Scott wrote: > > > ...It looks like a problem in TCP. Anyone got any hints where to look? > > not specifically, but it looks like the server is acking up to byte 112; > the 3 repeated acks up to 112 trigger the fast retx as they should. > > isn't the server supposed to ack 1 past (because the ack itself counts as > a byte)? ie., to 113. (certainly that would explain why the client keeps > treating the ack to 112 as a request for retx.) > > > -- > Cheers, > > R. > > > > > This message and any attachment are intended solely for the addressee and > may contain confidential information. If you have received this message in > error, please send it back to me, and immediately delete it. Please do > not use, copy or disclose the information contained in this message or in > any attachment. Any views or opinions expressed by the author of this > email do not necessarily reflect the views of the University of Nottingham. > > This message has been checked for viruses but the contents of an attachment > may still contain software viruses which could damage your computer system: > you are advised to perform your own checks. Email communications with the > University of Nottingham may be monitored as permitted by UK legislation. > --f46d04428cc84da35b04c91debe7 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

No, sequence numbers apply to bytes of payload data

On 7 Sep 2012 15:21, "Richard Mortier"= <Richard.Mortier@no= ttingham.ac.uk> wrote:

On 7 Sep 2012, at 14:02, David Scott wrote:

> ...It looks like a problem in TCP. Anyone got any hints where to look?=

not specifically, but it looks like the server is acking up to byte 112; th= e 3 repeated acks up to 112 trigger the fast retx as they should.

isn't the server supposed to ack 1 past (because the ack itself counts = as a byte)? =A0ie., to 113. =A0(certainly that would explain why the client= keeps treating the ack to 112 as a request for retx.)


--
Cheers,

R.




This message and any attachment are intended solely for the addressee and m= ay contain confidential information. If you have received this message in e= rror, please send it back to me, and immediately delete it. =A0 Please do n= ot use, copy or disclose the information contained in this message or in an= y attachment. =A0Any views or opinions expressed by the author of this emai= l do not necessarily reflect the views of the University of Nottingham.

This message has been checked for viruses but the contents of an attachment=
may still contain software viruses which could damage your computer system:=
you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation.
--f46d04428cc84da35b04c91debe7-- From Richard.Mortier@nottingham.ac.uk Fri Sep 07 17:18:54 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TA1Gg-0001Wy-4v (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 07 Sep 2012 17:18:54 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1381532 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from ixe-mta-19-tx.emailfiltering.com ([194.116.198.150]:40300 helo=ixe-mta-19.emailfiltering.com) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1TA1Ga-0003wv-YX (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 07 Sep 2012 17:18:54 +0100 Received: from smtp4.nottingham.ac.uk ([128.243.220.65]) by ixe-mta-19.emailfiltering.com with emfmta (version 4.8.5.104) by TLS id 2363348432 ;6fcf5c450fe868de; Fri, 07 Sep 2012 17:18:02 +0100 Received: from uiwexhub01.ad.nottingham.ac.uk ([128.243.15.133]) by smtp4.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1TA1Fq-0002iK-MY; Fri, 07 Sep 2012 17:18:02 +0100 Received: from EXCHANGE1.ad.nottingham.ac.uk ([fe80::60c2:2879:5a5a:29a6]) by UIWEXHUB01.ad.nottingham.ac.uk ([2002:80f3:f85::80f3:f85]) with mapi; Fri, 7 Sep 2012 17:18:02 +0100 From: Richard Mortier To: Jon Crowcroft Date: Fri, 7 Sep 2012 17:18:01 +0100 Subject: Re: mirage-www Thread-Topic: mirage-www Thread-Index: Ac2NFFqK547wZyrKQ+SNLD1vfXiQ9Q== Message-ID: <1FF98AD3-7C87-4FFE-96D5-8C0641252600@nottingham.ac.uk> References: In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: David Scott , Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 16:18:54 -0000 X-Keywords: X-UID: 534 Status: O Content-Length: 2339 Lines: 69 On 7 Sep 2012, at 16:04, Jon Crowcroft wrote: > No, sequence numbers apply to bytes of payload data ah- it's just the SYN that applies to -- from rfc 793, p31: In line 2 of figure 7, TCP A begins by sending a SYN segment indicating that it will use sequence numbers starting with sequence number 100. In line 3, TCP B sends a SYN and acknowledges the SYN it received from TCP A. Note that the acknowledgment field indicates TCP B is now expecting to hear sequence 101, acknowledging the SYN which occupied sequence 100. in that case, no idea :) looks like the client certainly thinks things should be ack-ed further into= the stream. wireshark is having one of its (many) "moments" on my mac so i= can't actually open the file again (opening wireshark currently causes it = to fork bomb the mac with processes that activity monitor reports as "wires= hark" and ps reports as "dumpcap"...). > On 7 Sep 2012 15:21, "Richard Mortier" = wrote: >=20 > On 7 Sep 2012, at 14:02, David Scott wrote: >=20 > > ...It looks like a problem in TCP. Anyone got any hints where to look? >=20 > not specifically, but it looks like the server is acking up to byte 112; = the 3 repeated acks up to 112 trigger the fast retx as they should. >=20 > isn't the server supposed to ack 1 past (because the ack itself counts as= a byte)? ie., to 113. (certainly that would explain why the client keeps= treating the ack to 112 as a request for retx.) >=20 >=20 > -- > Cheers, >=20 > R. >=20 >=20 >=20 >=20 > This message and any attachment are intended solely for the addressee and= may contain confidential information. If you have received this message in= error, please send it back to me, and immediately delete it. Please do n= ot use, copy or disclose the information contained in this message or in an= y attachment. Any views or opinions expressed by the author of this email = do not necessarily reflect the views of the University of Nottingham. >=20 > This message has been checked for viruses but the contents of an attachme= nt > may still contain software viruses which could damage your computer syste= m: > you are advised to perform your own checks. Email communications with the > University of Nottingham may be monitored as permitted by UK legislation. --=20 Cheers, R. From avsm@dark.recoil.org Fri Sep 07 17:59:08 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TA1tc-0002Pw-Kl (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 07 Sep 2012 17:59:08 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1381532 * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:37359 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1TA1tb-0006vg-SJ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 07 Sep 2012 17:59:08 +0100 Received: (qmail 28771 invoked by uid 10000); 7 Sep 2012 16:59:07 -0000 Date: Fri, 7 Sep 2012 17:59:06 +0100 From: Anil Madhavapeddy To: David Scott , Balraj Singh Subject: Re: mirage-www Message-ID: <20120907165905.GA25178@dark.recoil.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 16:59:08 -0000 X-Keywords: X-UID: 535 Status: O Content-Length: 4811 Lines: 117 On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott wrote: > Hi, > > I've built mirage-www for xen and modified the static IP to fit my > environment. (I tried DHCP first but this didn't work -- if I get the > time I'll try to debug). > > I can ping the server fine, and it's certainly receiving a lot of > traffic on my (probably fairly busy) local network. When I try to > fetch a URL the TCP connection hangs. On the console I get: > > Dispatch: dynamic URL / > ... irrelevant spam > TCP retransmission on timer seq = -889321980 > > I've attached a small tcpdump of the conversation. I started with ping > and then tried HTTP. According to tcpdump/wireshark it went like this: > > $ tcpdump -r mirage.pcap -n > reading from file mirage.pcap, link-type EN10MB (Ethernet) > 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id > 9300, seq 1, length 64 > 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id > 9300, seq 1, length 64 > 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id > 9300, seq 2, length 64 > 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id > 9300, seq 2, length 64 > 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id > 9300, seq 3, length 64 > 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id > 9300, seq 3, length 64 > 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id > 9300, seq 4, length 64 > 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id > 9300, seq 4, length 64 > 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id > 9300, seq 5, length 64 > 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id > 9300, seq 5, length 64 > > -- so far so good, this is just my initial pings. Switching to 'wget > http://10.80.239.140/' > > 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [S], seq > 2284582709, win 5840, options [mss 1460,sackOK,TS val 909789846 ecr > 0,nop,wscale 6], length 0 > 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [S.], > seq 3536243828, ack 2284582710, win 65535, options [mss 1380,wscale > 2,eol], length 0 > 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack > 1, win 92, length 0 > > -- TCP connection established > > 13:47:14.242417 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [P.], > seq 1:112, ack 1, win 92, length 111 > > -- HTTP GET / sent > > 13:47:14.242869 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], > seq 1:18, ack 112, win 65535, length 17 > > -- HTTP/1.1 200 OK replied > > 13:47:14.242880 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack > 18, win 92, length 0 > 13:47:14.243369 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], > seq 18:68, ack 112, win 65535, length 50 > > -- "content-length..." replied > > 13:47:14.243556 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], > seq 68:1448, ack 112, win 65535, length 1380 > > ... after this chunks of the blog post are transmitted > > 13:47:14.243562 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack > 18, win 92, length 0 > 13:47:14.243566 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], > seq 1448:2828, ack 112, win 65535, length 1380 > 13:47:14.243571 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack > 18, win 92, length 0 > 13:47:14.243575 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], > seq 2828:4164, ack 112, win 65535, length 1336 > 13:47:14.243579 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack > 18, win 92, length 0 > 13:47:14.243602 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], > seq 4164:5544, ack 112, win 65535, length 1380 > 13:47:14.243608 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack > 18, win 92, length 0 > 13:47:14.243776 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], > seq 18:68, ack 112, win 65535, length 50 > > -- fast retransmit of the "content-length" packet > > 13:47:18.242606 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], > seq 18:68, ack 112, win 65535, length 50 > So the server is receiving the HTTP body ACKs (i.e. after the content-length) on the wire, but not passing them up to the TCP stack for some reason. The stack believes they weren't ACKed, and is correctly retransmitting (from its perspective). Balraj saw the same problem a couple of months ago, and tracked it down to possible corruption of the body of packets. Balraj, did you ever narrow this down further? Iirc, the first ~50 bytes of some packets were overwritten near the very start. Dave, could you look at the packet bodies and see if they all look correct? Unfortunately, I can't reproduce this on my setup (mirage-www works fine). What is the TCP client / browser you are using? -- Anil Madhavapeddy http://anil.recoil.org From avsm2@cl.cam.ac.uk Fri Sep 07 18:49:13 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TA2g5-0002lT-4Q (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 07 Sep 2012 18:49:13 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1381532 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:9611 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1TA2g4-0003xw-Qg (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 07 Sep 2012 18:49:13 +0100 Received: (qmail 17802 invoked by uid 634); 7 Sep 2012 17:49:12 -0000 Received: from visnet-57.csl.sri.com (HELO visnet-57.csl.sri.com) (130.107.98.57) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 07 Sep 2012 18:49:12 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: installing xapi on debian Message-Id: <303A60B8-60F3-4540-9A60-898DCA1A1971@cl.cam.ac.uk> Date: Fri, 7 Sep 2012 10:49:05 -0700 To: Mirage List , Jon Ludlam Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 17:49:13 -0000 X-Keywords: X-UID: 536 Status: O Content-Length: 617 Lines: 15 Dave, Jon, Haris is developing a simulation backend to Mirage, based on bindings to = NS3. Each simulation node runs as a separate process, with events = transmitted over TCP connections (and thus, we can support a few = thousand cores in theory). As part of this, we'd like to spin up some real Mirage/Xen uVMs for = traffic generation. I suggested that the XAPI API is easiest for this, = but we now need to get it installed on a Debian box. What's the best = way of doing this at present? We don't need any storage at all at this = stage; just starting/stopping VMs and configuring network bridges. -anil= From jac22@cl.cam.ac.uk Sat Sep 08 05:59:47 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TAD91-0000Bk-4t (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 08 Sep 2012 05:59:47 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.1 from SpamAssassin-3.3.2-1381936 * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP * 0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mta0.cl.cam.ac.uk ([128.232.25.20]:55389) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1TAD90-0000iW-on (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 08 Sep 2012 05:59:47 +0100 Received: from ramsey.cl.cam.ac.uk ([128.232.1.219] helo=cl.cam.ac.uk) by mta0.cl.cam.ac.uk with esmtp (Exim 4.63) (envelope-from ) id 1TAD90-0006O9-0k; Sat, 08 Sep 2012 05:59:46 +0100 To: Richard Mortier Subject: Re: mirage-www In-reply-to: <1FF98AD3-7C87-4FFE-96D5-8C0641252600@nottingham.ac.uk> References: <1FF98AD3-7C87-4FFE-96D5-8C0641252600@nottingham.ac.uk> Comments: In-reply-to Richard Mortier message dated "Fri, 07 Sep 2012 17:18:01 +0100." MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <3079.1347080385.1@cl.cam.ac.uk> Date: Sat, 08 Sep 2012 05:59:45 +0100 From: Jon Crowcroft Message-Id: Cc: Jon.Crowcroft@cl.cam.ac.uk, Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 04:59:47 -0000 X-Keywords: X-UID: 537 Status: O Content-Length: 2753 Lines: 81 i wonder if its some thread deadlock/wedge that';'s not letting a TCP piece make progress...? In missive <1FF98AD3-7C87-4FFE-96D5-8C0641252600@nottingham.ac.uk>, Richard Mortier typed: >> >>On 7 Sep 2012, at 16:04, Jon Crowcroft wrote: >> >>> No, sequence numbers apply to bytes of payload data >> >>ah- it's just the SYN that applies to -- from rfc 793, p31: >> >> In line 2 of figure 7, TCP A begins by sending a SYN segment >> indicating that it will use sequence numbers starting with sequence >> number 100. In line 3, TCP B sends a SYN and acknowledges the SYN it >> received from TCP A. Note that the acknowledgment field indicates TCP >> B is now expecting to hear sequence 101, acknowledging the SYN which >> occupied sequence 100. >> >>in that case, no idea :) >> >>looks like the client certainly thinks things should be ack-ed further into= >> the stream. wireshark is having one of its (many) "moments" on my mac so i= >> can't actually open the file again (opening wireshark currently causes it = >>to fork bomb the mac with processes that activity monitor reports as "wires= >>hark" and ps reports as "dumpcap"...). >> >>> On 7 Sep 2012 15:21, "Richard Mortier" = >> wrote: >>>=20 >>> On 7 Sep 2012, at 14:02, David Scott wrote: >>>=20 >>> > ...It looks like a problem in TCP. Anyone got any hints where to look? >>>=20 >>> not specifically, but it looks like the server is acking up to byte 112; = >>the 3 repeated acks up to 112 trigger the fast retx as they should. >>>=20 >>> isn't the server supposed to ack 1 past (because the ack itself counts as= >> a byte)? ie., to 113. (certainly that would explain why the client keeps= >> treating the ack to 112 as a request for retx.) >>>=20 >>>=20 >>> -- >>> Cheers, >>>=20 >>> R. >>>=20 >>>=20 >>>=20 >>>=20 >>> This message and any attachment are intended solely for the addressee and= >> may contain confidential information. If you have received this message in= >> error, please send it back to me, and immediately delete it. Please do n= >>ot use, copy or disclose the information contained in this message or in an= >>y attachment. Any views or opinions expressed by the author of this email = >>do not necessarily reflect the views of the University of Nottingham. >>>=20 >>> This message has been checked for viruses but the contents of an attachme= >>nt >>> may still contain software viruses which could damage your computer syste= >>m: >>> you are advised to perform your own checks. Email communications with the >>> University of Nottingham may be monitored as permitted by UK legislation. >> >> >>--=20 >>Cheers, >> >>R. >> >> >> >> >> cheers jon From Dave.Scott@eu.citrix.com Sat Sep 08 09:06:17 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TAG3V-0002Fl-Rb (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 08 Sep 2012 09:06:17 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.4 from SpamAssassin-3.3.2-1381936 * -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, * medium trust * [62.200.22.115 listed in list.dnswl.dnsbl.ja.net] * 0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from smtp.eu.citrix.com ([62.200.22.115]:38095) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1TAG3V-0007ym-mX (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 08 Sep 2012 09:06:17 +0100 X-IronPort-AV: E=Sophos;i="4.80,391,1344211200"; d="scan'208";a="14421969" Received: from lonpmailmx01.citrite.net ([10.30.203.162]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/RC4-MD5; 08 Sep 2012 08:06:16 +0000 Received: from LONPMAILBOX01.citrite.net ([10.30.224.160]) by LONPMAILMX01.citrite.net ([10.30.203.162]) with mapi; Sat, 8 Sep 2012 09:06:16 +0100 From: Dave Scott To: Anil Madhavapeddy Date: Sat, 8 Sep 2012 09:06:14 +0100 Subject: Re: mirage-www Thread-Topic: mirage-www Thread-Index: Ac2NmNG+0+Kfu+zgRg2CQykFuh8WUw== Message-ID: References: <20120907165905.GA25178@dark.recoil.org> In-Reply-To: <20120907165905.GA25178@dark.recoil.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Balraj Singh , Mirage List , David Scott X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 08:06:17 -0000 X-Keywords: X-UID: 538 Status: O Content-Length: 5482 Lines: 134 On Sep 7, 2012, at 5:59 PM, "Anil Madhavapeddy" wrote: > On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott wrote: >> Hi, >>=20 >> I've built mirage-www for xen and modified the static IP to fit my >> environment. (I tried DHCP first but this didn't work -- if I get the >> time I'll try to debug). >>=20 >> I can ping the server fine, and it's certainly receiving a lot of >> traffic on my (probably fairly busy) local network. When I try to >> fetch a URL the TCP connection hangs. On the console I get: >>=20 >> Dispatch: dynamic URL / >> ... irrelevant spam >> TCP retransmission on timer seq =3D -889321980 >>=20 >> I've attached a small tcpdump of the conversation. I started with ping >> and then tried HTTP. According to tcpdump/wireshark it went like this: >>=20 >> $ tcpdump -r mirage.pcap -n >> reading from file mirage.pcap, link-type EN10MB (Ethernet) >> 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >> 9300, seq 1, length 64 >> 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >> 9300, seq 1, length 64 >> 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >> 9300, seq 2, length 64 >> 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >> 9300, seq 2, length 64 >> 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >> 9300, seq 3, length 64 >> 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >> 9300, seq 3, length 64 >> 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >> 9300, seq 4, length 64 >> 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >> 9300, seq 4, length 64 >> 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >> 9300, seq 5, length 64 >> 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >> 9300, seq 5, length 64 >>=20 >> -- so far so good, this is just my initial pings. Switching to 'wget >> http://10.80.239.140/' >>=20 >> 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [S], seq >> 2284582709, win 5840, options [mss 1460,sackOK,TS val 909789846 ecr >> 0,nop,wscale 6], length 0 >> 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [S.], >> seq 3536243828, ack 2284582710, win 65535, options [mss 1380,wscale >> 2,eol], length 0 >> 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack >> 1, win 92, length 0 >>=20 >> -- TCP connection established >>=20 >> 13:47:14.242417 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [P.], >> seq 1:112, ack 1, win 92, length 111 >>=20 >> -- HTTP GET / sent >>=20 >> 13:47:14.242869 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >> seq 1:18, ack 112, win 65535, length 17 >>=20 >> -- HTTP/1.1 200 OK replied >>=20 >> 13:47:14.242880 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack >> 18, win 92, length 0 >> 13:47:14.243369 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >> seq 18:68, ack 112, win 65535, length 50 >>=20 >> -- "content-length..." replied >>=20 >> 13:47:14.243556 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >> seq 68:1448, ack 112, win 65535, length 1380 >>=20 >> ... after this chunks of the blog post are transmitted >>=20 >> 13:47:14.243562 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack >> 18, win 92, length 0 >> 13:47:14.243566 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >> seq 1448:2828, ack 112, win 65535, length 1380 >> 13:47:14.243571 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack >> 18, win 92, length 0 >> 13:47:14.243575 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >> seq 2828:4164, ack 112, win 65535, length 1336 >> 13:47:14.243579 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack >> 18, win 92, length 0 >> 13:47:14.243602 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >> seq 4164:5544, ack 112, win 65535, length 1380 >> 13:47:14.243608 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack >> 18, win 92, length 0 >> 13:47:14.243776 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >> seq 18:68, ack 112, win 65535, length 50 >>=20 >> -- fast retransmit of the "content-length" packet >>=20 >> 13:47:18.242606 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >> seq 18:68, ack 112, win 65535, length 50 >>=20 >=20 > So the server is receiving the HTTP body ACKs (i.e. after the > content-length) on the wire, but not passing them up to the TCP stack=20 > for some reason. The stack believes they weren't ACKed, and is correctly > retransmitting (from its perspective). >=20 > Balraj saw the same problem a couple of months ago, and tracked it down t= o > possible corruption of the body of packets. Balraj, did you ever narrow= =20 > this down further? >=20 > Iirc, the first ~50 bytes of some packets were overwritten near the very > start. Dave, could you look at the packet bodies and see if they all look > correct? Unfortunately, I can't reproduce this on my setup (mirage-www > works fine). What is the TCP client / browser you are using? I tried Linux chrome and when that failed, switched to wget. I'll investigate the packet corruption possibility first. I'll take the sce= nic route: I noticed one of the cstruct examples is for pcap file parsing, = I'll try to add the ability to log the packets directly to a local mirage b= lock device. I wanted a similar thing for xenstore anyway. If that fails I'll resort to printfs :) the problem always happens after a = handful of packets so this should be easy. Cheers, Dave From scott.dj@gmail.com Mon Sep 10 22:31:38 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBBZy-00009k-Jl (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 10 Sep 2012 22:31:38 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.9 from SpamAssassin-3.3.2-1382427 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.220.179 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-vc0-f179.google.com ([209.85.220.179]:33719) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1TBBZx-0000kP-Qo (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 10 Sep 2012 22:31:38 +0100 Received: by vcqp16 with SMTP id p16so2050087vcq.38 for ; Mon, 10 Sep 2012 14:31:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.175.130 with SMTP id ca2mr17643786vdc.112.1347312696450; Mon, 10 Sep 2012 14:31:36 -0700 (PDT) Received: by 10.58.213.168 with HTTP; Mon, 10 Sep 2012 14:31:36 -0700 (PDT) In-Reply-To: References: <20120907165905.GA25178@dark.recoil.org> Date: Mon, 10 Sep 2012 22:31:36 +0100 Message-ID: Subject: Re: mirage-www From: David Scott To: Dave Scott Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Balraj Singh , Mirage List , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2012 21:31:38 -0000 X-Keywords: X-UID: 539 Status: O Content-Length: 6093 Lines: 153 Right, problem solved: TCP checksum on xen was slightly broken. There was a fix made to Unix to cope with odd-length packets, this needed to be applied to xen as well. Many of the packets had valid checksums, and linux would ACK up to the sequence number in the last one of those. Now that the TCP bug is squashed, I can get back to writing my blog post -- all just a day in the life of a mirage hacker :-) On Sat, Sep 8, 2012 at 9:06 AM, Dave Scott wrote= : > > > On Sep 7, 2012, at 5:59 PM, "Anil Madhavapeddy" wrote: > >> On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott wrote: >>> Hi, >>> >>> I've built mirage-www for xen and modified the static IP to fit my >>> environment. (I tried DHCP first but this didn't work -- if I get the >>> time I'll try to debug). >>> >>> I can ping the server fine, and it's certainly receiving a lot of >>> traffic on my (probably fairly busy) local network. When I try to >>> fetch a URL the TCP connection hangs. On the console I get: >>> >>> Dispatch: dynamic URL / >>> ... irrelevant spam >>> TCP retransmission on timer seq =3D -889321980 >>> >>> I've attached a small tcpdump of the conversation. I started with ping >>> and then tried HTTP. According to tcpdump/wireshark it went like this: >>> >>> $ tcpdump -r mirage.pcap -n >>> reading from file mirage.pcap, link-type EN10MB (Ethernet) >>> 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >>> 9300, seq 1, length 64 >>> 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>> 9300, seq 1, length 64 >>> 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >>> 9300, seq 2, length 64 >>> 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>> 9300, seq 2, length 64 >>> 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >>> 9300, seq 3, length 64 >>> 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>> 9300, seq 3, length 64 >>> 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >>> 9300, seq 4, length 64 >>> 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>> 9300, seq 4, length 64 >>> 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >>> 9300, seq 5, length 64 >>> 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>> 9300, seq 5, length 64 >>> >>> -- so far so good, this is just my initial pings. Switching to 'wget >>> http://10.80.239.140/' >>> >>> 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [S], seq >>> 2284582709, win 5840, options [mss 1460,sackOK,TS val 909789846 ecr >>> 0,nop,wscale 6], length 0 >>> 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [S.], >>> seq 3536243828, ack 2284582710, win 65535, options [mss 1380,wscale >>> 2,eol], length 0 >>> 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack >>> 1, win 92, length 0 >>> >>> -- TCP connection established >>> >>> 13:47:14.242417 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [P.], >>> seq 1:112, ack 1, win 92, length 111 >>> >>> -- HTTP GET / sent >>> >>> 13:47:14.242869 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>> seq 1:18, ack 112, win 65535, length 17 >>> >>> -- HTTP/1.1 200 OK replied >>> >>> 13:47:14.242880 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack >>> 18, win 92, length 0 >>> 13:47:14.243369 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>> seq 18:68, ack 112, win 65535, length 50 >>> >>> -- "content-length..." replied --- and this one had the invalid checksum. >>> >>> 13:47:14.243556 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>> seq 68:1448, ack 112, win 65535, length 1380 >>> >>> ... after this chunks of the blog post are transmitted >>> >>> 13:47:14.243562 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack >>> 18, win 92, length 0 >>> 13:47:14.243566 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>> seq 1448:2828, ack 112, win 65535, length 1380 >>> 13:47:14.243571 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack >>> 18, win 92, length 0 >>> 13:47:14.243575 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>> seq 2828:4164, ack 112, win 65535, length 1336 >>> 13:47:14.243579 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack >>> 18, win 92, length 0 >>> 13:47:14.243602 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>> seq 4164:5544, ack 112, win 65535, length 1380 >>> 13:47:14.243608 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack >>> 18, win 92, length 0 >>> 13:47:14.243776 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>> seq 18:68, ack 112, win 65535, length 50 >>> >>> -- fast retransmit of the "content-length" packet >>> >>> 13:47:18.242606 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>> seq 18:68, ack 112, win 65535, length 50 >>> >> >> So the server is receiving the HTTP body ACKs (i.e. after the >> content-length) on the wire, but not passing them up to the TCP stack >> for some reason. The stack believes they weren't ACKed, and is correctly >> retransmitting (from its perspective). >> >> Balraj saw the same problem a couple of months ago, and tracked it down = to >> possible corruption of the body of packets. Balraj, did you ever narrow >> this down further? >> >> Iirc, the first ~50 bytes of some packets were overwritten near the very >> start. Dave, could you look at the packet bodies and see if they all loo= k >> correct? Unfortunately, I can't reproduce this on my setup (mirage-www >> works fine). What is the TCP client / browser you are using? > > I tried Linux chrome and when that failed, switched to wget. > > I'll investigate the packet corruption possibility first. I'll take the s= cenic route: I noticed one of the cstruct examples is for pcap file parsing= , I'll try to add the ability to log the packets directly to a local mirage= block device. I wanted a similar thing for xenstore anyway. > > If that fails I'll resort to printfs :) the problem always happens after = a handful of packets so this should be easy. > > Cheers, > Dave > --=20 Dave Scott From anil@recoil.org Tue Sep 11 05:32:21 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBI97-0003fc-TP (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 05:32:21 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1382695 * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:43926 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1TBI97-0004ON-WU (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 05:32:21 +0100 Received: (qmail 20873 invoked by uid 634); 11 Sep 2012 04:32:20 -0000 Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.2.2]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 11 Sep 2012 05:32:20 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: mirage-www From: Anil Madhavapeddy In-Reply-To: Date: Tue, 11 Sep 2012 05:32:19 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <5DF2CDEF-60A5-493D-903F-027A74A645B8@recoil.org> References: <20120907165905.GA25178@dark.recoil.org> To: David Scott X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Balraj Singh , Dave Scott , Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 04:32:22 -0000 X-Keywords: X-UID: 540 Status: O Content-Length: 6632 Lines: 184 That was my fault, sorry! This sort of thing should be fixed soon when we can share C bindings more easily among the cross-compilation targets. -anil On 10 Sep 2012, at 22:31, David Scott wrote: > Right, problem solved: TCP checksum on xen was slightly broken. There > was a fix made to Unix to cope with odd-length packets, this needed to > be applied to xen as well. Many of the packets had valid checksums, > and linux would ACK up to the sequence number in the last one of > those. >=20 > Now that the TCP bug is squashed, I can get back to writing my blog > post -- all just a day in the life of a mirage hacker :-) >=20 > On Sat, Sep 8, 2012 at 9:06 AM, Dave Scott = wrote: >>=20 >>=20 >> On Sep 7, 2012, at 5:59 PM, "Anil Madhavapeddy" = wrote: >>=20 >>> On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott wrote: >>>> Hi, >>>>=20 >>>> I've built mirage-www for xen and modified the static IP to fit my >>>> environment. (I tried DHCP first but this didn't work -- if I get = the >>>> time I'll try to debug). >>>>=20 >>>> I can ping the server fine, and it's certainly receiving a lot of >>>> traffic on my (probably fairly busy) local network. When I try to >>>> fetch a URL the TCP connection hangs. On the console I get: >>>>=20 >>>> Dispatch: dynamic URL / >>>> ... irrelevant spam >>>> TCP retransmission on timer seq =3D -889321980 >>>>=20 >>>> I've attached a small tcpdump of the conversation. I started with = ping >>>> and then tried HTTP. According to tcpdump/wireshark it went like = this: >>>>=20 >>>> $ tcpdump -r mirage.pcap -n >>>> reading from file mirage.pcap, link-type EN10MB (Ethernet) >>>> 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, = id >>>> 9300, seq 1, length 64 >>>> 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>>> 9300, seq 1, length 64 >>>> 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, = id >>>> 9300, seq 2, length 64 >>>> 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>>> 9300, seq 2, length 64 >>>> 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, = id >>>> 9300, seq 3, length 64 >>>> 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>>> 9300, seq 3, length 64 >>>> 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, = id >>>> 9300, seq 4, length 64 >>>> 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>>> 9300, seq 4, length 64 >>>> 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, = id >>>> 9300, seq 5, length 64 >>>> 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>>> 9300, seq 5, length 64 >>>>=20 >>>> -- so far so good, this is just my initial pings. Switching to = 'wget >>>> http://10.80.239.140/' >>>>=20 >>>> 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [S], = seq >>>> 2284582709, win 5840, options [mss 1460,sackOK,TS val 909789846 ecr >>>> 0,nop,wscale 6], length 0 >>>> 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [S.], >>>> seq 3536243828, ack 2284582710, win 65535, options [mss 1380,wscale >>>> 2,eol], length 0 >>>> 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], = ack >>>> 1, win 92, length 0 >>>>=20 >>>> -- TCP connection established >>>>=20 >>>> 13:47:14.242417 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [P.], >>>> seq 1:112, ack 1, win 92, length 111 >>>>=20 >>>> -- HTTP GET / sent >>>>=20 >>>> 13:47:14.242869 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>>> seq 1:18, ack 112, win 65535, length 17 >>>>=20 >>>> -- HTTP/1.1 200 OK replied >>>>=20 >>>> 13:47:14.242880 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], = ack >>>> 18, win 92, length 0 >>>> 13:47:14.243369 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>>> seq 18:68, ack 112, win 65535, length 50 >>>>=20 >>>> -- "content-length..." replied >=20 > --- and this one had the invalid checksum. >=20 >>>>=20 >>>> 13:47:14.243556 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>>> seq 68:1448, ack 112, win 65535, length 1380 >>>>=20 >>>> ... after this chunks of the blog post are transmitted >>>>=20 >>>> 13:47:14.243562 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], = ack >>>> 18, win 92, length 0 >>>> 13:47:14.243566 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>>> seq 1448:2828, ack 112, win 65535, length 1380 >>>> 13:47:14.243571 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], = ack >>>> 18, win 92, length 0 >>>> 13:47:14.243575 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>>> seq 2828:4164, ack 112, win 65535, length 1336 >>>> 13:47:14.243579 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], = ack >>>> 18, win 92, length 0 >>>> 13:47:14.243602 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>>> seq 4164:5544, ack 112, win 65535, length 1380 >>>> 13:47:14.243608 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], = ack >>>> 18, win 92, length 0 >>>> 13:47:14.243776 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>>> seq 18:68, ack 112, win 65535, length 50 >>>>=20 >>>> -- fast retransmit of the "content-length" packet >>>>=20 >>>> 13:47:18.242606 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [P.], >>>> seq 18:68, ack 112, win 65535, length 50 >>>>=20 >>>=20 >>> So the server is receiving the HTTP body ACKs (i.e. after the >>> content-length) on the wire, but not passing them up to the TCP = stack >>> for some reason. The stack believes they weren't ACKed, and is = correctly >>> retransmitting (from its perspective). >>>=20 >>> Balraj saw the same problem a couple of months ago, and tracked it = down to >>> possible corruption of the body of packets. Balraj, did you ever = narrow >>> this down further? >>>=20 >>> Iirc, the first ~50 bytes of some packets were overwritten near the = very >>> start. Dave, could you look at the packet bodies and see if they all = look >>> correct? Unfortunately, I can't reproduce this on my setup = (mirage-www >>> works fine). What is the TCP client / browser you are using? >>=20 >> I tried Linux chrome and when that failed, switched to wget. >>=20 >> I'll investigate the packet corruption possibility first. I'll take = the scenic route: I noticed one of the cstruct examples is for pcap file = parsing, I'll try to add the ability to log the packets directly to a = local mirage block device. I wanted a similar thing for xenstore anyway. >>=20 >> If that fails I'll resort to printfs :) the problem always happens = after a handful of packets so this should be easy. >>=20 >> Cheers, >> Dave >>=20 >=20 >=20 >=20 > --=20 > Dave Scott >=20 From anil@recoil.org Tue Sep 11 07:27:54 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBJww-0004zx-Sv (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 07:27:54 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1382695 * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:25520 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1TBJww-00057k-WM (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 07:27:54 +0100 Received: (qmail 1232 invoked by uid 634); 11 Sep 2012 06:27:51 -0000 Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.2.2]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 11 Sep 2012 07:27:51 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: mirage-www From: Anil Madhavapeddy In-Reply-To: <5DF2CDEF-60A5-493D-903F-027A74A645B8@recoil.org> Date: Tue, 11 Sep 2012 07:27:50 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20120907165905.GA25178@dark.recoil.org> <5DF2CDEF-60A5-493D-903F-027A74A645B8@recoil.org> To: David Scott X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Balraj Singh , Dave Scott , Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 06:27:55 -0000 X-Keywords: X-UID: 541 Status: O Content-Length: 7195 Lines: 210 I've also just ported http://github.com/mirage/mirage-tutorial to all the new interfaces (you will need the very latest cow-0.3.2 from OPAM). The content is still the out-of-date stuff with respect to Mirage, but I'm forking it to add the Async/Lwt tutorial content for Thursday. -anil On 11 Sep 2012, at 05:32, Anil Madhavapeddy wrote: > That was my fault, sorry! This sort of thing should be fixed soon when > we can share C bindings more easily among the cross-compilation = targets. >=20 > -anil >=20 > On 10 Sep 2012, at 22:31, David Scott wrote: >=20 >> Right, problem solved: TCP checksum on xen was slightly broken. There >> was a fix made to Unix to cope with odd-length packets, this needed = to >> be applied to xen as well. Many of the packets had valid checksums, >> and linux would ACK up to the sequence number in the last one of >> those. >>=20 >> Now that the TCP bug is squashed, I can get back to writing my blog >> post -- all just a day in the life of a mirage hacker :-) >>=20 >> On Sat, Sep 8, 2012 at 9:06 AM, Dave Scott = wrote: >>>=20 >>>=20 >>> On Sep 7, 2012, at 5:59 PM, "Anil Madhavapeddy" = wrote: >>>=20 >>>> On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott wrote: >>>>> Hi, >>>>>=20 >>>>> I've built mirage-www for xen and modified the static IP to fit my >>>>> environment. (I tried DHCP first but this didn't work -- if I get = the >>>>> time I'll try to debug). >>>>>=20 >>>>> I can ping the server fine, and it's certainly receiving a lot of >>>>> traffic on my (probably fairly busy) local network. When I try to >>>>> fetch a URL the TCP connection hangs. On the console I get: >>>>>=20 >>>>> Dispatch: dynamic URL / >>>>> ... irrelevant spam >>>>> TCP retransmission on timer seq =3D -889321980 >>>>>=20 >>>>> I've attached a small tcpdump of the conversation. I started with = ping >>>>> and then tried HTTP. According to tcpdump/wireshark it went like = this: >>>>>=20 >>>>> $ tcpdump -r mirage.pcap -n >>>>> reading from file mirage.pcap, link-type EN10MB (Ethernet) >>>>> 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, = id >>>>> 9300, seq 1, length 64 >>>>> 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>>>> 9300, seq 1, length 64 >>>>> 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, = id >>>>> 9300, seq 2, length 64 >>>>> 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>>>> 9300, seq 2, length 64 >>>>> 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, = id >>>>> 9300, seq 3, length 64 >>>>> 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>>>> 9300, seq 3, length 64 >>>>> 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, = id >>>>> 9300, seq 4, length 64 >>>>> 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>>>> 9300, seq 4, length 64 >>>>> 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, = id >>>>> 9300, seq 5, length 64 >>>>> 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>>>> 9300, seq 5, length 64 >>>>>=20 >>>>> -- so far so good, this is just my initial pings. Switching to = 'wget >>>>> http://10.80.239.140/' >>>>>=20 >>>>> 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [S], = seq >>>>> 2284582709, win 5840, options [mss 1460,sackOK,TS val 909789846 = ecr >>>>> 0,nop,wscale 6], length 0 >>>>> 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags = [S.], >>>>> seq 3536243828, ack 2284582710, win 65535, options [mss = 1380,wscale >>>>> 2,eol], length 0 >>>>> 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], = ack >>>>> 1, win 92, length 0 >>>>>=20 >>>>> -- TCP connection established >>>>>=20 >>>>> 13:47:14.242417 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags = [P.], >>>>> seq 1:112, ack 1, win 92, length 111 >>>>>=20 >>>>> -- HTTP GET / sent >>>>>=20 >>>>> 13:47:14.242869 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags = [P.], >>>>> seq 1:18, ack 112, win 65535, length 17 >>>>>=20 >>>>> -- HTTP/1.1 200 OK replied >>>>>=20 >>>>> 13:47:14.242880 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], = ack >>>>> 18, win 92, length 0 >>>>> 13:47:14.243369 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags = [P.], >>>>> seq 18:68, ack 112, win 65535, length 50 >>>>>=20 >>>>> -- "content-length..." replied >>=20 >> --- and this one had the invalid checksum. >>=20 >>>>>=20 >>>>> 13:47:14.243556 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags = [P.], >>>>> seq 68:1448, ack 112, win 65535, length 1380 >>>>>=20 >>>>> ... after this chunks of the blog post are transmitted >>>>>=20 >>>>> 13:47:14.243562 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], = ack >>>>> 18, win 92, length 0 >>>>> 13:47:14.243566 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags = [P.], >>>>> seq 1448:2828, ack 112, win 65535, length 1380 >>>>> 13:47:14.243571 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], = ack >>>>> 18, win 92, length 0 >>>>> 13:47:14.243575 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags = [P.], >>>>> seq 2828:4164, ack 112, win 65535, length 1336 >>>>> 13:47:14.243579 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], = ack >>>>> 18, win 92, length 0 >>>>> 13:47:14.243602 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags = [P.], >>>>> seq 4164:5544, ack 112, win 65535, length 1380 >>>>> 13:47:14.243608 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], = ack >>>>> 18, win 92, length 0 >>>>> 13:47:14.243776 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags = [P.], >>>>> seq 18:68, ack 112, win 65535, length 50 >>>>>=20 >>>>> -- fast retransmit of the "content-length" packet >>>>>=20 >>>>> 13:47:18.242606 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags = [P.], >>>>> seq 18:68, ack 112, win 65535, length 50 >>>>>=20 >>>>=20 >>>> So the server is receiving the HTTP body ACKs (i.e. after the >>>> content-length) on the wire, but not passing them up to the TCP = stack >>>> for some reason. The stack believes they weren't ACKed, and is = correctly >>>> retransmitting (from its perspective). >>>>=20 >>>> Balraj saw the same problem a couple of months ago, and tracked it = down to >>>> possible corruption of the body of packets. Balraj, did you ever = narrow >>>> this down further? >>>>=20 >>>> Iirc, the first ~50 bytes of some packets were overwritten near the = very >>>> start. Dave, could you look at the packet bodies and see if they = all look >>>> correct? Unfortunately, I can't reproduce this on my setup = (mirage-www >>>> works fine). What is the TCP client / browser you are using? >>>=20 >>> I tried Linux chrome and when that failed, switched to wget. >>>=20 >>> I'll investigate the packet corruption possibility first. I'll take = the scenic route: I noticed one of the cstruct examples is for pcap file = parsing, I'll try to add the ability to log the packets directly to a = local mirage block device. I wanted a similar thing for xenstore anyway. >>>=20 >>> If that fails I'll resort to printfs :) the problem always happens = after a handful of packets so this should be easy. >>>=20 >>> Cheers, >>> Dave >>>=20 >>=20 >>=20 >>=20 >> --=20 >> Dave Scott >>=20 >=20 >=20 From scott.dj@gmail.com Tue Sep 11 07:52:46 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBKL0-0005cL-S5 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 07:52:46 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.9 from SpamAssassin-3.3.2-1382695 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.51 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-vb0-f51.google.com ([209.85.212.51]:33231) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1TBKKz-0004Gn-RW (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 07:52:46 +0100 Received: by vbbfn1 with SMTP id fn1so199424vbb.38 for ; Mon, 10 Sep 2012 23:52:44 -0700 (PDT) MIME-Version: 1.0 Received: by 10.221.11.71 with SMTP id pd7mr24075382vcb.45.1347346364727; Mon, 10 Sep 2012 23:52:44 -0700 (PDT) Received: by 10.58.213.168 with HTTP; Mon, 10 Sep 2012 23:52:44 -0700 (PDT) In-Reply-To: <5DF2CDEF-60A5-493D-903F-027A74A645B8@recoil.org> References: <20120907165905.GA25178@dark.recoil.org> <5DF2CDEF-60A5-493D-903F-027A74A645B8@recoil.org> Date: Tue, 11 Sep 2012 07:52:44 +0100 Message-ID: Subject: Re: mirage-www From: David Scott To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=bcaec54fb0c07582e404c9678594 Cc: Balraj Singh , Dave Scott , Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 06:52:47 -0000 X-Keywords: X-UID: 542 Status: O Content-Length: 9419 Lines: 218 --bcaec54fb0c07582e404c9678594 Content-Type: text/plain; charset=ISO-8859-1 Heh, no problem. While I was debugging this I did a bit of "yak shaving" and now have a nice bit of code which can capture packets to a block device-- this may be useful in future. On Tuesday, September 11, 2012, Anil Madhavapeddy wrote: > That was my fault, sorry! This sort of thing should be fixed soon when > we can share C bindings more easily among the cross-compilation targets. > > -anil > > On 10 Sep 2012, at 22:31, David Scott wrote: > > > Right, problem solved: TCP checksum on xen was slightly broken. There > > was a fix made to Unix to cope with odd-length packets, this needed to > > be applied to xen as well. Many of the packets had valid checksums, > > and linux would ACK up to the sequence number in the last one of > > those. > > > > Now that the TCP bug is squashed, I can get back to writing my blog > > post -- all just a day in the life of a mirage hacker :-) > > > > On Sat, Sep 8, 2012 at 9:06 AM, Dave Scott > wrote: > >> > >> > >> On Sep 7, 2012, at 5:59 PM, "Anil Madhavapeddy" > wrote: > >> > >>> On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott wrote: > >>>> Hi, > >>>> > >>>> I've built mirage-www for xen and modified the static IP to fit my > >>>> environment. (I tried DHCP first but this didn't work -- if I get the > >>>> time I'll try to debug). > >>>> > >>>> I can ping the server fine, and it's certainly receiving a lot of > >>>> traffic on my (probably fairly busy) local network. When I try to > >>>> fetch a URL the TCP connection hangs. On the console I get: > >>>> > >>>> Dispatch: dynamic URL / > >>>> ... irrelevant spam > >>>> TCP retransmission on timer seq = -889321980 > >>>> > >>>> I've attached a small tcpdump of the conversation. I started with ping > >>>> and then tried HTTP. According to tcpdump/wireshark it went like this: > >>>> > >>>> $ tcpdump -r mirage.pcap -n > >>>> reading from file mirage.pcap, link-type EN10MB (Ethernet) > >>>> 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id > >>>> 9300, seq 1, length 64 > >>>> 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id > >>>> 9300, seq 1, length 64 > >>>> 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id > >>>> 9300, seq 2, length 64 > >>>> 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id > >>>> 9300, seq 2, length 64 > >>>> 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id > >>>> 9300, seq 3, length 64 > >>>> 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id > >>>> 9300, seq 3, length 64 > >>>> 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id > >>>> 9300, seq 4, length 64 > >>>> 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id > >>>> 9300, seq 4, length 64 > >>>> 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id > >>>> 9300, seq 5, length 64 > >>>> 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id > >>>> 9300, seq 5, length 64 > >>>> > >>>> -- so far so good, this is just my initial pings. Switching to 'wget > >>>> http://10.80.239.140/' > >>>> > >>>> 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [S], seq > >>>> 2284582709, win 5840, options [mss 1460,sackOK,TS val 909789846 ecr > >>>> 0,nop,wscale 6], length 0 > >>>> 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [S.], > >>>> seq 3536243828, ack 2284582710, win 65535, options [mss 1380,wscale > >>>> 2,eol], length 0 > >>>> 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], ack > >>>> 1, win 92, length 0 > >>>> > >>>> -- TCP connection established > >>>> > >>>> 13: -- Dave Scott --bcaec54fb0c07582e404c9678594 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Heh, no problem. While I was debugging this I did a bit of "yak shavin= g" and now have a nice bit of code which can capture packets to a bloc= k device-- this may be useful in future.

On Tuesday, Se= ptember 11, 2012, Anil Madhavapeddy wrote:
That was my fault, sorry! This sort of thing= should be fixed soon when
we can share C bindings more easily among the cross-compilation targets.
-anil

On 10 Sep 2012, at 22:31, David Scott <scott.dj@gmail.com> wro= te:

> Right, problem solved: TCP checksum on xen was slightly broken. There<= br> > was a fix made to Unix to cope with odd-length packets, this needed to=
> be applied to xen as well. Many of the packets had valid checksums, > and linux would ACK up to the sequence number in the last one of
> those.
>
> Now that the TCP bug is squashed, I can get back to writing my blog > post -- all just a day in the life of a mirage hacker :-)
>
> On Sat, Sep 8, 2012 at 9:06 AM, Dave Scott <Dave.Scott@eu.citrix= .com> wrote:
>>
>>
>> On Sep 7, 2012, at 5:59 PM, "Anil Madhavapeddy" <a= nil@recoil.org> wrote:
>>
>>> On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott wrote: >>>> Hi,
>>>>
>>>> I've built mirage-www for xen and modified the static = IP to fit my
>>>> environment. (I tried DHCP first but this didn't work = -- if I get the
>>>> time I'll try to debug).
>>>>
>>>> I can ping the server fine, and it's certainly receivi= ng a lot of
>>>> traffic on my (probably fairly busy) local network. When I= try to
>>>> fetch a URL the TCP connection hangs. On the console I get= :
>>>>
>>>> Dispatch: dynamic URL /
>>>> ... irrelevant spam
>>>> TCP retransmission on timer seq =3D -889321980
>>>>
>>>> I've attached a small tcpdump of the conversation. I s= tarted with ping
>>>> and then tried HTTP. According to tcpdump/wireshark it wen= t like this:
>>>>
>>>> $ tcpdump -r mirage.pcap -n
>>>> reading from file mirage.pcap, link-type EN10MB (Ethernet)=
>>>> 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id
>>>> 9300, seq 1, length 64
>>>> 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id
>>>> 9300, seq 1, length 64
>>>> 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id
>>>> 9300, seq 2, length 64
>>>> 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id
>>>> 9300, seq 2, length 64
>>>> 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id
>>>> 9300, seq 3, length 64
>>>> 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id
>>>> 9300, seq 3, length 64
>>>> 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id
>>>> 9300, seq 4, length 64
>>>> 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id
>>>> 9300, seq 4, length 64
>>>> 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id
>>>> 9300, seq 5, length 64
>>>> 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id
>>>> 9300, seq 5, length 64
>>>>
>>>> -- so far so good, this is just my initial pings. Switchin= g to 'wget
>>>> http:/= /10.80.239.140/'
>>>>
>>>> 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80:= Flags [S], seq
>>>> 2284582709, win 5840, options [mss 1460,sackOK,TS val 9097= 89846 ecr
>>>> 0,nop,wscale 6], length 0
>>>> 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158:= Flags [S.],
>>>> seq 3536243828, ack 2284582710, win 65535, options [mss 13= 80,wscale
>>>> 2,eol], length 0
>>>> 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80:= Flags [.], ack
>>>> 1, win 92, length 0
>>>>
>>>> -- TCP connection established
>>>>
>>>> 13:


--
Dave Scott
--bcaec54fb0c07582e404c9678594-- From Richard.Mortier@nottingham.ac.uk Tue Sep 11 09:25:14 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBLmU-0000zA-JJ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 09:25:14 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.6 from SpamAssassin-3.3.2-1382695 * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from thb-mta-19-tx.emailfiltering.com ([194.116.199.150]:43224 helo=thb-mta-19.emailfiltering.com) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1TBLmO-0000R0-QO (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 09:25:14 +0100 Received: from smtp4.nottingham.ac.uk ([128.243.220.65]) by thb-mta-19.emailfiltering.com with emfmta (version 4.8.5.104) by TLS id 2554855482 ;2bc268d57c1d256c; Tue, 11 Sep 2012 09:24:18 +0100 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp4.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1TBLla-0000SV-L2; Tue, 11 Sep 2012 09:24:18 +0100 Received: from EXCHANGE1.ad.nottingham.ac.uk ([fe80::60c2:2879:5a5a:29a6]) by UIWEXHUB02.ad.nottingham.ac.uk ([2002:80f3:f84::80f3:f84]) with mapi; Tue, 11 Sep 2012 09:24:10 +0100 From: Richard Mortier To: David Scott Date: Tue, 11 Sep 2012 09:24:09 +0100 Subject: Re: mirage-www Thread-Topic: mirage-www Thread-Index: Ac2P9tD1NTqAMuhJQr6UMfeW6GSKbQ== Message-ID: References: <20120907165905.GA25178@dark.recoil.org> <5DF2CDEF-60A5-493D-903F-027A74A645B8@recoil.org> In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Balraj Singh , Mirage, Dave Scott , List , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 08:25:14 -0000 X-Keywords: X-UID: 543 Status: O Content-Length: 8892 Lines: 167
On 11 Sep 20= 12, at 07:52, David Scott wrote:

Heh, no problem. While I was debugging this I = did a bit of "yak shaving" and now have a nice bit of code which can captur= e packets to a block device-- this may be useful in future.

oh- nice. can you point me to it please? = (I can sense some student project proposals coming on :)


On Tuesday, September 11, 2012, Anil Madhavapeddy wro= te:
That was my fault, sorry! This sort of thing= should be fixed soon when
we can share C bindings more easily among the cross-compilation targets.
-anil

On 10 Sep 2012, at 22:31, David Scott <scott.dj@gmail.com> wro= te:

> Right, problem solved: TCP checksum on xen was slightly broken. There<= br> > was a fix made to Unix to cope with odd-length packets, this needed to=
> be applied to xen as well. Many of the packets had valid checksums, > and linux would ACK up to the sequence number in the last one of
> those.
>
> Now that the TCP bug is squashed, I can get back to writing my blog > post -- all just a day in the life of a mirage hacker :-)
>
> On Sat, Sep 8, 2012 at 9:06 AM, Dave Scott <Dave.Scott@eu.citrix= .com> wrote:
>>
>>
>> On Sep 7, 2012, at 5:59 PM, "Anil Madhavapeddy" <anil@recoil= .org> wrote:
>>
>>> On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott wrote: >>>> Hi,
>>>>
>>>> I've built mirage-www for xen and modified the static IP t= o fit my
>>>> environment. (I tried DHCP first but this didn't work -- i= f I get the
>>>> time I'll try to debug).
>>>>
>>>> I can ping the server fine, and it's certainly receiving a= lot of
>>>> traffic on my (probably fairly busy) local network. When I= try to
>>>> fetch a URL the TCP connection hangs. On the console I get= :
>>>>
>>>> Dispatch: dynamic URL /
>>>> ... irrelevant spam
>>>> TCP retransmission on timer seq =3D -889321980
>>>>
>>>> I've attached a small tcpdump of the conversation. I start= ed with ping
>>>> and then tried HTTP. According to tcpdump/wireshark it wen= t like this:
>>>>
>>>> $ tcpdump -r mirage.pcap -n
>>>> reading from file mirage.pcap, link-type EN10MB (Ethernet)=
>>>> 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id
>>>> 9300, seq 1, length 64
>>>> 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id
>>>> 9300, seq 1, length 64
>>>> 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id
>>>> 9300, seq 2, length 64
>>>> 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id
>>>> 9300, seq 2, length 64
>>>> 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id
>>>> 9300, seq 3, length 64
>>>> 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id
>>>> 9300, seq 3, length 64
>>>> 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id
>>>> 9300, seq 4, length 64
>>>> 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id
>>>> 9300, seq 4, length 64
>>>> 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id
>>>> 9300, seq 5, length 64
>>>> 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id
>>>> 9300, seq 5, length 64
>>>>
>>>> -- so far so good, this is just my initial pings. Switchin= g to 'wget
>>>> http:/= /10.80.239.140/'
>>>>
>>>> 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80:= Flags [S], seq
>>>> 2284582709, win 5840, options [mss 1460,sackOK,TS val 9097= 89846 ecr
>>>> 0,nop,wscale 6], length 0
>>>> 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158:= Flags [S.],
>>>> seq 3536243828, ack 2284582710, win 65535, options [mss 13= 80,wscale
>>>> 2,eol], length 0
>>>> 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80:= Flags [.], ack
>>>> 1, win 92, length 0
>>>>
>>>> -- TCP connection established
>>>>
>>>> 13:


--
Dave Scott


-- 
Cheers,




=
=0D=0A

=0D=0AThis message and any attachment are intended solely= for the addressee and may=20=0D=0Acontain confidential information. If= you have received this message in error,=20=0D=0Aplease send it back t= o me, and immediately delete it. Please do not use,=20=0D=0Acopy or d= isclose the information contained in this message or in any attachment.= =20=0D=0AAny views or opinions expressed by the author of this email d= o not necessarily=20=0D=0Areflect the views of the University of Nottin= gham.=0D=0A

=0D=0A

=0D=0AThis message has been checked for viruses= but the contents of an attachment=0D=0Amay still contain software viru= ses which could damage your computer system:=0D=0Ayou are advised to pe= rform your own checks. Email communications with the=0D=0AUniversity of= Nottingham may be monitored as permitted by UK legislation.=0D=0A

= = From anil@recoil.org Tue Sep 11 09:27:47 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBLox-00012R-5k (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 09:27:47 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1382695 * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:47031 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1TBLow-0006Aq-Wo (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 09:27:47 +0100 Received: (qmail 23597 invoked by uid 634); 11 Sep 2012 08:27:46 -0000 Received: from no-dns-yet.demon.co.uk (HELO [192.168.15.13]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 11 Sep 2012 09:27:46 +0100 Content-Type: multipart/alternative; boundary="Apple-Mail=_A2A77824-95A3-45FC-9000-6C43C0C3926E" Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: mirage-www From: Anil Madhavapeddy In-Reply-To: Date: Tue, 11 Sep 2012 09:27:36 +0100 Message-Id: <93057C5B-CCC6-401A-8492-09AB55A8F754@recoil.org> References: <20120907165905.GA25178@dark.recoil.org> <5DF2CDEF-60A5-493D-903F-027A74A645B8@recoil.org> To: Richard Mortier X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 08:27:47 -0000 X-Keywords: X-UID: 544 Status: O Content-Length: 14581 Lines: 369 --Apple-Mail=_A2A77824-95A3-45FC-9000-6C43C0C3926E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 On 11 Sep 2012, at 09:24, Richard Mortier = wrote: >=20 > On 11 Sep 2012, at 07:52, David Scott wrote: >=20 >> Heh, no problem. While I was debugging this I did a bit of "yak = shaving" and now have a nice bit of code which can capture packets to a = block device-- this may be useful in future. >=20 > oh- nice. can you point me to it please? (I can sense some student = project proposals coming on :) https://github.com/djs55/mirage-net/tree/pcap https://github.com/djs55/ocaml-pcap -anil >=20 >>=20 >> On Tuesday, September 11, 2012, Anil Madhavapeddy wrote: >> That was my fault, sorry! This sort of thing should be fixed soon = when >> we can share C bindings more easily among the cross-compilation = targets. >>=20 >> -anil >>=20 >> On 10 Sep 2012, at 22:31, David Scott wrote: >>=20 >> > Right, problem solved: TCP checksum on xen was slightly broken. = There >> > was a fix made to Unix to cope with odd-length packets, this needed = to >> > be applied to xen as well. Many of the packets had valid checksums, >> > and linux would ACK up to the sequence number in the last one of >> > those. >> > >> > Now that the TCP bug is squashed, I can get back to writing my blog >> > post -- all just a day in the life of a mirage hacker :-) >> > >> > On Sat, Sep 8, 2012 at 9:06 AM, Dave Scott = wrote: >> >> >> >> >> >> On Sep 7, 2012, at 5:59 PM, "Anil Madhavapeddy" = wrote: >> >> >> >>> On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott wrote: >> >>>> Hi, >> >>>> >> >>>> I've built mirage-www for xen and modified the static IP to fit = my >> >>>> environment. (I tried DHCP first but this didn't work -- if I = get the >> >>>> time I'll try to debug). >> >>>> >> >>>> I can ping the server fine, and it's certainly receiving a lot = of >> >>>> traffic on my (probably fairly busy) local network. When I try = to >> >>>> fetch a URL the TCP connection hangs. On the console I get: >> >>>> >> >>>> Dispatch: dynamic URL / >> >>>> ... irrelevant spam >> >>>> TCP retransmission on timer seq =3D -889321980 >> >>>> >> >>>> I've attached a small tcpdump of the conversation. I started = with ping >> >>>> and then tried HTTP. According to tcpdump/wireshark it went like = this: >> >>>> >> >>>> $ tcpdump -r mirage.pcap -n >> >>>> reading from file mirage.pcap, link-type EN10MB (Ethernet) >> >>>> 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo = request, id >> >>>> 9300, seq 1, length 64 >> >>>> 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, = id >> >>>> 9300, seq 1, length 64 >> >>>> 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo = request, id >> >>>> 9300, seq 2, length 64 >> >>>> 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, = id >> >>>> 9300, seq 2, length 64 >> >>>> 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo = request, id >> >>>> 9300, seq 3, length 64 >> >>>> 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, = id >> >>>> 9300, seq 3, length 64 >> >>>> 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo = request, id >> >>>> 9300, seq 4, length 64 >> >>>> 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, = id >> >>>> 9300, seq 4, length 64 >> >>>> 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo = request, id >> >>>> 9300, seq 5, length 64 >> >>>> 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, = id >> >>>> 9300, seq 5, length 64 >> >>>> >> >>>> -- so far so good, this is just my initial pings. Switching to = 'wget >> >>>> http://10.80.239.140/' >> >>>> >> >>>> 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags = [S], seq >> >>>> 2284582709, win 5840, options [mss 1460,sackOK,TS val 909789846 = ecr >> >>>> 0,nop,wscale 6], length 0 >> >>>> 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags = [S.], >> >>>> seq 3536243828, ack 2284582710, win 65535, options [mss = 1380,wscale >> >>>> 2,eol], length 0 >> >>>> 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags = [.], ack >> >>>> 1, win 92, length 0 >> >>>> >> >>>> -- TCP connection established >> >>>> >> >>>> 13: >>=20 >>=20 >> --=20 >> Dave Scott >=20 >=20 > --=20 > Cheers, >=20 > R. >=20 >=20 >=20 >=20 >=20 > This message and any attachment are intended solely for the addressee = and may contain confidential information. If you have received this = message in error, please send it back to me, and immediately delete it. = Please do not use, copy or disclose the information contained in this = message or in any attachment. Any views or opinions expressed by the = author of this email do not necessarily reflect the views of the = University of Nottingham. >=20 > This message has been checked for viruses but the contents of an = attachment may still contain software viruses which could damage your = computer system: you are advised to perform your own checks. Email = communications with the University of Nottingham may be monitored as = permitted by UK legislation. >=20 --Apple-Mail=_A2A77824-95A3-45FC-9000-6C43C0C3926E Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 Richard.Mortier@nottingha= m.ac.uk> wrote:

On 11 Sep 2012, = at 07:52, David Scott wrote:

Heh, no = problem. While I was debugging this I did a bit of "yak shaving" and now = have a nice bit of code which can capture packets to a block device-- = this may be useful in = future.

oh- nice. can = you point me to it please? (I can sense some student project proposals = coming on :)

https://github.com/= djs55/mirage-net/tree/pcap

-anil



On Tuesday, September 11, 2012, Anil Madhavapeddy = wrote:
That was my fault, = sorry! This sort of thing should be fixed soon when
we can share C bindings more easily among the cross-compilation = targets.

-anil

On 10 Sep 2012, at 22:31, David Scott <scott.dj@gmail.com> = wrote:

> Right, problem solved: TCP checksum on xen was slightly broken. = There
> was a fix made to Unix to cope with odd-length packets, this needed = to
> be applied to xen as well. Many of the packets had valid = checksums,
> and linux would ACK up to the sequence number in the last one = of
> those.
>
> Now that the TCP bug is squashed, I can get back to writing my = blog
> post -- all just a day in the life of a mirage hacker :-)
>
> On Sat, Sep 8, 2012 at 9:06 AM, Dave Scott = <Dave.Scott@eu.citrix.com> wrote:
>>
>>
>> On Sep 7, 2012, at 5:59 PM, "Anil Madhavapeddy" = <anil@recoil.org> wrote:
>>
>>> On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott = wrote:
>>>> Hi,
>>>>
>>>> I've built mirage-www for xen and modified the static = IP to fit my
>>>> environment. (I tried DHCP first but this didn't work = -- if I get the
>>>> time I'll try to debug).
>>>>
>>>> I can ping the server fine, and it's certainly = receiving a lot of
>>>> traffic on my (probably fairly busy) local network. = When I try to
>>>> fetch a URL the TCP connection hangs. On the console I = get:
>>>>
>>>> Dispatch: dynamic URL /
>>>> ... irrelevant spam
>>>> TCP retransmission on timer seq =3D -889321980
>>>>
>>>> I've attached a small tcpdump of the conversation. I = started with ping
>>>> and then tried HTTP. According to tcpdump/wireshark it = went like this:
>>>>
>>>> $ tcpdump -r mirage.pcap -n
>>>> reading from file mirage.pcap, link-type EN10MB = (Ethernet)
>>>> 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP = echo request, id
>>>> 9300, seq 1, length 64
>>>> 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo = reply, id
>>>> 9300, seq 1, length 64
>>>> 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP = echo request, id
>>>> 9300, seq 2, length 64
>>>> 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo = reply, id
>>>> 9300, seq 2, length 64
>>>> 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP = echo request, id
>>>> 9300, seq 3, length 64
>>>> 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo = reply, id
>>>> 9300, seq 3, length 64
>>>> 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP = echo request, id
>>>> 9300, seq 4, length 64
>>>> 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo = reply, id
>>>> 9300, seq 4, length 64
>>>> 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP = echo request, id
>>>> 9300, seq 5, length 64
>>>> 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo = reply, id
>>>> 9300, seq 5, length 64
>>>>
>>>> -- so far so good, this is just my initial pings. = Switching to 'wget
>>>> http://10.80.239.140/'
>>>>
>>>> 13:47:14.241216 IP 10.80.2.32.37158 > = 10.80.239.140.80: Flags [S], seq
>>>> 2284582709, win 5840, options [mss 1460,sackOK,TS val = 909789846 ecr
>>>> 0,nop,wscale 6], length 0
>>>> 13:47:14.242365 IP 10.80.239.140.80 > = 10.80.2.32.37158: Flags [S.],
>>>> seq 3536243828, ack 2284582710, win 65535, options [mss = 1380,wscale
>>>> 2,eol], length 0
>>>> 13:47:14.242387 IP 10.80.2.32.37158 > = 10.80.239.140.80: Flags [.], ack
>>>> 1, win 92, length 0
>>>>
>>>> -- TCP connection established
>>>>
>>>> 13:


--
Dave Scott


--
Cheers,
R.



This message and any attachment are intended solely for the addressee = and may=20 contain confidential information. If you have received this message in = error,=20 please send it back to me, and immediately delete it. Please do not = use,=20 copy or disclose the information contained in this message or in any = attachment. =20 Any views or opinions expressed by the author of this email do not = necessarily=20 reflect the views of the University of Nottingham.

This message has been checked for viruses but the contents of an = attachment may still contain software viruses which could damage your computer = system: you are advised to perform your own checks. Email communications with = the University of Nottingham may be monitored as permitted by UK = legislation.


= --Apple-Mail=_A2A77824-95A3-45FC-9000-6C43C0C3926E-- From scott.dj@gmail.com Tue Sep 11 09:36:04 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBLwy-0001SX-DR (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 09:36:04 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.9 from SpamAssassin-3.3.2-1382695 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.51 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-vb0-f51.google.com ([209.85.212.51]:60969) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1TBLwx-0005Px-pl (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 09:36:04 +0100 Received: by vbbfn1 with SMTP id fn1so286569vbb.38 for ; Tue, 11 Sep 2012 01:36:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.35.116 with SMTP id g20mr19336495vdj.97.1347352562164; Tue, 11 Sep 2012 01:36:02 -0700 (PDT) Received: by 10.58.213.168 with HTTP; Tue, 11 Sep 2012 01:36:02 -0700 (PDT) In-Reply-To: References: <20120907165905.GA25178@dark.recoil.org> <5DF2CDEF-60A5-493D-903F-027A74A645B8@recoil.org> Date: Tue, 11 Sep 2012 09:36:02 +0100 Message-ID: Subject: Re: mirage-www From: David Scott To: Richard Mortier Content-Type: text/plain; charset=ISO-8859-1 Cc: Balraj Singh , Dave Scott , Mirage List , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 08:36:04 -0000 X-Keywords: X-UID: 545 Status: O Content-Length: 5786 Lines: 162 On Tue, Sep 11, 2012 at 9:24 AM, Richard Mortier wrote: > > On 11 Sep 2012, at 07:52, David Scott wrote: > > Heh, no problem. While I was debugging this I did a bit of "yak shaving" and > now have a nice bit of code which can capture packets to a block device-- > this may be useful in future. > > > oh- nice. can you point me to it please? (I can sense some student project > proposals coming on :) This patch causes all incoming/outgoing ethernet frames to be 'captured' by which I mean added to a non-blocking Lwt_stream: https://github.com/djs55/mirage-net/commit/6bbb77f7af9f6d210b7af2f6b3b239f0272861d2 Initially the stream size is set to 0, so all packets are dropped i.e. no recording is actually done. This small repo extracts the pcap example from ocaml-cstruct and adds some scaffolding around it: https://github.com/djs55/ocaml-pcap It's a bit minimal but could be easily expanded. This module: https://github.com/djs55/ocaml-pcap/blob/master/mirage/pcap_mirage.ml contains code to pull packets from the Lwt_stream and append them -- with pcap headers -- to the block device. The top function is basically an experiment in making a 'file-like thing'. I started the capturing in the mirage-www main loop: https://github.com/djs55/mirage-www/blob/djs-blog/src/server.ml HTH, Dave > > > On Tuesday, September 11, 2012, Anil Madhavapeddy wrote: >> >> That was my fault, sorry! This sort of thing should be fixed soon when >> we can share C bindings more easily among the cross-compilation targets. >> >> -anil >> >> On 10 Sep 2012, at 22:31, David Scott wrote: >> >> > Right, problem solved: TCP checksum on xen was slightly broken. There >> > was a fix made to Unix to cope with odd-length packets, this needed to >> > be applied to xen as well. Many of the packets had valid checksums, >> > and linux would ACK up to the sequence number in the last one of >> > those. >> > >> > Now that the TCP bug is squashed, I can get back to writing my blog >> > post -- all just a day in the life of a mirage hacker :-) >> > >> > On Sat, Sep 8, 2012 at 9:06 AM, Dave Scott >> > wrote: >> >> >> >> >> >> On Sep 7, 2012, at 5:59 PM, "Anil Madhavapeddy" >> >> wrote: >> >> >> >>> On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott wrote: >> >>>> Hi, >> >>>> >> >>>> I've built mirage-www for xen and modified the static IP to fit my >> >>>> environment. (I tried DHCP first but this didn't work -- if I get the >> >>>> time I'll try to debug). >> >>>> >> >>>> I can ping the server fine, and it's certainly receiving a lot of >> >>>> traffic on my (probably fairly busy) local network. When I try to >> >>>> fetch a URL the TCP connection hangs. On the console I get: >> >>>> >> >>>> Dispatch: dynamic URL / >> >>>> ... irrelevant spam >> >>>> TCP retransmission on timer seq = -889321980 >> >>>> >> >>>> I've attached a small tcpdump of the conversation. I started with >> >>>> ping >> >>>> and then tried HTTP. According to tcpdump/wireshark it went like >> >>>> this: >> >>>> >> >>>> $ tcpdump -r mirage.pcap -n >> >>>> reading from file mirage.pcap, link-type EN10MB (Ethernet) >> >>>> 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >> >>>> 9300, seq 1, length 64 >> >>>> 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >> >>>> 9300, seq 1, length 64 >> >>>> 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >> >>>> 9300, seq 2, length 64 >> >>>> 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >> >>>> 9300, seq 2, length 64 >> >>>> 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >> >>>> 9300, seq 3, length 64 >> >>>> 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >> >>>> 9300, seq 3, length 64 >> >>>> 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >> >>>> 9300, seq 4, length 64 >> >>>> 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >> >>>> 9300, seq 4, length 64 >> >>>> 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >> >>>> 9300, seq 5, length 64 >> >>>> 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >> >>>> 9300, seq 5, length 64 >> >>>> >> >>>> -- so far so good, this is just my initial pings. Switching to 'wget >> >>>> http://10.80.239.140/' >> >>>> >> >>>> 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [S], >> >>>> seq >> >>>> 2284582709, win 5840, options [mss 1460,sackOK,TS val 909789846 ecr >> >>>> 0,nop,wscale 6], length 0 >> >>>> 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [S.], >> >>>> seq 3536243828, ack 2284582710, win 65535, options [mss 1380,wscale >> >>>> 2,eol], length 0 >> >>>> 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], >> >>>> ack >> >>>> 1, win 92, length 0 >> >>>> >> >>>> -- TCP connection established >> >>>> >> >>>> 13: > > > > -- > Dave Scott > > > > -- > Cheers, > > R. > > > > > > This message and any attachment are intended solely for the addressee and > may contain confidential information. If you have received this message in > error, please send it back to me, and immediately delete it. Please do not > use, copy or disclose the information contained in this message or in any > attachment. Any views or opinions expressed by the author of this email do > not necessarily reflect the views of the University of Nottingham. > > This message has been checked for viruses but the contents of an attachment > may still contain software viruses which could damage your computer system: > you are advised to perform your own checks. Email communications with the > University of Nottingham may be monitored as permitted by UK legislation. -- Dave Scott From anil@recoil.org Tue Sep 11 09:50:59 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBMBP-00021L-N2 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 09:50:59 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1382695 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:25021 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1TBMBP-0007vD-XI (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 09:50:59 +0100 Received: (qmail 32604 invoked by uid 634); 11 Sep 2012 08:50:59 -0000 Received: from no-dns-yet.demon.co.uk (HELO [192.168.15.13]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 11 Sep 2012 09:50:59 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: starting alpha releases Message-Id: Date: Tue, 11 Sep 2012 09:50:53 +0100 To: Mirage List Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 08:50:59 -0000 X-Keywords: X-UID: 546 Status: O Content-Length: 1816 Lines: 58 Since we're going to be showing this stuff off at CUFP this week, it's a = good time to start doing alpha releases that can show up in OPAM as = non-dev packages. The interfaces are by no means settled yet, of = course, but we have enough applications that it's worthwhile. So far, I've cut releases of: - uri - re - cow - cstruct - dyntype They are all available via OPAM. We need to cut releases of: - cohttp (Anil, will do today) - openflow (with help from Haris and his shiny new Mac, hopefully) - dns (it needs a Cohttp-style Mirage+Lwt adapter. Mort, got time to = try?) - lwt-logger (Anil, will try to do something this week, as it's a neat = demo of cohttp) - crypto-keys (Haris/Anil -- Lwt only for now) - shelf (Thomas?) - ORM (Thomas? Sqlite3 only for now, and probably forever) - SSH (Anil, needs porting to cstruct, so not this week) - github API (Anil, will do today) I propose that we also cut immediate alphas of: - mirage-platform - mirage-net - mirage-fs (completely missing the Block module, but it has mir-crunch) - ocaml-xenstore ...with the current featureset, as we've evaluated them fairly well so = far. Then, we can start merging in all the outstanding features for the = first 'public' release, including: - kFreeBSD backend (Gabor) - NS3 backend (Haris) - Debugging patches for Netif (Dave) - external Xenstore library instead of bundled one in OS (Dave/Anil) - TCP offload for Xen (Anil) - Cryptokit (Anil) All the release repositories are under the Mirage Github organisation, = and releases are Git tags. So keep your personal forks for features, = and send pull requests up to the Mirage/* repos. I will *delete* these repos on Mirage as they are now well out-of-date: - cohttp - cohttp-server So that's quite a bit of stuff! Comments welcome from all named :) -anil From thomas.gazagnaire@gmail.com Tue Sep 11 10:41:02 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBMxq-00047w-J3 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 10:41:02 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.9 from SpamAssassin-3.3.2-1382695 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.51 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-wg0-f51.google.com ([74.125.82.51]:63104) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1TBMxo-0002T0-rS (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 10:41:02 +0100 Received: by wgbed3 with SMTP id ed3so198407wgb.20 for ; Tue, 11 Sep 2012 02:41:00 -0700 (PDT) Received: by 10.180.20.11 with SMTP id j11mr23861497wie.12.1347356460349; Tue, 11 Sep 2012 02:41:00 -0700 (PDT) Received: from [192.168.0.14] (gou06-3-88-170-165-56.fbx.proxad.net. [88.170.165.56]) by mx.google.com with ESMTPS id j6sm2715060wiy.4.2012.09.11.02.40.58 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 11 Sep 2012 02:40:59 -0700 (PDT) Subject: Re: starting alpha releases Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: Date: Tue, 11 Sep 2012 11:40:57 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <6B7A8ACE-E03F-4D46-8A91-1FE86E6D0B1E@gmail.com> References: To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1084) Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 09:41:02 -0000 X-Keywords: X-UID: 547 Status: O Content-Length: 324 Lines: 12 > - shelf (Thomas?) > - ORM (Thomas? Sqlite3 only for now, and probably forever) I had a quick try, but they both depend on an older release of dyntype = which itself depends on an older version of type_conv that I can't find = online anymore... will try to update them to new dyntype in the plane to = OUD... -- Thomas From anil@recoil.org Tue Sep 11 11:04:30 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBNKY-000533-NL (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 11:04:30 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1382695 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:19080 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1TBNKX-0000YF-ZC (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 11:04:30 +0100 Received: (qmail 20280 invoked by uid 634); 11 Sep 2012 10:04:29 -0000 Received: from no-dns-yet.demon.co.uk (HELO [192.168.15.13]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 11 Sep 2012 11:04:29 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: starting alpha releases From: Anil Madhavapeddy In-Reply-To: <6B7A8ACE-E03F-4D46-8A91-1FE86E6D0B1E@gmail.com> Date: Tue, 11 Sep 2012 11:04:27 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <4AACC4B8-CBD5-4FB5-B6B6-46D07ED09D80@recoil.org> References: <6B7A8ACE-E03F-4D46-8A91-1FE86E6D0B1E@gmail.com> To: Thomas Gazagnaire X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 10:04:30 -0000 X-Keywords: X-UID: 548 Status: O Content-Length: 914 Lines: 26 On 11 Sep 2012, at 10:40, Thomas Gazagnaire = wrote: >> - shelf (Thomas?) >> - ORM (Thomas? Sqlite3 only for now, and probably forever) >=20 > I had a quick try, but they both depend on an older release of dyntype = which itself depends on an older version of type_conv that I can't find = online anymore... will try to update them to new dyntype in the plane to = OUD... Same deal with the type_conv version of JSON (which has loads of users = now, it seems). I'm going with ATDgen for the JSON part of the tutorial (and it's used = in ocaml-github too for the JSON parsing): https://github.com/avsm/ocaml-github/blob/master/lib/github.atd It's pretty good, except for the lack of a module system for the = generated code (and so record fields all need to have a prefix). I much = prefer the more 'integrated' type_conv code generation, even though it's = less flexible. -a= From pgj@caesar.elte.hu Tue Sep 11 14:10:28 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBQEW-0003Ah-Df (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 14:10:28 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.1 from SpamAssassin-3.3.2-1382695 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [157.181.1.138 listed in list.dnswl.dnsbl.ja.net] * -0.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mx3.mail.elte.hu ([157.181.1.138]:47715) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1TBQEV-000734-oS (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 14:10:28 +0100 Received: from login01.caesar.elte.hu ([157.181.151.130]) by mx3.mail.elte.hu with esmtp (Exim) id 1TBQEU-0002Ia-OY from ; Tue, 11 Sep 2012 15:10:27 +0200 Received: (Authenticated sender: pgj) by login01.caesar.elte.hu with local (Exim 4.72) (envelope-from ) id 1TBQES-0006Xf-Eu; Tue, 11 Sep 2012 15:10:24 +0200 Date: Tue, 11 Sep 2012 15:10:24 +0200 From: PALI Gabor Janos To: Anil Madhavapeddy Subject: Re: starting alpha releases Message-ID: <20120911131024.GA22854@caesar.elte.hu> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Sender: Pali Gabor Janos X-ELTE-SpamScore: -2.0 X-ELTE-SpamLevel: X-ELTE-SpamCheck: no X-ELTE-SpamVersion: ELTE 2.0 X-ELTE-SpamCheck-Details: score=-2.0 required=5.9 tests=BAYES_00 autolearn=no SpamAssassin version=3.3.1 -2.0 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 13:10:28 -0000 X-Keywords: X-UID: 549 Status: O Content-Length: 1278 Lines: 30 Hi Anil, On Tue, Sep 11, 2012 at 09:50:53AM +0100, Anil Madhavapeddy wrote: > Then, we can start merging in all the outstanding features for the first 'public' release, including: > > - kFreeBSD backend (Gabor) Is there an exact deadline for this? I have not had the chance to put larger amount of stress on this backend, although it implements the mirage-os interfaces. I was pondering perhaps I could test this with the mirage web site application. I do not see too many obstacles except the eliminiation of use of floating-point numbers in the packages, but the devil is in the details. Speaking of floating-point numbers: it is essentially a requirement to switch from real-valued time representation of OCaml to microseconds, otherwise we will not be able to merge the kFreeBSD backend. On the other hand, the FreeBSD pfil(9) hook interface has just been changed to support for link-layer filtering [1] so we can stop piggybacking on ng_ether(4). If I had some time I could adapt my code to use this. Note that my current code has a build solution different from OPAM, so it would require some effort to make fit with it. I think this will be another thing we should be aware of. [1] http://lists.freebsd.org/pipermail/svn-src-head/2012-September/040067.html From balraj885@gmail.com Tue Sep 11 14:56:23 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBQwx-0004An-SN (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 14:56:23 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.5 from SpamAssassin-3.3.2-1382695 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.179 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (balraj885[at]gmail.com) * 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in * digit (balraj885[at]gmail.com) * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-we0-f179.google.com ([74.125.82.179]:58047) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1TBQwv-0007NS-QX (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 14:56:23 +0100 Received: by weyx10 with SMTP id x10so285136wey.38 for ; Tue, 11 Sep 2012 06:56:21 -0700 (PDT) MIME-Version: 1.0 Received: by 10.180.102.136 with SMTP id fo8mr25350468wib.19.1347371780832; Tue, 11 Sep 2012 06:56:20 -0700 (PDT) Sender: balraj885@gmail.com Received: by 10.194.56.100 with HTTP; Tue, 11 Sep 2012 06:56:20 -0700 (PDT) In-Reply-To: References: <20120907165905.GA25178@dark.recoil.org> <5DF2CDEF-60A5-493D-903F-027A74A645B8@recoil.org> Date: Tue, 11 Sep 2012 14:56:20 +0100 X-Google-Sender-Auth: iJBEKDl5nygg1B3rdBRKsfiPjBw Message-ID: Subject: Re: mirage-www From: Balraj Singh To: David Scott Content-Type: text/plain; charset=ISO-8859-1 Cc: Richard Mortier , Dave Scott , Mirage List , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 13:56:24 -0000 X-Keywords: X-UID: 550 Status: O Content-Length: 8429 Lines: 210 Sorry, I was offline for a few days. This was the exact bug that I was working on before vacation/August/... >From the outside the main symptom was that packets were getting lost. This was compounded because of a bug that got introduced when I added the last bits of window scaling - the bug prevented the sender from recognising dup-acks and so fast rexmit was never triggered. That meant that all rexmits happened on timer which really slowed things down. But after that was fixed there was still a problem because sometimes multiple packets in a window would get lost and since we don't yet do SACK the second+ loss would still only be sent on timer. So the real problem was pkt loss, which there should have been none of in my test rig. It turned out that the packets weren't being lost, they were just arriving with invalid tcp checksums. Unfortunately this was not because the checksum was computed incorrectly, but because the payload was damaged after the checksum computation. More unfortunately, my retransmission code saves the payload after the first transmission and if the pkt has to be retransmitted, it would recompute the checksum. So if, as it happens now, the payload is corrupted after the checksum is computed, the packet would correctly be dropped by the receiver, but when the pkt is rexmited it would recompute the checksum of the corrupted payload, which would result in the packet being accepted and the whole stream getting corrupted! This is the worst kind of TCP bug - badness factor 100 to me. What the tcp code should do is compute the checksum as soon as it receives the payload from the application and then only adjust the chksum for any changes it needs to make on first or subsequent transmissions. That would prevent the stream corruption which is at the top of tcp's guarantees - it would just stop all transmission on the stream but not let it get corrupted. So, I was/am trying to figure out what was causing the corruption. It has an interesting signature - most of the time the IP+TCP header of that particular packet was being written over the first 40 bytes of the payload. Dave, would the problem that you fixed in the xen tcp checksum code cause this kind of corruption? If yes that would be a relief. I'll try to get your fix and see if the corruption goes away (prob do this tomorrow). Thx, sorry about the saga. Balraj On Tue, Sep 11, 2012 at 9:36 AM, David Scott wrote: > On Tue, Sep 11, 2012 at 9:24 AM, Richard Mortier > wrote: >> >> On 11 Sep 2012, at 07:52, David Scott wrote: >> >> Heh, no problem. While I was debugging this I did a bit of "yak shaving" and >> now have a nice bit of code which can capture packets to a block device-- >> this may be useful in future. >> >> >> oh- nice. can you point me to it please? (I can sense some student project >> proposals coming on :) > > This patch causes all incoming/outgoing ethernet frames to be > 'captured' by which I mean added to a non-blocking Lwt_stream: > > https://github.com/djs55/mirage-net/commit/6bbb77f7af9f6d210b7af2f6b3b239f0272861d2 > > Initially the stream size is set to 0, so all packets are dropped i.e. > no recording is actually done. > > This small repo extracts the pcap example from ocaml-cstruct and adds > some scaffolding around it: > > https://github.com/djs55/ocaml-pcap > > It's a bit minimal but could be easily expanded. This module: > > https://github.com/djs55/ocaml-pcap/blob/master/mirage/pcap_mirage.ml > > contains code to pull packets from the Lwt_stream and append them -- > with pcap headers -- to the block device. The top function is > basically an experiment in making a 'file-like thing'. > > I started the capturing in the mirage-www main loop: > > https://github.com/djs55/mirage-www/blob/djs-blog/src/server.ml > > HTH, > Dave > >> >> >> On Tuesday, September 11, 2012, Anil Madhavapeddy wrote: >>> >>> That was my fault, sorry! This sort of thing should be fixed soon when >>> we can share C bindings more easily among the cross-compilation targets. >>> >>> -anil >>> >>> On 10 Sep 2012, at 22:31, David Scott wrote: >>> >>> > Right, problem solved: TCP checksum on xen was slightly broken. There >>> > was a fix made to Unix to cope with odd-length packets, this needed to >>> > be applied to xen as well. Many of the packets had valid checksums, >>> > and linux would ACK up to the sequence number in the last one of >>> > those. >>> > >>> > Now that the TCP bug is squashed, I can get back to writing my blog >>> > post -- all just a day in the life of a mirage hacker :-) >>> > >>> > On Sat, Sep 8, 2012 at 9:06 AM, Dave Scott >>> > wrote: >>> >> >>> >> >>> >> On Sep 7, 2012, at 5:59 PM, "Anil Madhavapeddy" >>> >> wrote: >>> >> >>> >>> On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott wrote: >>> >>>> Hi, >>> >>>> >>> >>>> I've built mirage-www for xen and modified the static IP to fit my >>> >>>> environment. (I tried DHCP first but this didn't work -- if I get the >>> >>>> time I'll try to debug). >>> >>>> >>> >>>> I can ping the server fine, and it's certainly receiving a lot of >>> >>>> traffic on my (probably fairly busy) local network. When I try to >>> >>>> fetch a URL the TCP connection hangs. On the console I get: >>> >>>> >>> >>>> Dispatch: dynamic URL / >>> >>>> ... irrelevant spam >>> >>>> TCP retransmission on timer seq = -889321980 >>> >>>> >>> >>>> I've attached a small tcpdump of the conversation. I started with >>> >>>> ping >>> >>>> and then tried HTTP. According to tcpdump/wireshark it went like >>> >>>> this: >>> >>>> >>> >>>> $ tcpdump -r mirage.pcap -n >>> >>>> reading from file mirage.pcap, link-type EN10MB (Ethernet) >>> >>>> 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >>> >>>> 9300, seq 1, length 64 >>> >>>> 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>> >>>> 9300, seq 1, length 64 >>> >>>> 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >>> >>>> 9300, seq 2, length 64 >>> >>>> 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>> >>>> 9300, seq 2, length 64 >>> >>>> 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >>> >>>> 9300, seq 3, length 64 >>> >>>> 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>> >>>> 9300, seq 3, length 64 >>> >>>> 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >>> >>>> 9300, seq 4, length 64 >>> >>>> 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>> >>>> 9300, seq 4, length 64 >>> >>>> 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo request, id >>> >>>> 9300, seq 5, length 64 >>> >>>> 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, id >>> >>>> 9300, seq 5, length 64 >>> >>>> >>> >>>> -- so far so good, this is just my initial pings. Switching to 'wget >>> >>>> http://10.80.239.140/' >>> >>>> >>> >>>> 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [S], >>> >>>> seq >>> >>>> 2284582709, win 5840, options [mss 1460,sackOK,TS val 909789846 ecr >>> >>>> 0,nop,wscale 6], length 0 >>> >>>> 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags [S.], >>> >>>> seq 3536243828, ack 2284582710, win 65535, options [mss 1380,wscale >>> >>>> 2,eol], length 0 >>> >>>> 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags [.], >>> >>>> ack >>> >>>> 1, win 92, length 0 >>> >>>> >>> >>>> -- TCP connection established >>> >>>> >>> >>>> 13: >> >> >> >> -- >> Dave Scott >> >> >> >> -- >> Cheers, >> >> R. >> >> >> >> >> >> This message and any attachment are intended solely for the addressee and >> may contain confidential information. If you have received this message in >> error, please send it back to me, and immediately delete it. Please do not >> use, copy or disclose the information contained in this message or in any >> attachment. Any views or opinions expressed by the author of this email do >> not necessarily reflect the views of the University of Nottingham. >> >> This message has been checked for viruses but the contents of an attachment >> may still contain software viruses which could damage your computer system: >> you are advised to perform your own checks. Email communications with the >> University of Nottingham may be monitored as permitted by UK legislation. > > > > -- > Dave Scott From anil@recoil.org Tue Sep 11 15:01:02 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBR1S-0004HO-NT (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 15:01:02 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1382695 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:26597 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1TBR1S-0003eF-pl (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 15:01:02 +0100 Received: (qmail 8689 invoked by uid 634); 11 Sep 2012 14:01:01 -0000 Received: from no-dns-yet.demon.co.uk (HELO [192.168.15.13]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 11 Sep 2012 15:01:01 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: starting alpha releases From: Anil Madhavapeddy In-Reply-To: <20120911131024.GA22854@caesar.elte.hu> Date: Tue, 11 Sep 2012 15:00:58 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20120911131024.GA22854@caesar.elte.hu> To: PALI Gabor Janos X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 14:01:02 -0000 X-Keywords: X-UID: 551 Status: O Content-Length: 2325 Lines: 64 On 11 Sep 2012, at 14:10, PALI Gabor Janos wrote: > Hi Anil, >=20 > On Tue, Sep 11, 2012 at 09:50:53AM +0100, Anil Madhavapeddy wrote: >> Then, we can start merging in all the outstanding features for the = first 'public' release, including: >>=20 >> - kFreeBSD backend (Gabor) >=20 > Is there an exact deadline for this? Nope; I'd like the kFreeBSD build bits to be integrated sooner rather = than later (to avoid divergence), but I expect it will take some time for the backend to be as stable as some of the other targets. =20 > I have not had the chance to put larger amount of stress on this = backend, > although it implements the mirage-os interfaces. I was pondering = perhaps > I could test this with the mirage web site application. I do not see = too > many obstacles except the eliminiation of use of floating-point = numbers > in the packages, but the devil is in the details. >=20 > Speaking of floating-point numbers: it is essentially a requirement to > switch from real-valued time representation of OCaml to microseconds, > otherwise we will not be able to merge the kFreeBSD backend. The Mirage website would be ideal, but we will certainly need to think harder about the floating point problem. Getting rid of all the = individual cases is too fragile (especially when using third party library code). Perhaps modifying the ocamlopt code generator to support soft-FP on x86 would appropriate. The specific case of the timer should be easy enough to modify, however. It's internally a uint64_t anyway. > On the other hand, the FreeBSD pfil(9) hook interface has just been = changed > to support for link-layer filtering [1] so we can stop piggybacking on > ng_ether(4). If I had some time I could adapt my code to use this. >=20 > Note that my current code has a build solution different from OPAM, so = it > would require some effort to make fit with it. I think this will be > another thing we should be aware of. Aww, I like netgraph! Fond memories of hacking on FreeBSD ATM with Dave = Scott back in 2003... I think we can merge the kFreeBSD BSD-Makefiles and OPAM reasonably = well. I've almost got OPAM working under *BSD [1], and once that is working we should be set to try a merge and get all of this into ports. [1] https://github.com/OCamlPro/opam/issues/132 -anil= From anil@recoil.org Tue Sep 11 15:14:39 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBREd-0004gI-U2 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 15:14:39 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1382695 * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:10196 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1TBREc-0002mU-rm (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 15:14:39 +0100 Received: (qmail 21977 invoked by uid 634); 11 Sep 2012 14:14:34 -0000 Received: from no-dns-yet.demon.co.uk (HELO [192.168.15.13]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 11 Sep 2012 15:14:34 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: mirage-www From: Anil Madhavapeddy In-Reply-To: Date: Tue, 11 Sep 2012 15:14:32 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <4465A853-DF9D-42D2-8E36-2CBD181DD018@recoil.org> References: <20120907165905.GA25178@dark.recoil.org> <5DF2CDEF-60A5-493D-903F-027A74A645B8@recoil.org> To: Balraj Singh X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Richard Mortier , Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 14:14:40 -0000 X-Keywords: X-UID: 552 Status: O Content-Length: 9485 Lines: 273 This corruption is *really* bad, but I can't reproduce it myself for some reason. We definitely need a little regression test that checks the body of a packet (our iperf tests don't do this at the moment). Corrupting the header on an outgoing packet implies that a Cstruct page is being free'd too early, and being reused being before flushed from the ring by the dom0 backend. It definitely wouldn't be fixed by Dave's most recent fix (which just calculates the checksum correctly for an odd number of bytes). -anil On 11 Sep 2012, at 14:56, Balraj Singh = wrote: > Sorry, I was offline for a few days. This was the exact bug that I > was working on before vacation/August/... >=20 >> =46rom the outside the main symptom was that packets were getting = lost. > This was compounded because of a bug that got introduced when I added > the last bits of window scaling - the bug prevented the sender from > recognising dup-acks and so fast rexmit was never triggered. That > meant that all rexmits happened on timer which really slowed things > down. But after that was fixed there was still a problem because > sometimes multiple packets in a window would get lost and since we > don't yet do SACK the second+ loss would still only be sent on timer. >=20 > So the real problem was pkt loss, which there should have been none of > in my test rig. It turned out that the packets weren't being lost, > they were just arriving with invalid tcp checksums. Unfortunately > this was not because the checksum was computed incorrectly, but > because the payload was damaged after the checksum computation. More > unfortunately, my retransmission code saves the payload after the > first transmission and if the pkt has to be retransmitted, it would > recompute the checksum. So if, as it happens now, the payload is > corrupted after the checksum is computed, the packet would correctly > be dropped by the receiver, but when the pkt is rexmited it would > recompute the checksum of the corrupted payload, which would result in > the packet being accepted and the whole stream getting corrupted! > This is the worst kind of TCP bug - badness factor 100 to me. >=20 > What the tcp code should do is compute the checksum as soon as it > receives the payload from the application and then only adjust the > chksum for any changes it needs to make on first or subsequent > transmissions. That would prevent the stream corruption which is at > the top of tcp's guarantees - it would just stop all transmission on > the stream but not let it get corrupted. >=20 > So, I was/am trying to figure out what was causing the corruption. It > has an interesting signature - most of the time the IP+TCP header of > that particular packet was being written over the first 40 bytes of > the payload. Dave, would the problem that you fixed in the xen tcp > checksum code cause this kind of corruption? If yes that would be a > relief. I'll try to get your fix and see if the corruption goes away > (prob do this tomorrow). >=20 > Thx, sorry about the saga. >=20 > Balraj >=20 >=20 >=20 > On Tue, Sep 11, 2012 at 9:36 AM, David Scott = wrote: >> On Tue, Sep 11, 2012 at 9:24 AM, Richard Mortier >> wrote: >>>=20 >>> On 11 Sep 2012, at 07:52, David Scott wrote: >>>=20 >>> Heh, no problem. While I was debugging this I did a bit of "yak = shaving" and >>> now have a nice bit of code which can capture packets to a block = device-- >>> this may be useful in future. >>>=20 >>>=20 >>> oh- nice. can you point me to it please? (I can sense some student = project >>> proposals coming on :) >>=20 >> This patch causes all incoming/outgoing ethernet frames to be >> 'captured' by which I mean added to a non-blocking Lwt_stream: >>=20 >> = https://github.com/djs55/mirage-net/commit/6bbb77f7af9f6d210b7af2f6b3b239f= 0272861d2 >>=20 >> Initially the stream size is set to 0, so all packets are dropped = i.e. >> no recording is actually done. >>=20 >> This small repo extracts the pcap example from ocaml-cstruct and adds >> some scaffolding around it: >>=20 >> https://github.com/djs55/ocaml-pcap >>=20 >> It's a bit minimal but could be easily expanded. This module: >>=20 >> https://github.com/djs55/ocaml-pcap/blob/master/mirage/pcap_mirage.ml >>=20 >> contains code to pull packets from the Lwt_stream and append them -- >> with pcap headers -- to the block device. The top function is >> basically an experiment in making a 'file-like thing'. >>=20 >> I started the capturing in the mirage-www main loop: >>=20 >> https://github.com/djs55/mirage-www/blob/djs-blog/src/server.ml >>=20 >> HTH, >> Dave >>=20 >>>=20 >>>=20 >>> On Tuesday, September 11, 2012, Anil Madhavapeddy wrote: >>>>=20 >>>> That was my fault, sorry! This sort of thing should be fixed soon = when >>>> we can share C bindings more easily among the cross-compilation = targets. >>>>=20 >>>> -anil >>>>=20 >>>> On 10 Sep 2012, at 22:31, David Scott wrote: >>>>=20 >>>>> Right, problem solved: TCP checksum on xen was slightly broken. = There >>>>> was a fix made to Unix to cope with odd-length packets, this = needed to >>>>> be applied to xen as well. Many of the packets had valid = checksums, >>>>> and linux would ACK up to the sequence number in the last one of >>>>> those. >>>>>=20 >>>>> Now that the TCP bug is squashed, I can get back to writing my = blog >>>>> post -- all just a day in the life of a mirage hacker :-) >>>>>=20 >>>>> On Sat, Sep 8, 2012 at 9:06 AM, Dave Scott = >>>>> wrote: >>>>>>=20 >>>>>>=20 >>>>>> On Sep 7, 2012, at 5:59 PM, "Anil Madhavapeddy" >>>>>> wrote: >>>>>>=20 >>>>>>> On Fri, Sep 07, 2012 at 02:02:45PM +0100, David Scott wrote: >>>>>>>> Hi, >>>>>>>>=20 >>>>>>>> I've built mirage-www for xen and modified the static IP to fit = my >>>>>>>> environment. (I tried DHCP first but this didn't work -- if I = get the >>>>>>>> time I'll try to debug). >>>>>>>>=20 >>>>>>>> I can ping the server fine, and it's certainly receiving a lot = of >>>>>>>> traffic on my (probably fairly busy) local network. When I try = to >>>>>>>> fetch a URL the TCP connection hangs. On the console I get: >>>>>>>>=20 >>>>>>>> Dispatch: dynamic URL / >>>>>>>> ... irrelevant spam >>>>>>>> TCP retransmission on timer seq =3D -889321980 >>>>>>>>=20 >>>>>>>> I've attached a small tcpdump of the conversation. I started = with >>>>>>>> ping >>>>>>>> and then tried HTTP. According to tcpdump/wireshark it went = like >>>>>>>> this: >>>>>>>>=20 >>>>>>>> $ tcpdump -r mirage.pcap -n >>>>>>>> reading from file mirage.pcap, link-type EN10MB (Ethernet) >>>>>>>> 13:47:07.543119 IP 10.80.2.32 > 10.80.239.140: ICMP echo = request, id >>>>>>>> 9300, seq 1, length 64 >>>>>>>> 13:47:07.543756 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, = id >>>>>>>> 9300, seq 1, length 64 >>>>>>>> 13:47:08.542112 IP 10.80.2.32 > 10.80.239.140: ICMP echo = request, id >>>>>>>> 9300, seq 2, length 64 >>>>>>>> 13:47:08.542422 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, = id >>>>>>>> 9300, seq 2, length 64 >>>>>>>> 13:47:09.541288 IP 10.80.2.32 > 10.80.239.140: ICMP echo = request, id >>>>>>>> 9300, seq 3, length 64 >>>>>>>> 13:47:09.541609 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, = id >>>>>>>> 9300, seq 3, length 64 >>>>>>>> 13:47:10.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo = request, id >>>>>>>> 9300, seq 4, length 64 >>>>>>>> 13:47:10.541580 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, = id >>>>>>>> 9300, seq 4, length 64 >>>>>>>> 13:47:11.541286 IP 10.80.2.32 > 10.80.239.140: ICMP echo = request, id >>>>>>>> 9300, seq 5, length 64 >>>>>>>> 13:47:11.541932 IP 10.80.239.140 > 10.80.2.32: ICMP echo reply, = id >>>>>>>> 9300, seq 5, length 64 >>>>>>>>=20 >>>>>>>> -- so far so good, this is just my initial pings. Switching to = 'wget >>>>>>>> http://10.80.239.140/' >>>>>>>>=20 >>>>>>>> 13:47:14.241216 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags = [S], >>>>>>>> seq >>>>>>>> 2284582709, win 5840, options [mss 1460,sackOK,TS val 909789846 = ecr >>>>>>>> 0,nop,wscale 6], length 0 >>>>>>>> 13:47:14.242365 IP 10.80.239.140.80 > 10.80.2.32.37158: Flags = [S.], >>>>>>>> seq 3536243828, ack 2284582710, win 65535, options [mss = 1380,wscale >>>>>>>> 2,eol], length 0 >>>>>>>> 13:47:14.242387 IP 10.80.2.32.37158 > 10.80.239.140.80: Flags = [.], >>>>>>>> ack >>>>>>>> 1, win 92, length 0 >>>>>>>>=20 >>>>>>>> -- TCP connection established >>>>>>>>=20 >>>>>>>> 13: >>>=20 >>>=20 >>>=20 >>> -- >>> Dave Scott >>>=20 >>>=20 >>>=20 >>> -- >>> Cheers, >>>=20 >>> R. >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>> This message and any attachment are intended solely for the = addressee and >>> may contain confidential information. If you have received this = message in >>> error, please send it back to me, and immediately delete it. Please = do not >>> use, copy or disclose the information contained in this message or = in any >>> attachment. Any views or opinions expressed by the author of this = email do >>> not necessarily reflect the views of the University of Nottingham. >>>=20 >>> This message has been checked for viruses but the contents of an = attachment >>> may still contain software viruses which could damage your computer = system: >>> you are advised to perform your own checks. Email communications = with the >>> University of Nottingham may be monitored as permitted by UK = legislation. >>=20 >>=20 >>=20 >> -- >> Dave Scott >=20 From pgj@caesar.elte.hu Tue Sep 11 20:38:31 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBWI3-0005Ia-UJ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 20:38:31 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.1 from SpamAssassin-3.3.2-1382695 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [157.181.151.9 listed in list.dnswl.dnsbl.ja.net] * -0.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mx2.mail.elte.hu ([157.181.151.9]:60459) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1TBWI3-0005Nc-Q7 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 11 Sep 2012 20:38:31 +0100 Received: from login01.caesar.elte.hu ([157.181.151.130]) by mx2.mail.elte.hu with esmtp (Exim) id 1TBWI0-0004l7-Dc from ; Tue, 11 Sep 2012 21:38:30 +0200 Received: (Authenticated sender: pgj) by login01.caesar.elte.hu with local (Exim 4.72) (envelope-from ) id 1TBWHz-00029T-Q7; Tue, 11 Sep 2012 21:38:27 +0200 Date: Tue, 11 Sep 2012 21:38:27 +0200 From: PALI Gabor Janos To: Anil Madhavapeddy Subject: Re: starting alpha releases Message-ID: <20120911193827.GA6153@caesar.elte.hu> References: <20120911131024.GA22854@caesar.elte.hu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Sender: Pali Gabor Janos X-ELTE-SpamScore: -2.0 X-ELTE-SpamLevel: X-ELTE-SpamCheck: no X-ELTE-SpamVersion: ELTE 2.0 X-ELTE-SpamCheck-Details: score=-2.0 required=5.9 tests=BAYES_00 autolearn=no SpamAssassin version=3.3.1 -2.0 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 19:38:32 -0000 X-Keywords: X-UID: 553 Status: O Content-Length: 1741 Lines: 46 On Tue, Sep 11, 2012 at 03:00:58PM +0100, Anil Madhavapeddy wrote: > I'd like the kFreeBSD build bits to be integrated sooner rather than > later [..], but I expect it will take some time for the > backend to be [..] stable Okay. > The Mirage website would be ideal, but we will certainly need to think > harder about the floating point problem. Getting rid of all the individual > cases is too fragile (especially when using third party library code). > Perhaps modifying the ocamlopt code generator to support soft-FP on x86 > would appropriate. I would try to avoid using any FP code because I think it is possible to do so. I have not met too many third-party packagesv only lwt, but it seemed to be without FP code. I will make the mirage website application build first to see what modifications are actually needed. > The specific case of the timer should be easy enough to modify, however. > It's internally a uint64_t anyway. That would be great! > Aww, I like netgraph! Fond memories of hacking on FreeBSD ATM with Dave Scott > back in 2003... :-) Well, actually "abuse" would be proper word for our current usage. We just steal the ng_ether(4) hook in ether_input() -- while the pfil(9) hooks provide us a natural way. Stealing the ng_ether(4) hook also means that it has to have an ether_output() hook, otherwise some of the assert() statements will complain and stop the kernel. So now I have an empty ether_output() hook just because of this. > I think we can merge the kFreeBSD BSD-Makefiles and OPAM reasonably well. Sounds good, I will look into this. > I've almost got OPAM working under *BSD [1], and once that is working we > should be set to try a merge and get all of this into ports. All right! From anil@recoil.org Wed Sep 12 01:57:28 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBbGg-000111-St (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 12 Sep 2012 01:57:27 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1382695 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:19656 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1TBbGf-0003mZ-Z9 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 12 Sep 2012 01:57:26 +0100 Received: (qmail 18641 invoked by uid 634); 12 Sep 2012 00:57:25 -0000 Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.2.2]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 12 Sep 2012 01:57:25 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: starting alpha releases From: Anil Madhavapeddy In-Reply-To: Date: Wed, 12 Sep 2012 01:57:22 +0100 Content-Transfer-Encoding: 7bit Message-Id: <3AC4F67C-33D2-4A6C-9DC8-3D7233D0B246@recoil.org> References: To: Mirage List X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 00:57:28 -0000 X-Keywords: X-UID: 554 Status: O Content-Length: 390 Lines: 16 On 11 Sep 2012, at 09:50, Anil Madhavapeddy wrote: > > - cohttp (Anil, will do today) > - github API (Anil, will do today) > > I propose that we also cut immediate alphas of: > > - mirage-platform > - mirage-net > - mirage-fs (completely missing the Block module, but it has mir-crunch) I've done releases of all the above now and send Thomas a pull request. -anil From Richard.Mortier@nottingham.ac.uk Wed Sep 12 09:37:39 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBiS3-0007OA-Om (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 12 Sep 2012 09:37:39 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1383293 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from ixe-mta-19-tx.emailfiltering.com ([194.116.198.150]:34898 helo=ixe-mta-19.emailfiltering.com) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1TBiRy-0005gJ-nV (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 12 Sep 2012 09:37:39 +0100 Received: from smtp3.nottingham.ac.uk ([128.243.44.55]) by ixe-mta-19.emailfiltering.com with emfmta (version 4.8.5.104) by TLS id 2367154016 ;8e457d9b1383f60e; Wed, 12 Sep 2012 09:37:14 +0100 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp3.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1TBiRd-0003pa-Uz for cl-mirage@lists.cam.ac.uk; Wed, 12 Sep 2012 09:37:13 +0100 Received: from EXCHANGE1.ad.nottingham.ac.uk ([fe80::7962:f868:e6ee:6267]) by UIWEXHUB02.ad.nottingham.ac.uk ([2002:80f3:f84::80f3:f84]) with mapi; Wed, 12 Sep 2012 09:37:00 +0100 From: Richard Mortier To: Anil Madhavapeddy Date: Wed, 12 Sep 2012 09:36:59 +0100 Subject: Re: starting alpha releases Thread-Topic: starting alpha releases Thread-Index: Ac2QwcZORqHNBg9SSkuamgaZb47I3w== Message-ID: <564E18FD-437B-42F5-A2BC-1074428F3DBF@nottingham.ac.uk> References: In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 08:37:39 -0000 X-Keywords: X-UID: 555 Status: O Content-Length: 1142 Lines: 32 On 11 Sep 2012, at 09:50, Anil Madhavapeddy wrote: > - dns (it needs a Cohttp-style Mirage+Lwt adapter. Mort, got time to try?= ) sorry- not before friday, no. i'll see if i can get time on mon/tue (when i= 'm also planning to finally do the bitstring vs cstruct comparison) else it= won't happen before the 24th either. --=20 Cheers, R. = This message and any attachment are intended solely for the addressee a= nd may contain confidential information. If you have received this mess= age in error, please send it back to me, and immediately delete it. P= lease do not use, copy or disclose the information contained in this me= ssage or in any attachment. Any views or opinions expressed by the aut= hor of this email do not necessarily reflect the views of the Universit= y of Nottingham.=0D=0A=0D=0AThis message has been checked for viruses b= ut the contents of an attachment=0D=0Amay still contain software viruse= s which could damage your computer system:=0D=0Ayou are advised to perf= orm your own checks. Email communications with the=0D=0AUniversity of N= ottingham may be monitored as permitted by UK legislation.= From thomas.gazagnaire@gmail.com Wed Sep 12 14:29:02 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBn01-0000x8-V0 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 12 Sep 2012 14:29:01 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.9 from SpamAssassin-3.3.2-1383293 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.173 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-wi0-f173.google.com ([209.85.212.173]:42117) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1TBn01-0003cL-nx (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 12 Sep 2012 14:29:01 +0100 Received: by wibhm6 with SMTP id hm6so4101557wib.2 for ; Wed, 12 Sep 2012 06:29:01 -0700 (PDT) Received: by 10.180.74.133 with SMTP id t5mr32892995wiv.2.1347456541329; Wed, 12 Sep 2012 06:29:01 -0700 (PDT) Received: from [10.0.1.191] ([188.182.230.77]) by mx.google.com with ESMTPS id do5sm12218786wib.10.2012.09.12.06.28.59 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 12 Sep 2012 06:29:00 -0700 (PDT) Subject: Re: starting alpha releases Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: <4AACC4B8-CBD5-4FB5-B6B6-46D07ED09D80@recoil.org> Date: Wed, 12 Sep 2012 15:28:57 +0200 Content-Transfer-Encoding: 7bit Message-Id: <36470A09-BFE0-4C9C-A657-31DDE6EFAB84@gmail.com> References: <6B7A8ACE-E03F-4D46-8A91-1FE86E6D0B1E@gmail.com> <4AACC4B8-CBD5-4FB5-B6B6-46D07ED09D80@recoil.org> To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1084) Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 13:29:02 -0000 X-Keywords: X-UID: 556 Status: O Content-Length: 215 Lines: 9 >>> - ORM (Thomas? Sqlite3 only for now, and probably forever) I've just added a new version of dyntype and orm to opam.ocamlpro.com. Shelf needs a major rewriting and I don't think it is worth it ... -- Thomas From anil@recoil.org Wed Sep 12 15:37:17 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TBo44-000363-WF (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 12 Sep 2012 15:37:17 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1383293 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:32639 helo=dark.recoil.org) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1TBo44-0002UN-o6 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 12 Sep 2012 15:37:16 +0100 Received: (qmail 533 invoked by uid 634); 12 Sep 2012 14:37:16 -0000 Received: from dab-bhx2-nat-blade-3-23.dab.02.net (HELO [10.42.55.222]) (82.132.233.77) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 12 Sep 2012 15:37:16 +0100 References: <6B7A8ACE-E03F-4D46-8A91-1FE86E6D0B1E@gmail.com> <4AACC4B8-CBD5-4FB5-B6B6-46D07ED09D80@recoil.org> <36470A09-BFE0-4C9C-A657-31DDE6EFAB84@gmail.com> In-Reply-To: <36470A09-BFE0-4C9C-A657-31DDE6EFAB84@gmail.com> Mime-Version: 1.0 (1.0) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Message-Id: X-Mailer: iPhone Mail (9B206) From: Anil Madhavapeddy Subject: Re: starting alpha releases Date: Wed, 12 Sep 2012 15:37:12 +0100 To: Thomas Gazagnaire X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 14:37:17 -0000 X-Keywords: X-UID: 557 Status: O Content-Length: 413 Lines: 20 Great! Yes, retiring Shelf for now seems fine. Cow is an able replacement...= Anil On 12 Sep 2012, at 14:28, Thomas Gazagnaire wr= ote: >>>> - ORM (Thomas? Sqlite3 only for now, and probably forever) >=20 > I've just added a new version of dyntype and orm to opam.ocamlpro.com. >=20 > Shelf needs a major rewriting and I don't think it is worth it ... >=20 > -- > Thomas >=20 From anil@recoil.org Thu Sep 13 15:11:05 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TCA8H-0000o7-Mj (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 13 Sep 2012 15:11:05 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1383849 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:45283 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1TCA8D-0005Em-Y6 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 13 Sep 2012 15:11:05 +0100 Received: (qmail 15499 invoked by uid 634); 13 Sep 2012 14:11:01 -0000 Received: from Unknown (HELO [10.0.8.153]) (87.54.61.21) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 13 Sep 2012 15:11:01 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: xen stub domains Message-Id: <147CA5EB-5652-4F5D-8F97-DD0567FF70F6@recoil.org> Date: Thu, 13 Sep 2012 16:10:59 +0200 To: Mirage List Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Sep 2012 14:11:05 -0000 X-Keywords: X-UID: 558 Status: O Content-Length: 224 Lines: 8 Dave has been cramming Mirage, Linux and Xen together to replace the = Xenstore service in dom0 with a stub domain equivalent. All the pieces = are coming together! http://openmirage.org/blog/xenstore-stub-domain -anil= From anil@recoil.org Tue Sep 18 18:11:19 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TE1KR-00000x-FX (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 18:11:19 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1386514 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0003] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:24548 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1TE1KQ-0006Vp-Z3 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 18:11:19 +0100 Received: (qmail 27136 invoked by uid 634); 18 Sep 2012 17:11:18 -0000 Received: from static-64-61-115-26.isp.broadviewnet.net (HELO [172.17.36.219]) (64.61.115.26) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 18 Sep 2012 18:11:18 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: OPAM tutorial Message-Id: <89F623EB-A836-434F-ACA0-22E416F67E2D@recoil.org> Date: Tue, 18 Sep 2012 13:11:15 -0400 To: Mirage List Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2012 17:11:19 -0000 X-Keywords: X-UID: 559 Status: O Content-Length: 354 Lines: 10 OPAM is getting stable and (by extension) Mirage is also getting easier = to use! Thomas and team have written a neat tutorial about it here: https://github.com/OCamlPro/opam/wiki/Creation-of-OPAM%C2%A0packages I'm flowing my way through all the mirage-www to make it consistent = again (before I give a talk in Princeton next week about it). -anil= From anil@recoil.org Tue Sep 18 19:23:12 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TE2S0-000163-58 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 19:23:12 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1386514 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:16506 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1TE2Rz-0003Mz-Qz (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 19:23:12 +0100 Received: (qmail 13538 invoked by uid 634); 18 Sep 2012 18:23:11 -0000 Received: from static-64-61-115-26.isp.broadviewnet.net (HELO [172.17.36.219]) (64.61.115.26) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 18 Sep 2012 19:23:11 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Fwd: [Caml-list] [ANN] Release 1.0.0 - A multi-process daemon framework for OCaml Date: Tue, 18 Sep 2012 14:23:06 -0400 References: <1347646999.13892.31.camel@andre> To: Mirage List , Andre Nathan Message-Id: <8255987E-93C7-40FA-AEE4-09062C8667C8@recoil.org> Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2012 18:23:12 -0000 X-Keywords: X-UID: 560 Status: O Content-Length: 2153 Lines: 64 This is a really useful Lwt-based 'inetd' for Lwt, which we could adapt to work with Mirage/Xen as well. Dave is looking into a more Erlang-style actor module (with process trees), so I packaged up=20 Release in OPAM and pushed it to the trunk.=20 Try it with "opam install release" if interested. -a Begin forwarded message: > From: Andre Nathan > Subject: [Caml-list] [ANN] Release 1.0.0 - A multi-process daemon = framework for OCaml > Date: 14 September 2012 14:23:19 EDT > To: caml-list@inria.fr >=20 > Hello >=20 > I'm happy to announce version 1.0.0 of the Release daemon > framework for OCaml: >=20 > Github repository: https://github.com/andrenth/release > Forge download: http://forge.ocamlcore.org/frs/?group_id=3D290 > ODB package page: http://oasis.ocamlcore.org/dev/view/release/1.0.0 >=20 > =46rom the README: >=20 > Release is a multi-process Lwt-enabled daemon framework for > OCaml, providing facilities for type-safe inter-process > communication and privilege-dropping. >=20 > Its goal is to make it easy to write servers that are > released from the calling terminal and to release root > privileges when those are not necessary. >=20 > Release provides the following features: >=20 > - Easy creation of 1-to-1 master-slave or 1-to-n master-slaves > process architectures; > - Type-safe and thread-safe interprocess communication; > - Supervisor features (the master process detects unexpected > deaths of its slaves and restarts them); > - Integer operations on buffers with support for 16, 32, 64 > and 128 bit integers with support for little- and big-endian > representations, plus a functorial interface allowing the > definition of operations on other integer sizes in the > release.bytes submodule; > - Generic configuration file parser with built-in validation > in the release.config submodule. >=20 > Best regards, > Andre >=20 >=20 >=20 > --=20 > Caml-list mailing list. Subscription management and archives: > https://sympa-roc.inria.fr/wws/info/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs >=20 From anil@recoil.org Tue Sep 18 20:00:01 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TE31d-0001k9-IW (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 20:00:01 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1386514 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:23784 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1TE31c-0002Jr-RK (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 20:00:01 +0100 Received: (qmail 13106 invoked by uid 634); 18 Sep 2012 18:59:59 -0000 Received: from static-64-61-115-26.isp.broadviewnet.net (HELO [172.17.36.219]) (64.61.115.26) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 18 Sep 2012 19:59:59 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: cstruct signature generator added Message-Id: Date: Tue, 18 Sep 2012 14:59:53 -0400 To: Mirage List Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2012 19:00:01 -0000 X-Keywords: X-UID: 561 Status: O Content-Length: 1423 Lines: 44 Dave has been using cstruct for his pcap generator, so I extended the = syntax extension to also work in .mli files. You can now do something = like: module LE : sig val endian : endian cstruct pcap_header { uint32_t magic_number; (* magic number *) uint16_t version_major; (* major version number *) uint16_t version_minor; (* minor version number *) uint32_t thiszone; (* GMT to local correction *) uint32_t sigfigs; (* accuracy of timestamps *) uint32_t snaplen; (* max length of captured packets, in = octets *) uint32_t network (* data link type *) } as little_endian cstruct pcap_packet { uint32_t ts_sec; (* timestamp seconds *) uint32_t ts_usec; (* timestamp microseconds *) uint32_t incl_len; (* number of octets of packet saved in file = *) uint32_t orig_len (* actual length of packet *) } as little_endian end ... just as you could before in a .ml file. I also switched the extension over to using a different quotation = expander (due to some rather obscure historical limitations in the = original vs revised syntaxes). This means the changes were a little = bigger than usual, so I'll wait a few days before releasing = cstruct-0.5.0 with the new generator.=20 Dave, pull request sent to you for djs55/ocaml-pcap... when it works, = lets tag a release on mirage/ocaml-pcap. -anil From scott.dj@gmail.com Tue Sep 18 22:43:16 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TE5Zc-0003WT-2H (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 22:43:16 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.9 from SpamAssassin-3.3.2-1386514 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.220.179 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-vc0-f179.google.com ([209.85.220.179]:61333) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1TE5Zb-0003gv-XK (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 22:43:15 +0100 Received: by vcbf13 with SMTP id f13so418806vcb.38 for ; Tue, 18 Sep 2012 14:43:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.221.11.71 with SMTP id pd7mr780510vcb.45.1348004594628; Tue, 18 Sep 2012 14:43:14 -0700 (PDT) Received: by 10.58.213.168 with HTTP; Tue, 18 Sep 2012 14:43:14 -0700 (PDT) Date: Tue, 18 Sep 2012 22:43:14 +0100 Message-ID: Subject: library for reading and writing pcap format files From: David Scott To: Mirage List Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2012 21:43:16 -0000 X-Keywords: X-UID: 562 Status: O Content-Length: 445 Lines: 18 Hi, I've added a simple ocaml pcap reading/writing library based on cstruct. It's available via opam ("opam install pcap" will fetch it if you have set up your opam to consume mirage packages) The tree is here: https://github.com/mirage/ocaml-pcap So far I've used it to extend a mirage xen www server with the ability to capture packets to a block device for later analysis. If you give it a go, let me know how you get on. Cheers, Dave From anil@recoil.org Tue Sep 18 22:44:49 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TE5b7-0003Xw-Ee (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 22:44:49 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1386514 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:46363 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1TE5b6-0001bp-R6 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 22:44:49 +0100 Received: (qmail 3616 invoked by uid 634); 18 Sep 2012 21:44:48 -0000 Received: from static-64-61-115-26.isp.broadviewnet.net (HELO [172.17.36.219]) (64.61.115.26) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 18 Sep 2012 22:44:48 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: library for reading and writing pcap format files From: Anil Madhavapeddy In-Reply-To: Date: Tue, 18 Sep 2012 17:44:43 -0400 Content-Transfer-Encoding: 7bit Message-Id: References: To: David Scott X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2012 21:44:49 -0000 X-Keywords: X-UID: 563 Status: O Content-Length: 742 Lines: 28 Nice! Minor cleanup: you left the mirage/mirage.mllib file in the repo (presumably, the Xen pcap consumers are elsewhere now, and this is just the core library and the UNIX version). -anil On 18 Sep 2012, at 17:43, David Scott wrote: > Hi, > > I've added a simple ocaml pcap reading/writing library based on > cstruct. It's available via opam ("opam install pcap" will fetch it if > you have set up your opam to consume mirage packages) > > The tree is here: > > https://github.com/mirage/ocaml-pcap > > So far I've used it to extend a mirage xen www server with the ability > to capture packets to a block device for later analysis. > > If you give it a go, let me know how you get on. > > Cheers, > Dave > From thomas.gazagnaire@gmail.com Tue Sep 18 22:46:08 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TE5cO-0003Ya-Ki (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 22:46:08 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.9 from SpamAssassin-3.3.2-1386514 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.173 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-wi0-f173.google.com ([209.85.212.173]:55330) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1TE5cM-0004AS-Y0 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 22:46:08 +0100 Received: by wibhm6 with SMTP id hm6so3774905wib.2 for ; Tue, 18 Sep 2012 14:46:06 -0700 (PDT) Received: by 10.216.201.156 with SMTP id b28mr599803weo.4.1348004766371; Tue, 18 Sep 2012 14:46:06 -0700 (PDT) Received: from [192.168.0.14] (gou06-3-88-170-165-56.fbx.proxad.net. [88.170.165.56]) by mx.google.com with ESMTPS id ct3sm24593755wib.5.2012.09.18.14.46.04 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 18 Sep 2012 14:46:05 -0700 (PDT) Subject: Re: library for reading and writing pcap format files Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: Date: Tue, 18 Sep 2012 23:46:03 +0200 Content-Transfer-Encoding: 7bit Message-Id: References: To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1084) Cc: Mirage List , David Scott X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2012 21:46:08 -0000 X-Keywords: X-UID: 564 Status: O Content-Length: 918 Lines: 38 And also feel free to pull-request to OCamlPro/opam-repository :-) -- Thomas On Sep 18, 2012, at 11:44 PM, Anil Madhavapeddy wrote: > Nice! Minor cleanup: you left the mirage/mirage.mllib file in the > repo (presumably, the Xen pcap consumers are elsewhere now, and this > is just the core library and the UNIX version). > > -anil > > On 18 Sep 2012, at 17:43, David Scott wrote: > >> Hi, >> >> I've added a simple ocaml pcap reading/writing library based on >> cstruct. It's available via opam ("opam install pcap" will fetch it if >> you have set up your opam to consume mirage packages) >> >> The tree is here: >> >> https://github.com/mirage/ocaml-pcap >> >> So far I've used it to extend a mirage xen www server with the ability >> to capture packets to a block device for later analysis. >> >> If you give it a go, let me know how you get on. >> >> Cheers, >> Dave >> > > From scott.dj@gmail.com Tue Sep 18 22:56:49 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TE5mj-0003ec-C7 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 22:56:49 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.2 from SpamAssassin-3.3.2-1386514 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.51 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 0.9 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-vb0-f51.google.com ([209.85.212.51]:45829) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1TE5mi-0003k8-Rr (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 18 Sep 2012 22:56:49 +0100 Received: by vbbfn1 with SMTP id fn1so417446vbb.38 for ; Tue, 18 Sep 2012 14:56:48 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.32.233 with SMTP id m9mr628200vdi.88.1348005407927; Tue, 18 Sep 2012 14:56:47 -0700 (PDT) Received: by 10.58.213.168 with HTTP; Tue, 18 Sep 2012 14:56:47 -0700 (PDT) In-Reply-To: References: Date: Tue, 18 Sep 2012 22:56:47 +0100 Message-ID: Subject: Re: library for reading and writing pcap format files From: David Scott To: Anil Madhavapeddy Content-Type: text/plain; charset=ISO-8859-1 Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2012 21:56:49 -0000 X-Keywords: X-UID: 565 Status: O Content-Length: 1086 Lines: 44 On Tue, Sep 18, 2012 at 10:44 PM, Anil Madhavapeddy wrote: > Nice! Minor cleanup: you left the mirage/mirage.mllib file in the > repo (presumably, the Xen pcap consumers are elsewhere now, and this > is just the core library and the UNIX version). Whoops, the freshly-minted 0.3.2 release should fix that. "opam update" and "opam install" should work (I hope) I'm going to move the mirage code elsewhere, once I've upstreamed the low-level network hooks. Cheers, Dave > > -anil > > On 18 Sep 2012, at 17:43, David Scott wrote: > >> Hi, >> >> I've added a simple ocaml pcap reading/writing library based on >> cstruct. It's available via opam ("opam install pcap" will fetch it if >> you have set up your opam to consume mirage packages) >> >> The tree is here: >> >> https://github.com/mirage/ocaml-pcap >> >> So far I've used it to extend a mirage xen www server with the ability >> to capture packets to a block device for later analysis. >> >> If you give it a go, let me know how you get on. >> >> Cheers, >> Dave >> > -- Dave Scott From andre@digirati.com.br Wed Sep 19 00:30:16 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TE7FA-0004SE-Oa (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 19 Sep 2012 00:30:16 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.2 from SpamAssassin-3.3.2-1386514 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature * -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no * trust * [187.73.32.184 listed in list.dnswl.dnsbl.ja.net] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mta112.f1.k8.com.br ([187.73.32.184]:46908) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1TE7F9-0006Yk-pN (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 19 Sep 2012 00:30:16 +0100 Received: from localhost (localhost [127.0.0.1]) by smtpz.f1.k8.com.br (Postfix) with ESMTP id 849E0802D9; Tue, 18 Sep 2012 23:30:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at k8.com.br Received: from smtpz.f1.k8.com.br ([127.0.0.1]) by localhost (mta112.f1.k8.com.br [127.0.0.1]) (amavisd-new, port 10024) with LMTP id SUJ5k7if1RqZ; Tue, 18 Sep 2012 23:30:12 +0000 (UTC) Received: from [192.168.1.5] (unknown [177.98.107.69]) by smtpz.f1.k8.com.br (Postfix) with ESMTPA id 61CFA802C1; Tue, 18 Sep 2012 23:30:10 +0000 (UTC) X-DKIM: OpenDKIM Filter v2.5.2 smtpz.f1.k8.com.br 61CFA802C1 Message-ID: <1348011010.5479.8.camel@dissident> Subject: Re: Fwd: [Caml-list] [ANN] Release 1.0.0 - A multi-process daemon framework for OCaml From: Andre Nathan To: Anil Madhavapeddy Date: Tue, 18 Sep 2012 20:30:10 -0300 In-Reply-To: <8255987E-93C7-40FA-AEE4-09062C8667C8@recoil.org> References: <1347646999.13892.31.camel@andre> <8255987E-93C7-40FA-AEE4-09062C8667C8@recoil.org> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.3-0ubuntu6 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2012 23:30:16 -0000 X-Keywords: X-UID: 566 Status: O Content-Length: 743 Lines: 20 On Tue, 2012-09-18 at 14:23 -0400, Anil Madhavapeddy wrote: > This is a really useful Lwt-based 'inetd' for Lwt, which we could > adapt to work with Mirage/Xen as well. Dave is looking into a more > Erlang-style actor module (with process trees), so I packaged up > Release in OPAM and pushed it to the trunk. Thanks for the interest in Release! Interestingly, I had never thought of it as an "inetd", although something like that could easily be built on top of Release. Currently Release features supervisor-style restarting of child processes but there's no support for process trees, only "process stars" (1 master process with N slaves) but if there's interest, this could be an interesting addition to the library. Cheers, Andre From h.rotsos@gmail.com Wed Sep 19 21:37:19 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TER1K-0003Cz-W6 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 19 Sep 2012 21:37:19 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.2 from SpamAssassin-3.3.2-1387055 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.160.51 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (h.rotsos[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 0.9 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-pb0-f51.google.com ([209.85.160.51]:42812) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1TER1J-00082T-sd (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 19 Sep 2012 21:37:18 +0100 Received: by pbbro8 with SMTP id ro8so3384621pbb.38 for ; Wed, 19 Sep 2012 13:37:16 -0700 (PDT) Received: by 10.68.239.5 with SMTP id vo5mr1035732pbc.102.1348087036934; Wed, 19 Sep 2012 13:37:16 -0700 (PDT) Received: from [10.255.241.224] ([192.5.67.11]) by mx.google.com with ESMTPS id to6sm2347588pbc.12.2012.09.19.13.37.15 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 19 Sep 2012 13:37:15 -0700 (PDT) From: Charalampos Rotsos Content-Type: multipart/alternative; boundary="Apple-Mail=_51463F5E-2E15-4D6C-B912-226DF62004AE" Subject: various questions Message-Id: <46CE6ED5-B8FB-432A-B961-69879B276D00@gmail.com> Date: Wed, 19 Sep 2012 13:37:14 -0700 To: cl-mirage@lists.cam.ac.uk Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Sep 2012 20:37:19 -0000 X-Keywords: X-UID: 567 Status: O Content-Length: 5581 Lines: 114 --Apple-Mail=_51463F5E-2E15-4D6C-B912-226DF62004AE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi all,=20 As a quick intro, over summer I have been working on getting mirage to = work as a network simulation and emulation platform. I have developed so = far an ns3 backend tha allow to run applications as event driven = simulations and a mechanism to bootstrap distributed xen-based emulation = using the xapi interface. Also, because this work was focused to develop = a tool that allow to understand the impact of sdn network architectures, = I have develop a number of openflow and traffic generation libraries. So = far the work is done on the old mirage tree and I will try to port my = work into the main mirage-platform tools in the near future.=20 I am currently trying to get a first cut of the xen emulation task and I = have a couple of question over various things:=20 1) In order to install the xapi libraries, I need to install a number of = libraries, that they are different from the libraries distributed by = opam. The problem is that some of the libraries use some very weird = installation mechanisms. The xen people have solved the problem by = distributed a hacked version of these libraries through = https://github.com/xen-org/xen-api-libs/. How can I introduce an opam = recipe in the opam repo that will notify the installation mechanism that = the libraries are available through this repo and override the default = recipes for the specific libraries? I tries also to rewrite the default = recipes for some libraries but I got stuck on finding a way to generate = a custom META file and I gave up.=20 2) For the ocaml-pcap library is there some example code and the device = configurations I need in order to get it working as a network capture = mechanism for a xen vm?=20 3) I have written some code that allows to generate network topologies = and boot vm over xapi and I have tested and verified the code on a = debian vm. This unfortunately is constrained on testing the network = throttling configurations. Is there any xapi configured machine I could = get access to to verify the runtime correctnesss of the code?=20 thanks in advance for any replies.=20 --=20 Charalampos Rotsos PhD student The University of Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD Phone: +44-(0) 1223 767032 Email: cr409@cl.cam.ac.uk --Apple-Mail=_51463F5E-2E15-4D6C-B912-226DF62004AE Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Hi = all, 

As a quick intro, over  summer I have = been working on getting mirage to work as a network simulation and = emulation platform. I have developed so far an ns3 backend tha allow to = run applications as event driven simulations and a mechanism to = bootstrap distributed xen-based emulation using the xapi interface. = Also, because this work was focused to develop a tool that allow to = understand the impact of sdn network architectures, I have develop a = number of openflow and traffic generation libraries. So far the work is = done on the old mirage tree and I will try to port my work into the main = mirage-platform tools in the near = future. 

I am currently trying to get a = first cut of the xen emulation task and I have a couple of question over = various things: 

1) In order to install = the xapi libraries, I need to install a number of libraries, that they = are different from the libraries distributed by opam. The problem is = that some of the libraries use some very weird installation mechanisms. = The xen people have solved the problem by distributed a hacked version = of these libraries through https://github.com/xen-o= rg/xen-api-libs/. How can I introduce an opam recipe in the opam = repo that will notify the installation mechanism that the libraries are = available through this repo and override the default recipes for the = specific libraries? I tries also to rewrite the default recipes for some = libraries but I got stuck on finding a way to generate a custom META = file and I gave up. 

2) For the ocaml-pcap = library is there some example code and the device configurations I need = in order to get it working as a network capture mechanism for a xen = vm? 

3) I have written some code that = allows to generate network topologies and boot vm over xapi and I have = tested and verified the code on a debian vm. This unfortunately is = constrained on testing the network throttling configurations. Is there = any xapi configured machine I could get access to to verify the runtime = correctnesss of the code? 

thanks in = advance for any replies. 

-- 
Charalampos Rotsos
PhD student
The University of = Cambridge
Computer Laboratory
William Gates Building
JJ Thomson = Avenue
Cambridge
CB3 0FD

Phone: +44-(0) 1223 = 767032
Email: cr409@cl.cam.ac.uk


= --Apple-Mail=_51463F5E-2E15-4D6C-B912-226DF62004AE-- From Dave.Scott@eu.citrix.com Thu Sep 20 15:04:16 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TEhMW-0002RZ-KV (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 15:04:16 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.9 from SpamAssassin-3.3.2-1387490 * -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, * medium trust * [62.200.22.115 listed in list.dnswl.dnsbl.ja.net] * -0.5 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from smtp.ctxuk.citrix.com ([62.200.22.115]:14811 helo=SMTP.EU.CITRIX.COM) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1TEhMV-00065G-pH (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 15:04:16 +0100 X-IronPort-AV: E=Sophos;i="4.80,453,1344211200"; d="scan'208";a="14659433" Received: from lonpmailmx01.citrite.net ([10.30.203.162]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/RC4-MD5; 20 Sep 2012 14:03:33 +0000 Received: from LONPMAILBOX01.citrite.net ([10.30.224.160]) by LONPMAILMX01.citrite.net ([10.30.203.162]) with mapi; Thu, 20 Sep 2012 15:03:34 +0100 From: Dave Scott To: 'Charalampos Rotsos' , "cl-mirage@lists.cam.ac.uk" Date: Thu, 20 Sep 2012 15:03:32 +0100 Subject: RE: various questions Thread-Topic: various questions Thread-Index: Ac2WppLLR9hYIvvYQ0maFjGKOL7PywAkC4ug Message-ID: <81A73678E76EA642801C8F2E4823AD21011A6DCCE15B@LONPMAILBOX01.citrite.net> References: <46CE6ED5-B8FB-432A-B961-69879B276D00@gmail.com> In-Reply-To: <46CE6ED5-B8FB-432A-B961-69879B276D00@gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "'xen-api@lists.xen.org'" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2012 14:04:16 -0000 X-Keywords: X-UID: 568 Status: O Content-Length: 3435 Lines: 87 Hi, [cc:d xen-api@lists.xen.org where xen-api/xen-api-libs are developed] Charalampos Rotsos wrote: > Hi all,=A0 > As a quick intro, over =A0summer I have been working on getting mirage > to work as a network simulation and emulation platform. I have developed > so far an ns3 backend tha allow to run applications as event driven > simulations and a mechanism to bootstrap distributed xen-based emulation > using the xapi interface. Also, because this work was focused to develop > a tool that allow to understand the impact of sdn network architectures, > I have develop a number of openflow and traffic generation libraries. So > far the work is done on the old mirage tree and I will try to port my > work into the main mirage-platform tools in the near future.=A0 This sounds really interesting! > I am currently trying to get a first cut of the xen emulation task and I > have a couple of question over various things:=A0 > 1) In order to install the xapi libraries, I need to install a number of > libraries, that they are different from the libraries distributed by opam= . > The problem is that some of the libraries use some very weird installatio= n > mechanisms. The xen people have solved the problem by distributed a hacke= d > version of these libraries through https://github.com/xen-org/xen-api-lib= s/. > How can I introduce an opam recipe in the opam repo that will notify the > installation mechanism that the libraries are available through this repo > and override the default recipes for the specific libraries? I tries also > to rewrite the default recipes for some libraries but I got stuck on > finding a way to generate a custom META file and I gave up.=A0 The long-term plan for xen-api-libs is to use the upstream packages instead of entirely custom ones. For example, now that "ocaml-rpc" is available through opam I'd like to remove the custom fork "xen-api-libs/rpc-light". Some of the code in "xen-api-libs" is a (not very high quality IMHO) custom "standard library". I think we should probably switch to a better alternati= ve, such as (Jane Street) Core. However in the short term we could probably jus= t opam-package what we have. Which particular packages are you having trouble with? I could take a quick look and see if I could help. Also, if you're running debian you might be able to just use "apt" to fetch the build dependencies of xapi in your opam "system" compiler. > 2) For the ocaml-pcap library is there some example code and the device > configurations I need in order to get it working as a network capture > mechanism for a xen vm?=A0 The extra pieces are: 1. mirage-net netif support for capture https://github.com/mirage/mirage-net/pull/5 (I just made the pull request) 2. a bit of glue to read the captured frames and write them to a block device. I'm half-way through a bit of refactoring -- I'll send you a link when I'm done. > 3) I have written some code that allows to generate network topologies an= d > boot vm over xapi and I have tested and verified the code on a debian vm. > This unfortunately is constrained on testing the network throttling > configurations. Is there any xapi configured machine I could get access t= o > to verify the runtime correctnesss of the code?=A0 When you say "network throttling" do you mean netback VIF rate limiting? If so there is a bug against the xen-api master branch and a fix in a pull request. Cheers, Dave From anil@recoil.org Thu Sep 20 16:44:09 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TEivB-0005zz-3P (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 16:44:09 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387490 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:14325 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1TEivA-0003iI-RF (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 16:44:09 +0100 Received: (qmail 26026 invoked by uid 634); 20 Sep 2012 15:44:08 -0000 Received: from c-68-36-200-153.hsd1.nj.comcast.net (HELO [192.168.1.16]) (68.36.200.153) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 20 Sep 2012 16:44:08 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: various questions From: Anil Madhavapeddy In-Reply-To: <81A73678E76EA642801C8F2E4823AD21011A6DCCE15B@LONPMAILBOX01.citrite.net> Date: Thu, 20 Sep 2012 11:44:06 -0400 Content-Transfer-Encoding: 7bit Message-Id: <951DEF21-B12C-4301-8FFF-1220C4D4DA4C@recoil.org> References: <46CE6ED5-B8FB-432A-B961-69879B276D00@gmail.com> <81A73678E76EA642801C8F2E4823AD21011A6DCCE15B@LONPMAILBOX01.citrite.net> To: Dave Scott X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "'xen-api@lists.xen.org'" , "cl-mirage@lists.cam.ac.uk" , 'Charalampos Rotsos' X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2012 15:44:09 -0000 X-Keywords: X-UID: 569 Status: O Content-Length: 831 Lines: 25 On 20 Sep 2012, at 10:03, Dave Scott wrote: >> 2) For the ocaml-pcap library is there some example code and the device >> configurations I need in order to get it working as a network capture >> mechanism for a xen vm? > > The extra pieces are: > > 1. mirage-net netif support for capture > > https://github.com/mirage/mirage-net/pull/5 > > (I just made the pull request) > > 2. a bit of glue to read the captured frames and write them to a block > device. I'm half-way through a bit of refactoring -- I'll send you a link > when I'm done. This conflicts slightly with some changes we've made to expose promiscuous interfaces from Mirage-net directly. Would it better to put the Mirage pcap code back into ocaml-pcap (beside the Lwt code), now that those functions are exposed properly? -anil From anil@recoil.org Thu Sep 20 16:45:27 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TEiwR-00061P-36 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 16:45:27 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387490 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:25023 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1TEiwP-0004Az-T1 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 16:45:26 +0100 Received: (qmail 9936 invoked by uid 634); 20 Sep 2012 15:45:25 -0000 Received: from c-68-36-200-153.hsd1.nj.comcast.net (HELO [192.168.1.16]) (68.36.200.153) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 20 Sep 2012 16:45:25 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: various questions From: Anil Madhavapeddy In-Reply-To: <81A73678E76EA642801C8F2E4823AD21011A6DCCE15B@LONPMAILBOX01.citrite.net> Date: Thu, 20 Sep 2012 11:45:23 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <46CE6ED5-B8FB-432A-B961-69879B276D00@gmail.com> <81A73678E76EA642801C8F2E4823AD21011A6DCCE15B@LONPMAILBOX01.citrite.net> To: Dave Scott X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "'xen-api@lists.xen.org'" , "cl-mirage@lists.cam.ac.uk" , 'Charalampos Rotsos' X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2012 15:45:27 -0000 X-Keywords: X-UID: 570 Status: O Content-Length: 1888 Lines: 54 On 20 Sep 2012, at 10:03, Dave Scott wrote: >> 1) In order to install the xapi libraries, I need to install a number = of >> libraries, that they are different from the libraries distributed by = opam. >> The problem is that some of the libraries use some very weird = installation >> mechanisms. The xen people have solved the problem by distributed a = hacked >> version of these libraries through = https://github.com/xen-org/xen-api-libs/. >> How can I introduce an opam recipe in the opam repo that will notify = the >> installation mechanism that the libraries are available through this = repo >> and override the default recipes for the specific libraries? I tries = also >> to rewrite the default recipes for some libraries but I got stuck on >> finding a way to generate a custom META file and I gave up.=20 >=20 > The long-term plan for xen-api-libs is to use the upstream packages = instead > of entirely custom ones. For example, now that "ocaml-rpc" is = available > through opam I'd like to remove the custom fork = "xen-api-libs/rpc-light". >=20 > Some of the code in "xen-api-libs" is a (not very high quality IMHO) = custom > "standard library". I think we should probably switch to a better = alternative, > such as (Jane Street) Core. However in the short term we could = probably just > opam-package what we have. >=20 > Which particular packages are you having trouble with? I could take a = quick > look and see if I could help. >=20 > Also, if you're running debian you might be able to just use "apt" to = fetch > the build dependencies of xapi in your opam "system" compiler. I think that Haris only needs all this stuff in order to compile the = client bindings, and hopefully the other thread on xen-api-client will be all = he needs to integrate his simulator with a real XCP installation. Correct = me if wrong, Haris! -anil= From anil@recoil.org Thu Sep 20 16:45:58 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TEiww-00062A-Lk (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 16:45:58 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387490 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:46999 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1TEiwv-0004Kc-SP (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 16:45:58 +0100 Received: (qmail 19585 invoked by uid 634); 20 Sep 2012 15:45:57 -0000 Received: from c-68-36-200-153.hsd1.nj.comcast.net (HELO [192.168.1.16]) (68.36.200.153) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 20 Sep 2012 16:45:57 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Fwd: [Xen-API] ocaml bindings to xenapi Date: Thu, 20 Sep 2012 11:45:57 -0400 References: <7503FC10-034F-4E0D-A22F-AA524A351714@recoil.org> To: "cl-mirage@lists.cam.ac.uk List" Message-Id: <1A49D2FB-D688-479C-AAFE-E0516D7228AD@recoil.org> Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2012 15:45:58 -0000 X-Keywords: X-UID: 571 Status: O Content-Length: 2491 Lines: 84 The other xen-api thread, which wasnt cross-posted. Begin forwarded message: > From: Anil Madhavapeddy > Subject: Re: [Xen-API] ocaml bindings to xenapi > Date: 20 September 2012 11:42:45 EDT > To: Dave Scott > Cc: Haris Rotsos , "xen-api@lists.xen.org" = >=20 > On 20 Sep 2012, at 10:12, Dave Scott wrote: >=20 >>=20 >> Anil wrote: >>> Hi, we'd like to use the OCaml bindings to the XAPI XML-RPC layer in = a >>> research project. Is there any way to get these separately of XAPI >>> itself? >>>=20 >>> We just need a simple client library (analogous to XenAPI.py) rather >>> than the whole lot. I can rip the bindings out of XAPI if there's = no >>> other solution... >>=20 >> I think you're in luck! Jon (cc:d) has started the process of = extracting >> the bindings from the main repo. >>=20 >> https://github.com/jonludlam/xen-api-client >>=20 >> It's a start :-) but it needs a bit of a tidy up. We need to = ruthlessly >> rip out any server-related code which may have accidentally been = copied >> across. >>=20 >> Jon has already functorised it so it should work with Lwt, Async and >> regular threads. >>=20 >=20 > Awesome! I've started OASIS'ing it up in my fork: > https://github.com/avsm/xen-api-client >=20 > I haven't quite figured out the runes to run it optionally through = pa_js > yet, but need to do that anyway. >=20 > Looks like doing a Cohttp_lwt_unix version should be very = straightforward, > but I need a XAPI box to try it on. I guess a Fusion installation is = the way > to go here, unless there is a fakeserver that still works? >=20 >> Looking at it I don't think Jon's included the actual code generator = (yet) >> but that's clearly one of the next steps. >>=20 >> So I reckon we need to: >> 1. oasis it up >> 2. add some test cases (using either lwt or async?) >> 3. add an interesting example (could use the xen-api/ocaml/graph = one?) >> 4. declare version 0.1 and add to opam >> 5. bask in the awesomeness of utop with tab-completion + xen-api >>=20 >> What do you think? >>=20 >> BTW this version of the bindings targets the xapi in the master = branch, >> but for basic stuff there should be no problem talking to older = releases. >=20 > All sounds good! >=20 > -a >=20 > _______________________________________________ > Xen-api mailing list > Xen-api@lists.xen.org > http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api >=20 From anil@recoil.org Thu Sep 20 21:36:38 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TEnUE-00033L-B9 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 21:36:38 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387490 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:34757 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1TEnUC-0000L1-ZQ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 21:36:38 +0100 Received: (qmail 325 invoked by uid 634); 20 Sep 2012 20:36:36 -0000 Received: from 171.sub-70-192-132.myvzw.com (HELO [192.168.1.16]) (70.192.132.171) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 20 Sep 2012 21:36:36 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: crypto-keys and cmdinfo Message-Id: <0E7CAED0-88C7-4221-ACBE-1D69DD19CB5C@recoil.org> Date: Thu, 20 Sep 2012 16:36:32 -0400 To: "cl-signpost@lists.cam.ac.uk" , "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2012 20:36:38 -0000 X-Keywords: X-UID: 572 Status: O Content-Length: 901 Lines: 24 I've started porting Haris' excellent crypto_keys library to be suitable = for release. There is proper command-line parsing and man page = generation now, thanks to this amazing library: http://erratique.ch/software/cmdliner/doc/Cmdliner.html Anyone wants to try it, install OPAM from = http://github.com/OCamlPro/opam, and add the Mirage dev repository: $ eval `opam config -env` $ opam remote -add dev git://github.com/mirage/opam-repo-dev $ opam install crypto_keys This doesnt actually install the binaries yet, but it gets all the = dependencies in place, so you can clone mirage/ocaml-crypto-keys and = have it compile very easily, for a change! Haris, I've got the DNSSEC-to-SSH converter working, but it looks you = are currently resolving DS records directly into SSH public keys, right? = Should we also look for the SSHFP and use those in preference to DS, if = specified? -anil= From scott.dj@gmail.com Thu Sep 20 21:54:04 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TEnl6-0003Cl-I7 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 21:54:04 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.2 from SpamAssassin-3.3.2-1387490 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.51 listed in list.dnswl.dnsbl.ja.net] * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 0.9 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-vb0-f51.google.com ([209.85.212.51]:56101) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1TEnl5-0000ZQ-pY (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 21:54:04 +0100 Received: by vbbfn1 with SMTP id fn1so2894641vbb.38 for ; Thu, 20 Sep 2012 13:54:03 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.116.175 with SMTP id jx15mr1819283veb.6.1348174443280; Thu, 20 Sep 2012 13:54:03 -0700 (PDT) Received: by 10.58.213.168 with HTTP; Thu, 20 Sep 2012 13:54:03 -0700 (PDT) Date: Thu, 20 Sep 2012 21:54:03 +0100 Message-ID: Subject: mirage xen pcap example From: David Scott To: Mirage List Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2012 20:54:04 -0000 X-Keywords: X-UID: 573 Status: O Content-Length: 665 Lines: 29 Hi, I did a bit of refactoring to the pcap stuff. It now consists of the following pieces: 1. The master branch of mirage-net https://github.com/mirage/mirage-net has an extended "promiscuous hook" 2. The master branch of ocaml-pcap https://github.com/mirage/ocaml-pcap has a library "pcap.mirage" There's an example showing how to turn it on in my branch of mirage-www: https://github.com/djs55/mirage-www/tree/pcap It will start appending packets to its first disk (disk 0 =/= 51712 =/= 'xvda') I added "make run" which invokes "mir-run" and "xl", but it would also be easy with xapi -- just make sure there's a disk for it to write to. Cheers, Dave From anil@recoil.org Thu Sep 20 22:37:16 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TEoQt-0003Vw-Vo (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 22:37:16 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387490 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:1530 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1TEoQt-0003K6-X9 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 22:37:15 +0100 Received: (qmail 31477 invoked by uid 634); 20 Sep 2012 21:37:14 -0000 Received: from 32.sub-70-192-134.myvzw.com (HELO [192.168.1.16]) (70.192.134.32) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 20 Sep 2012 22:37:14 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: mirage xen pcap example From: Anil Madhavapeddy In-Reply-To: Date: Thu, 20 Sep 2012 17:37:10 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <742FFBD7-A9F2-4009-997F-48886E74B277@recoil.org> References: To: David Scott X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2012 21:37:16 -0000 X-Keywords: X-UID: 574 Status: O Content-Length: 1104 Lines: 41 On 20 Sep 2012, at 16:54, David Scott wrote: > Hi, >=20 > I did a bit of refactoring to the pcap stuff. It now consists of the > following pieces: >=20 > 1. The master branch of mirage-net >=20 > https://github.com/mirage/mirage-net >=20 > has an extended "promiscuous hook" >=20 > 2. The master branch of ocaml-pcap >=20 > https://github.com/mirage/ocaml-pcap >=20 > has a library "pcap.mirage" >=20 > There's an example showing how to turn it on in my branch of = mirage-www: >=20 > https://github.com/djs55/mirage-www/tree/pcap >=20 > It will start appending packets to its first disk (disk 0 =3D/=3D = 51712 =3D/=3D 'xvda') >=20 > I added "make run" which invokes "mir-run" and "xl", but it would also > be easy with xapi -- just make sure there's a disk for it to write to. This will make debugging those TCP bugs hello easier. Thanks! To be clear, it will dump the raw pcap to the block device as-is, and = not using the FAT32 library, right? So it's best to make sure the first disk is a = blktap file, which will make it a valid pcap file to use from dom0... ? -anil= From scott.dj@gmail.com Thu Sep 20 22:45:59 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TEoZL-0003b9-UD (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 22:45:59 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.9 from SpamAssassin-3.3.2-1387490 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.220.179 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-vc0-f179.google.com ([209.85.220.179]:62813) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1TEoZK-0004qD-YO (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 22:45:59 +0100 Received: by vcbf13 with SMTP id f13so3014288vcb.38 for ; Thu, 20 Sep 2012 14:45:58 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.173.74 with SMTP id bi10mr1488503vdc.81.1348177557952; Thu, 20 Sep 2012 14:45:57 -0700 (PDT) Received: by 10.58.213.168 with HTTP; Thu, 20 Sep 2012 14:45:57 -0700 (PDT) In-Reply-To: <742FFBD7-A9F2-4009-997F-48886E74B277@recoil.org> References: <742FFBD7-A9F2-4009-997F-48886E74B277@recoil.org> Date: Thu, 20 Sep 2012 22:45:57 +0100 Message-ID: Subject: Re: mirage xen pcap example From: David Scott To: Anil Madhavapeddy Content-Type: text/plain; charset=ISO-8859-1 Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2012 21:46:00 -0000 X-Keywords: X-UID: 575 Status: O Content-Length: 1494 Lines: 48 On Thu, Sep 20, 2012 at 10:37 PM, Anil Madhavapeddy wrote: > On 20 Sep 2012, at 16:54, David Scott wrote: > >> Hi, >> >> I did a bit of refactoring to the pcap stuff. It now consists of the >> following pieces: >> >> 1. The master branch of mirage-net >> >> https://github.com/mirage/mirage-net >> >> has an extended "promiscuous hook" >> >> 2. The master branch of ocaml-pcap >> >> https://github.com/mirage/ocaml-pcap >> >> has a library "pcap.mirage" >> >> There's an example showing how to turn it on in my branch of mirage-www: >> >> https://github.com/djs55/mirage-www/tree/pcap >> >> It will start appending packets to its first disk (disk 0 =/= 51712 =/= 'xvda') >> >> I added "make run" which invokes "mir-run" and "xl", but it would also >> be easy with xapi -- just make sure there's a disk for it to write to. > > > This will make debugging those TCP bugs hello easier. Thanks! > > To be clear, it will dump the raw pcap to the block device as-is, and not using > the FAT32 library, right? So it's best to make sure the first disk is a blktap > file, which will make it a valid pcap file to use from dom0... ? Yup, it dumps to a raw block device. The 'make run' in the mirage-www branch will create a temporary file by dd'ing 128M of zeroes. If you tell tcpdump to read that then it will work, although it will get a bit confused by all the zeroes at the end. It works well enough to diagnose problems anyway! Cheers, Dave -- Dave Scott From anil@recoil.org Thu Sep 20 22:50:51 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TEoe3-0003eQ-UO (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 22:50:51 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387490 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:13037 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1TEoe3-0005fh-Xj (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 20 Sep 2012 22:50:51 +0100 Received: (qmail 15396 invoked by uid 634); 20 Sep 2012 21:50:51 -0000 Received: from 32.sub-70-192-134.myvzw.com (HELO [192.168.1.16]) (70.192.134.32) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 20 Sep 2012 22:50:51 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: mirage xen pcap example From: Anil Madhavapeddy In-Reply-To: Date: Thu, 20 Sep 2012 17:50:47 -0400 Content-Transfer-Encoding: 7bit Message-Id: <79634577-E4BF-43BB-894E-ACD00553504C@recoil.org> References: <742FFBD7-A9F2-4009-997F-48886E74B277@recoil.org> To: David Scott X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2012 21:50:52 -0000 X-Keywords: X-UID: 576 Status: O Content-Length: 776 Lines: 18 On 20 Sep 2012, at 17:45, David Scott wrote: > > Yup, it dumps to a raw block device. The 'make run' in the mirage-www > branch will create a temporary file by dd'ing 128M of zeroes. If you > tell tcpdump to read that then it will work, although it will get a > bit confused by all the zeroes at the end. It works well enough to > diagnose problems anyway! Speaking of mir-run... it has disappeared in the disaggregated Mirage. Should I reimport it into mirage-platform? The problem of running bare kernels still exists, of course... There used to be an XCP-compliant one, and we need some way to wrap the EC2 scripts more automatically. What was that alternative to this that you mentioned to me in OUD? Something to do with Openstack (!) -anil From h.rotsos@gmail.com Fri Sep 21 07:33:50 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TEwoA-0000Ox-K2 (Exim 4.70) (return-path ); Fri, 21 Sep 2012 07:33:50 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1387911 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.179 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (h.rotsos[at]gmail.com) * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-wi0-f179.google.com ([209.85.212.179]:36314) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1TEwo9-0004YQ-RZ (Exim 4.72) (return-path ); Fri, 21 Sep 2012 07:33:50 +0100 Received: by wibhq4 with SMTP id hq4so1264861wib.2 for ; Thu, 20 Sep 2012 23:33:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.41.195 with SMTP id h45mr2531640web.74.1348209229224; Thu, 20 Sep 2012 23:33:49 -0700 (PDT) Sender: h.rotsos@gmail.com Received: by 10.223.165.137 with HTTP; Thu, 20 Sep 2012 23:33:49 -0700 (PDT) In-Reply-To: <0E7CAED0-88C7-4221-ACBE-1D69DD19CB5C@recoil.org> References: <0E7CAED0-88C7-4221-ACBE-1D69DD19CB5C@recoil.org> Date: Thu, 20 Sep 2012 23:33:49 -0700 X-Google-Sender-Auth: mm6efUn3EsWVyrh9sy6Xxoy2yJU Message-ID: Subject: Re: crypto-keys and cmdinfo From: Haris Rotsos To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=e0cb4e6ff5b130d9c904ca306cc1 Cc: "cl-signpost@lists.cam.ac.uk" , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 06:33:50 -0000 X-Keywords: X-UID: 577 Status: O Content-Length: 7133 Lines: 149 --e0cb4e6ff5b130d9c904ca306cc1 Content-Type: text/plain; charset=UTF-8 thanks for the help. I only worked on it to get it running for the demo and I did try to make it a proper release. :S On 20 September 2012 13:36, Anil Madhavapeddy wrote: > I've started porting Haris' excellent crypto_keys library to be suitable > for release. There is proper command-line parsing and man page generation > now, thanks to this amazing library: > > http://erratique.ch/software/cmdliner/doc/Cmdliner.html > > Anyone wants to try it, install OPAM from http://github.com/OCamlPro/opam, > and add the Mirage dev repository: > > $ eval `opam config -env` > $ opam remote -add dev git://github.com/mirage/opam-repo-dev > $ opam install crypto_keys > > This doesnt actually install the binaries yet, but it gets all the > dependencies in place, so you can clone mirage/ocaml-crypto-keys and have > it compile very easily, for a change! > > Haris, I've got the DNSSEC-to-SSH converter working, but it looks you are > currently resolving DS records directly into SSH public keys, right? > Should we also look for the SSHFP and use those in preference to DS, if > specified? > yes I haven't yet looked that kind of record, but it is a big isue to incorporate them. The main issue there is the dns library that needs to add the parsing capability, and the rest should be straightforward. By the way, with this library I had a problem to use it as a library in signpost. If I used the library the program would segfault and debugging didn't gave me a progammatic reason. As a result, I was the command line tool to generate keys. My conlusion was that the problem had something to do with the c-ocaml binding with the ssl library. I need to try it again now, as I am thinking that this might be some bad memory handling code in c. > > -anil > -- Charalampos Rotsos PhD student The University of Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD Phone: +44-(0) 1223 767032 Email: cr409@cl.cam.ac.uk --e0cb4e6ff5b130d9c904ca306cc1 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable thanks for the help. I only worked on it to get it ru= nning for the demo and I did try to make it a proper rel= ease. :S

On 20 September 2012 13:36, Anil Madhavapedd= y <anil@recoil.org> wrote:
I've started porting Haris' excellent crypto_keys library to be sui= table for release. =C2=A0There is proper command-line parsing and man page = generation now, thanks to this amazing library:

http://erratique.ch/software/cmdliner/doc/Cmdliner.html

Anyone wants to try it, install OPAM from http://github.com/OCamlPro/opam, and add t= he Mirage dev repository:

$ eval `opam config -env`
$ opam remote -add dev git://github.com/mirage/opam-repo-dev<= /a>
$ opam install crypto_keys

This doesnt actually install the binaries yet, but it= gets all the dependencies in place, so you can clone mirage/= ocaml-crypto-keys and have it compile very easily, for a change!

Haris, I've got the DNSSEC-to-SSH converter working, but it looks you a= re currently resolving DS records directly into SSH public keys, right? =C2= =A0Should we also look for the SSHFP and use those in preference to DS, if = specified?


By the way, with this library I had a problem to use it= as a library in signpost. If I used the library the program would=C2=A0segfault and debugging didn= 9;t gave me a progammatic reason. As a re= sult, I was the command line tool to generate keys. My co= nlusion was that the problem had something to do with the c-ocaml binding with the ssl library. =C2=A0I= need to try it again now, as I am thinking that this might be some bad mem= ory handling code in c. =C2=A0
=C2=A0

-anil



-- Charalampos Rotsos
PhD student
The Unive= rsity of Cambridge
Computer Laboratory
William Gates Building
JJ Thomson Avenue
Cambr= idge
CB3 0FD

Phone: +44-(0) 1223 767032
Email: cr409@cl.cam.ac.uk
--e0cb4e6ff5b130d9c904ca306cc1-- From scott.dj@gmail.com Fri Sep 21 08:05:47 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TExJ5-00027F-LZ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 08:05:47 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.2 from SpamAssassin-3.3.2-1387911 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.220.179 listed in list.dnswl.dnsbl.ja.net] * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 0.9 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-vc0-f179.google.com ([209.85.220.179]:60505) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1TExJ5-0007xI-mm (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 08:05:47 +0100 Received: by vcbf13 with SMTP id f13so3365087vcb.38 for ; Fri, 21 Sep 2012 00:05:46 -0700 (PDT) MIME-Version: 1.0 Received: by 10.221.11.71 with SMTP id pd7mr2494295vcb.45.1348211146221; Fri, 21 Sep 2012 00:05:46 -0700 (PDT) Received: by 10.58.213.168 with HTTP; Fri, 21 Sep 2012 00:05:46 -0700 (PDT) In-Reply-To: <79634577-E4BF-43BB-894E-ACD00553504C@recoil.org> References: <742FFBD7-A9F2-4009-997F-48886E74B277@recoil.org> <79634577-E4BF-43BB-894E-ACD00553504C@recoil.org> Date: Fri, 21 Sep 2012 08:05:46 +0100 Message-ID: Subject: Re: mirage xen pcap example From: David Scott To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=bcaec54fb0c073e83d04ca30de80 Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 07:05:47 -0000 X-Keywords: X-UID: 578 Status: O Content-Length: 3034 Lines: 79 --bcaec54fb0c073e83d04ca30de80 Content-Type: text/plain; charset=ISO-8859-1 On Thursday, September 20, 2012, Anil Madhavapeddy wrote: > On 20 Sep 2012, at 17:45, David Scott > > wrote: > > > > Yup, it dumps to a raw block device. The 'make run' in the mirage-www > > branch will create a temporary file by dd'ing 128M of zeroes. If you > > tell tcpdump to read that then it will work, although it will get a > > bit confused by all the zeroes at the end. It works well enough to > > diagnose problems anyway! > > Speaking of mir-run... it has disappeared in the disaggregated Mirage. > Should I reimport it into mirage-platform? The problem of running bare > kernels still exists, of course... > > There used to be an XCP-compliant one, and we need some way to wrap the > EC2 scripts more automatically. What was that alternative to this > that you mentioned to me in OUD? Something to do with Openstack (!) Yeah I was thinking that we could use "puppet" or "chef" -- tools designed for quickly deploying services (www server, load balancer, db servers) on clouds like EC2. This might make the whole process a bit smoother. Cheers, Dave > > -anil > -- Dave Scott --bcaec54fb0c073e83d04ca30de80 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Thursday, September 20, 2012, Anil Madhavapeddy wrote:
On 20 Sep 2012, at 17:45, David Scott <scott.dj@gmail.com> wrote:
>
> Yup, it dumps to a raw block device. The 'make run' in the mir= age-www
> branch will create a temporary file by dd'ing 128M of zeroes. If y= ou
> tell tcpdump to read that then it will work, although it will get a > bit confused by all the zeroes at the end. It works well enough to
> diagnose problems anyway!

Speaking of mir-run... it has disappeared in the disaggregated Mirage.
Should I reimport it into mirage-platform? =A0The problem of running bare kernels still exists, of course...

There used to be an XCP-compliant one, and we need some way to wrap the
EC2 scripts more automatically. =A0What was that alternative to this
that you mentioned to me in OUD? =A0Something to do with Openstack (!)

Yeah I was thinking that we could use "pup= pet" or "chef" -- tools designed for quickly deploying servi= ces (www server, load balancer, db servers) on clouds like EC2. This might = make the whole process a bit smoother.

Cheers,
Dave=A0

-anil


--
Dave Scott
--bcaec54fb0c073e83d04ca30de80-- From anil@recoil.org Fri Sep 21 13:33:54 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TF2Qc-0004Ob-Gu (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 13:33:54 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387911 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:24714 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1TF2Qc-0006l2-pf (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 13:33:54 +0100 Received: (qmail 13916 invoked by uid 634); 21 Sep 2012 12:33:53 -0000 Received: from c-68-36-200-153.hsd1.nj.comcast.net (HELO [192.168.1.16]) (68.36.200.153) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 21 Sep 2012 13:33:53 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: crypto-keys and cmdinfo From: Anil Madhavapeddy In-Reply-To: Date: Fri, 21 Sep 2012 08:33:51 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <735C4853-A2DA-4606-96DD-51B69368269D@recoil.org> References: <0E7CAED0-88C7-4221-ACBE-1D69DD19CB5C@recoil.org> To: Haris Rotsos X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-signpost@lists.cam.ac.uk" , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 12:33:54 -0000 X-Keywords: X-UID: 579 Status: O Content-Length: 1561 Lines: 34 On 21 Sep 2012, at 02:33, Haris Rotsos wrote: >=20 > Haris, I've got the DNSSEC-to-SSH converter working, but it looks you = are currently resolving DS records directly into SSH public keys, right? = Should we also look for the SSHFP and use those in preference to DS, if = specified? >=20 > yes I haven't yet looked that kind of record, but it is a big isue to = incorporate them. The main issue there is the dns library that needs to = add the parsing capability, and the rest should be straightforward.=20 s/is a big/isnt a big I presume :) I'll take a look at SSHFP next week = (should be very easy, and it's probably better to derive a specific = SSHFP key than to use the DS directly). > By the way, with this library I had a problem to use it as a library = in signpost. If I used the library the program would segfault and = debugging didn't gave me a progammatic reason. As a result, I was the = command line tool to generate keys. My conlusion was that the problem = had something to do with the c-ocaml binding with the ssl library. I = need to try it again now, as I am thinking that this might be some bad = memory handling code in c.=20 That's very likely; those RSA bindings do need to be deleted before a = release since they are cut-and-pasted from an AGPL3 project (and so = really cannot be linked in). I implemented something similar in the SSH = library from years past, so I'll fix it up on the next pass. The swiss-army-knife command is really very useful though; good job on = hacking that together! -anil From anil@recoil.org Fri Sep 21 22:00:35 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TFAKx-0006SS-9w (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 22:00:35 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387911 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:25335 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1TFAKw-0002Kp-rW (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 22:00:35 +0100 Received: (qmail 19451 invoked by uid 634); 21 Sep 2012 21:00:32 -0000 Received: from 47.sub-70-192-128.myvzw.com (HELO [192.168.1.16]) (70.192.128.47) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 21 Sep 2012 22:00:32 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: lwt-json-logger and crunch releases Message-Id: <79E2848F-FBC2-4998-AEF8-E4ED03B7E360@recoil.org> Date: Fri, 21 Sep 2012 17:00:10 -0400 To: "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 21:00:35 -0000 X-Keywords: X-UID: 580 Status: O Content-Length: 1266 Lines: 39 I've released another two packages into OPAM today: ocaml-crunch: http://github.com/mirage/ocaml-crunch ------------ $ opam update $ opam install crunch $ man ocaml-crunch $ ocaml-crunch --help This is a tidied up version of the mirage-fs `mir-crunch` utility, which converts a directory into a static OCaml module that has no dependency on that filesystem. It uses cmdliner to generate man pages and nice help, so I separated it from the mirage-fs repository (which is where all the Block and Fat32 libs go, and this is useful independently. Secondly: ocaml-lwt-json-logger: http://github.com/avsm/ocaml-lwt-json-logger --------------------- $ opam install lwt-json-logger This uses crunch to embed a debug web server within any Lwt/Mirage binary, and serves up the log messages using fancy AJAX. To try it out, opam install the library (so you have all the dependencies) and then clone the repo and run ./_build/lib_test/lwt_json_logger_test.native It will start a webserver on localhost:8080, which you can connect to with a web browser to see the debug messages. The interface is still pretty beta, but I'm going to start using it in the Signpost code and see how useful it is. I hope to add more live progress bars using js_of_ocaml soon! -anil From jjl25@cam.ac.uk Fri Sep 21 22:34:49 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TFAs5-0006iL-HM (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 22:34:49 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from cpc2-cmbg14-2-0-cust622.5-4.cable.virginmedia.com ([86.26.2.111]:53079 helo=[192.168.1.140]) by ppsw-41.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.156]:587) with esmtpsa (PLAIN:jjl25) (TLSv1:AES128-SHA:128) id 1TFAs5-0003Qv-Qg (Exim 4.72) (return-path ); Fri, 21 Sep 2012 22:34:49 +0100 Subject: Re: lwt-json-logger and crunch releases Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: Jon Ludlam In-Reply-To: <79E2848F-FBC2-4998-AEF8-E4ED03B7E360@recoil.org> Date: Fri, 21 Sep 2012 22:34:48 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <79E2848F-FBC2-4998-AEF8-E4ED03B7E360@recoil.org> To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1278) Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 21:34:49 -0000 X-Keywords: X-UID: 581 Status: O Content-Length: 1656 Lines: 56 FYI I'm getting another failure to install 'crunch' into /usr/local/bin/ = - you pesky mac users keep breaking my opam! I'm frustrated because I _love_ the idea of lwt-json-logger and want to = use it immediately! Jon On 21 Sep 2012, at 22:00, Anil Madhavapeddy wrote: > I've released another two packages into OPAM today: >=20 > ocaml-crunch: http://github.com/mirage/ocaml-crunch > ------------ >=20 > $ opam update > $ opam install crunch > $ man ocaml-crunch > $ ocaml-crunch --help >=20 > This is a tidied up version of the mirage-fs `mir-crunch` utility, = which=20 > converts a directory into a static OCaml module that has no dependency = on > that filesystem. >=20 > It uses cmdliner to generate man pages and nice help, so I separated = it > from the mirage-fs repository (which is where all the Block and Fat32 > libs go, and this is useful independently.=20 >=20 > Secondly: >=20 > ocaml-lwt-json-logger: http://github.com/avsm/ocaml-lwt-json-logger > --------------------- >=20 > $ opam install lwt-json-logger >=20 > This uses crunch to embed a debug web server within any Lwt/Mirage > binary, and serves up the log messages using fancy AJAX. To try it > out, opam install the library (so you have all the dependencies) and > then clone the repo and run = ./_build/lib_test/lwt_json_logger_test.native >=20 > It will start a webserver on localhost:8080, which you can connect to > with a web browser to see the debug messages. >=20 > The interface is still pretty beta, but I'm going to start using it > in the Signpost code and see how useful it is. I hope to add more > live progress bars using js_of_ocaml soon! >=20 > -anil >=20 From anil@recoil.org Fri Sep 21 22:45:04 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TFB20-0006nu-Ac (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 22:45:04 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387911 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:24622 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1TFB1y-0001A6-r7 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 22:45:03 +0100 Received: (qmail 16893 invoked by uid 634); 21 Sep 2012 21:45:02 -0000 Received: from c-68-36-200-153.hsd1.nj.comcast.net (HELO [192.168.1.16]) (68.36.200.153) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 21 Sep 2012 22:45:02 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: lwt-json-logger and crunch releases From: Anil Madhavapeddy In-Reply-To: Date: Fri, 21 Sep 2012 17:44:58 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <79E2848F-FBC2-4998-AEF8-E4ED03B7E360@recoil.org> To: Jon Ludlam X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 21:45:04 -0000 X-Keywords: X-UID: 582 Status: O Content-Length: 3895 Lines: 105 Gosh darn it! Could you paste the problem? In my build, it does install = correctly into PREFIX under .opam: ./setup.bin -build -classic-display /usr/local/bin/ocamlopt.opt -I /usr/local/lib/ocaml/ocamlbuild unix.cmxa = /usr/local/lib/ocaml/ocamlbuild/ocamlbuildlib.cmxa myocamlbuild.ml = /usr/local/lib/ocaml/ocamlbuild/ocamlbuild.cmx -o myocamlbuild ocamlfind ocamldep -package unix -package cmdliner -modules = crunch/crunch.ml > crunch/crunch.ml.depends ocamlfind ocamlc -c -g -annot -package unix -package cmdliner -I crunch = -o crunch/crunch.cmo crunch/crunch.ml ocamlfind ocamlc -a crunch/crunch.cmo -o crunch/crunch.cma ocamlfind ocamlopt -c -g -annot -package unix -package cmdliner -I = crunch -o crunch/crunch.cmx crunch/crunch.ml ocamlfind ocamlopt -a crunch/crunch.cmx -o crunch/crunch.cmxa ocamlfind ocamlopt -shared -I crunch crunch/crunch.cmx -o = crunch/crunch.cmxs ocamlfind ocamlc -c crunch/custom_unix_stubs.c + ocamlfind ocamlc -c crunch/custom_unix_stubs.c clang: warning: argument unused during compilation: '-fno-defer-pop' mv custom_unix_stubs.o crunch/custom_unix_stubs.o /usr/local/bin/ocamlmklib -o crunch/crunch_stubs = crunch/custom_unix_stubs.o ocamlfind ocamlopt -g -linkpkg -package unix -package cmdliner = crunch/crunch.cmx crunch/libcrunch_stubs.a -o crunch/crunch.native ./setup.bin -test ./setup.bin -doc ./setup.bin -reinstall Installed /Users/avsm/.opam/system/lib/crunch/crunch.ml Installed /Users/avsm/.opam/system/lib/crunch/crunch.cma Installed /Users/avsm/.opam/system/lib/crunch/crunch.cmxa Installed /Users/avsm/.opam/system/lib/crunch/crunch.a Installed /Users/avsm/.opam/system/lib/crunch/crunch.cmxs Installed /Users/avsm/.opam/system/lib/crunch/crunch.cmi Installed /Users/avsm/.opam/system/lib/crunch/crunch.cmx Installed /Users/avsm/.opam/system/lib/crunch/META mkdir -p /Users/avsm/.opam/system/bin cp _build/crunch/crunch.native /Users/avsm/.opam/system/bin/ocaml-crunch mkdir -p /Users/avsm/.opam/system/share/man/man1 ./_build/crunch/crunch.native --help=3Dgroff > = /Users/avsm/.opam/system/share/man/man1/ocaml-crunch.1 || true -anil On 21 Sep 2012, at 17:34, Jon Ludlam wrote: > FYI I'm getting another failure to install 'crunch' into = /usr/local/bin/ - you pesky mac users keep breaking my opam! >=20 > I'm frustrated because I _love_ the idea of lwt-json-logger and want = to use it immediately! >=20 > Jon >=20 >=20 > On 21 Sep 2012, at 22:00, Anil Madhavapeddy wrote: >=20 >> I've released another two packages into OPAM today: >>=20 >> ocaml-crunch: http://github.com/mirage/ocaml-crunch >> ------------ >>=20 >> $ opam update >> $ opam install crunch >> $ man ocaml-crunch >> $ ocaml-crunch --help >>=20 >> This is a tidied up version of the mirage-fs `mir-crunch` utility, = which=20 >> converts a directory into a static OCaml module that has no = dependency on >> that filesystem. >>=20 >> It uses cmdliner to generate man pages and nice help, so I separated = it >> from the mirage-fs repository (which is where all the Block and Fat32 >> libs go, and this is useful independently.=20 >>=20 >> Secondly: >>=20 >> ocaml-lwt-json-logger: http://github.com/avsm/ocaml-lwt-json-logger >> --------------------- >>=20 >> $ opam install lwt-json-logger >>=20 >> This uses crunch to embed a debug web server within any Lwt/Mirage >> binary, and serves up the log messages using fancy AJAX. To try it >> out, opam install the library (so you have all the dependencies) and >> then clone the repo and run = ./_build/lib_test/lwt_json_logger_test.native >>=20 >> It will start a webserver on localhost:8080, which you can connect to >> with a web browser to see the debug messages. >>=20 >> The interface is still pretty beta, but I'm going to start using it >> in the Signpost code and see how useful it is. I hope to add more >> live progress bars using js_of_ocaml soon! >>=20 >> -anil >>=20 >=20 >=20 From jjl25@cam.ac.uk Fri Sep 21 22:55:45 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TFBCK-0006sf-Um (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 22:55:44 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from cpc2-cmbg14-2-0-cust622.5-4.cable.virginmedia.com ([86.26.2.111]:53736 helo=[192.168.1.140]) by ppsw-41.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.156]:587) with esmtpsa (PLAIN:jjl25) (TLSv1:AES128-SHA:128) id 1TFBCK-0007aI-SF (Exim 4.72) (return-path ); Fri, 21 Sep 2012 22:55:44 +0100 Subject: Re: lwt-json-logger and crunch releases Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: multipart/mixed; boundary="Apple-Mail=_DD75000C-69E7-4AB1-8D29-D3F98A4D0B10" From: Jon Ludlam In-Reply-To: Date: Fri, 21 Sep 2012 22:55:44 +0100 Message-Id: References: <79E2848F-FBC2-4998-AEF8-E4ED03B7E360@recoil.org> To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1278) Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 21:55:45 -0000 X-Keywords: X-UID: 583 Status: O Content-Length: 8517 Lines: 191 --Apple-Mail=_DD75000C-69E7-4AB1-8D29-D3F98A4D0B10 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii Here it is: --Apple-Mail=_DD75000C-69E7-4AB1-8D29-D3F98A4D0B10 Content-Disposition: attachment; filename=log Content-Type: application/octet-stream; name="log" Content-Transfer-Encoding: 7bit 3.12.1 The following actions will be performed: - install crunch.1.0.0 1 to install | 0 to reinstall | 0 to upgrade | 0 to downgrade | 0 to remove =-=-= crunch.1.0.0 =-=-= The archive for crunch.1.0.0 is in the local cache. Extracting /home/jon/.opam/archives/crunch.1.0.0+opam.tar.gz ... Build commands: make PREFIX=/home/jon/.opam/system make PREFIX=/home/jon/.opam/system install ocamlopt.opt -o setup.bin setup.ml || ocamlopt -o setup.bin setup.ml || ocamlc -o setup.bin setup.ml rm -f setup.cmx setup.cmi setup.o setup.cmo ./setup.bin -configure Configuration: ocamlfind: ........................................... /home/jon/.opam/system/bin/ocamlfind ocamlc: .............................................. /usr/bin/ocamlc.opt ocamlopt: ............................................ /usr/bin/ocamlopt.opt ocamlbuild: .......................................... /usr/bin/ocamlbuild Package name: ........................................ cstruct Package version: ..................................... 1.0.0 os_type: ............................................. Unix system: .............................................. linux architecture: ........................................ amd64 ccomp_type: .......................................... cc ocaml_version: ....................................... 3.12.1 standard_library_default: ............................ /usr/lib/ocaml standard_library: .................................... /usr/lib/ocaml standard_runtime: .................................... /usr/bin/ocamlrun bytecomp_c_compiler: ................................. gcc -D_FILE_OFFSET_BITS=64 -D_REENTRANT -fPIC native_c_compiler: ................................... gcc -D_FILE_OFFSET_BITS=64 -D_REENTRANT model: ............................................... default ext_obj: ............................................. .o ext_asm: ............................................. .s ext_lib: ............................................. .a ext_dll: ............................................. .so default_executable_name: ............................. a.out systhread_supported: ................................. true Install architecture-independent files dir: .......... /usr/local Install architecture-dependent files in dir: ......... $prefix User executables: .................................... $exec_prefix/bin System admin executables: ............................ $exec_prefix/sbin Program executables: ................................. $exec_prefix/libexec Read-only single-machine data: ....................... $prefix/etc Modifiable architecture-independent data: ............ $prefix/com Modifiable single-machine data: ...................... $prefix/var Object code libraries: ............................... $exec_prefix/lib Read-only arch-independent data root: ................ $prefix/share Read-only architecture-independent data: ............. $datarootdir Info documentation: .................................. $datarootdir/info Locale-dependent data: ............................... $datarootdir/locale Man documentation: ................................... $datarootdir/man Documentation root: .................................. $datarootdir/doc/$pkg_name HTML documentation: .................................. $docdir DVI documentation: ................................... $docdir PDF documentation: ................................... $docdir PS documentation: .................................... $docdir findlib_version: ..................................... 1.3.3 is_native: ........................................... true suffix_program: ...................................... Remove a file.: ...................................... rm -f Remove a directory.: ................................. rm -rf Turn ocaml debug flag on: ............................ true Turn ocaml profile flag on: .......................... false Compiler support generation of .cmxs.: ............... true OCamlbuild additional flags: ......................... Create documentations: ............................... true Compile tests executable and library and run them: ... false pkg_unix: ............................................ /usr/lib/ocaml pkg_cmdliner: ........................................ /home/jon/.opam/system/lib/cmdliner ./setup.bin -build -classic-display /usr/bin/ocamlopt.opt -I /usr/lib/ocaml/ocamlbuild unix.cmxa /usr/lib/ocaml/ocamlbuild/ocamlbuildlib.cmxa myocamlbuild.ml /usr/lib/ocaml/ocamlbuild/ocamlbuild.cmx -o myocamlbuild ocamlfind ocamldep -package unix -package cmdliner -modules crunch/crunch.ml > crunch/crunch.ml.depends ocamlfind ocamlc -c -g -annot -package unix -package cmdliner -I crunch -o crunch/crunch.cmo crunch/crunch.ml ocamlfind ocamlc -a crunch/crunch.cmo -o crunch/crunch.cma ocamlfind ocamlopt -c -g -annot -package unix -package cmdliner -I crunch -o crunch/crunch.cmx crunch/crunch.ml ocamlfind ocamlopt -a crunch/crunch.cmx -o crunch/crunch.cmxa ocamlfind ocamlopt -shared -I crunch crunch/crunch.cmx -o crunch/crunch.cmxs ocamlfind ocamlc -c crunch/custom_unix_stubs.c mv custom_unix_stubs.o crunch/custom_unix_stubs.o /usr/bin/ocamlmklib -o crunch/crunch_stubs crunch/custom_unix_stubs.o ocamlfind ocamlopt -g -linkpkg -package unix -package cmdliner crunch/crunch.cmx crunch/libcrunch_stubs.a -o crunch/crunch.native ./setup.bin -test ./setup.bin -doc ./setup.bin -reinstall Installed /home/jon/.opam/system/lib/crunch/crunch.ml Installed /home/jon/.opam/system/lib/crunch/crunch.cma Installed /home/jon/.opam/system/lib/crunch/crunch.cmxa Installed /home/jon/.opam/system/lib/crunch/crunch.a Installed /home/jon/.opam/system/lib/crunch/crunch.cmxs Installed /home/jon/.opam/system/lib/crunch/crunch.cmi Installed /home/jon/.opam/system/lib/crunch/crunch.cmx Installed /home/jon/.opam/system/lib/crunch/META cp: cannot create regular file `/usr/local/bin/crunch': Permission denied E: Failure("Command ''cp' '/home/jon/.opam/system/build/crunch.1.0.0/_build/crunch/crunch.native' '/usr/local/bin/crunch'' terminated with error code 1") make: *** [install] Error 1 Uninstalling crunch.1.0.0 ... The archive for crunch.1.0.0 is in the local cache. Extracting /home/jon/.opam/archives/crunch.1.0.0+opam.tar.gz ... --Apple-Mail=_DD75000C-69E7-4AB1-8D29-D3F98A4D0B10 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Jon On 21 Sep 2012, at 22:34, Jon Ludlam wrote: > FYI I'm getting another failure to install 'crunch' into = /usr/local/bin/ - you pesky mac users keep breaking my opam! >=20 > I'm frustrated because I _love_ the idea of lwt-json-logger and want = to use it immediately! >=20 > Jon >=20 >=20 > On 21 Sep 2012, at 22:00, Anil Madhavapeddy wrote: >=20 >> I've released another two packages into OPAM today: >>=20 >> ocaml-crunch: http://github.com/mirage/ocaml-crunch >> ------------ >>=20 >> $ opam update >> $ opam install crunch >> $ man ocaml-crunch >> $ ocaml-crunch --help >>=20 >> This is a tidied up version of the mirage-fs `mir-crunch` utility, = which=20 >> converts a directory into a static OCaml module that has no = dependency on >> that filesystem. >>=20 >> It uses cmdliner to generate man pages and nice help, so I separated = it >> from the mirage-fs repository (which is where all the Block and Fat32 >> libs go, and this is useful independently.=20 >>=20 >> Secondly: >>=20 >> ocaml-lwt-json-logger: http://github.com/avsm/ocaml-lwt-json-logger >> --------------------- >>=20 >> $ opam install lwt-json-logger >>=20 >> This uses crunch to embed a debug web server within any Lwt/Mirage >> binary, and serves up the log messages using fancy AJAX. To try it >> out, opam install the library (so you have all the dependencies) and >> then clone the repo and run = ./_build/lib_test/lwt_json_logger_test.native >>=20 >> It will start a webserver on localhost:8080, which you can connect to >> with a web browser to see the debug messages. >>=20 >> The interface is still pretty beta, but I'm going to start using it >> in the Signpost code and see how useful it is. I hope to add more >> live progress bars using js_of_ocaml soon! >>=20 >> -anil >>=20 >=20 >=20 --Apple-Mail=_DD75000C-69E7-4AB1-8D29-D3F98A4D0B10-- From anil@recoil.org Fri Sep 21 23:07:13 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TFBNR-00073o-3N (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 23:07:13 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387911 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:13663 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1TFBNQ-00052O-r7 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 23:07:13 +0100 Received: (qmail 24724 invoked by uid 634); 21 Sep 2012 22:07:12 -0000 Received: from c-68-36-200-153.hsd1.nj.comcast.net (HELO [192.168.1.16]) (68.36.200.153) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 21 Sep 2012 23:07:12 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: lwt-json-logger and crunch releases From: Anil Madhavapeddy In-Reply-To: Date: Fri, 21 Sep 2012 18:07:08 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <936DEA49-1B28-467A-BD67-DFE581B384EB@recoil.org> References: <79E2848F-FBC2-4998-AEF8-E4ED03B7E360@recoil.org> To: Jon Ludlam X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 22:07:13 -0000 X-Keywords: X-UID: 584 Status: O Content-Length: 2069 Lines: 77 Ahem, woops. crunch.1.0.1 is now available on OPAM :-) There should be no violation of your /usr/local now... -a On 21 Sep 2012, at 17:55, Jon Ludlam wrote: > Here it is: >=20 > >=20 > Jon >=20 >=20 > On 21 Sep 2012, at 22:34, Jon Ludlam wrote: >=20 >> FYI I'm getting another failure to install 'crunch' into = /usr/local/bin/ - you pesky mac users keep breaking my opam! >>=20 >> I'm frustrated because I _love_ the idea of lwt-json-logger and want = to use it immediately! >>=20 >> Jon >>=20 >>=20 >> On 21 Sep 2012, at 22:00, Anil Madhavapeddy wrote: >>=20 >>> I've released another two packages into OPAM today: >>>=20 >>> ocaml-crunch: http://github.com/mirage/ocaml-crunch >>> ------------ >>>=20 >>> $ opam update >>> $ opam install crunch >>> $ man ocaml-crunch >>> $ ocaml-crunch --help >>>=20 >>> This is a tidied up version of the mirage-fs `mir-crunch` utility, = which=20 >>> converts a directory into a static OCaml module that has no = dependency on >>> that filesystem. >>>=20 >>> It uses cmdliner to generate man pages and nice help, so I separated = it >>> from the mirage-fs repository (which is where all the Block and = Fat32 >>> libs go, and this is useful independently.=20 >>>=20 >>> Secondly: >>>=20 >>> ocaml-lwt-json-logger: http://github.com/avsm/ocaml-lwt-json-logger >>> --------------------- >>>=20 >>> $ opam install lwt-json-logger >>>=20 >>> This uses crunch to embed a debug web server within any Lwt/Mirage >>> binary, and serves up the log messages using fancy AJAX. To try it >>> out, opam install the library (so you have all the dependencies) and >>> then clone the repo and run = ./_build/lib_test/lwt_json_logger_test.native >>>=20 >>> It will start a webserver on localhost:8080, which you can connect = to >>> with a web browser to see the debug messages. >>>=20 >>> The interface is still pretty beta, but I'm going to start using it >>> in the Signpost code and see how useful it is. I hope to add more >>> live progress bars using js_of_ocaml soon! >>>=20 >>> -anil >>>=20 >>=20 >>=20 >=20 From jjl25@cam.ac.uk Fri Sep 21 23:19:59 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TFBZn-0007As-J2 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 23:19:59 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from cpc2-cmbg14-2-0-cust622.5-4.cable.virginmedia.com ([86.26.2.111]:54064 helo=[192.168.1.140]) by ppsw-43.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.159]:587) with esmtpsa (PLAIN:jjl25) (TLSv1:AES128-SHA:128) id 1TFBZn-0000Dw-nA (Exim 4.72) (return-path ); Fri, 21 Sep 2012 23:19:59 +0100 Subject: Re: lwt-json-logger and crunch releases Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: multipart/mixed; boundary="Apple-Mail=_0454962E-2ADB-44C6-87ED-000103078D60" From: Jon Ludlam In-Reply-To: Date: Fri, 21 Sep 2012 23:19:58 +0100 Message-Id: References: <79E2848F-FBC2-4998-AEF8-E4ED03B7E360@recoil.org> To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1278) Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 22:19:59 -0000 X-Keywords: X-UID: 585 Status: O Content-Length: 19995 Lines: 293 --Apple-Mail=_0454962E-2ADB-44C6-87ED-000103078D60 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Hmm. The good news is that crunch 1.0.1 installs flawlessly! The bad = news is - well, let's just say I now anxiously await cohttp.1.0.1=85 Logfile attached! Jon --Apple-Mail=_0454962E-2ADB-44C6-87ED-000103078D60 Content-Disposition: attachment; filename=log Content-Type: application/octet-stream; name="log" Content-Transfer-Encoding: 7bit 3.12.1 The following actions will be performed: - install cohttp.1.0.0 1 to install | 0 to reinstall | 0 to upgrade | 0 to downgrade | 0 to remove =-=-= cohttp.1.0.0 =-=-= The archive for cohttp.1.0.0 is in the local cache. Extracting /home/jon/.opam/archives/cohttp.1.0.0+opam.tar.gz ... Build commands: make all make install ocamlopt.opt -o setup.bin setup.ml || ocamlopt -o setup.bin setup.ml || ocamlc -o setup.bin setup.ml rm -f setup.cmx setup.cmi setup.o setup.cmo ./setup.bin -configure --enable-tests Configuration: ocamlfind: ........................................... /home/jon/.opam/system/bin/ocamlfind ocamlc: .............................................. /usr/bin/ocamlc.opt ocamlopt: ............................................ /usr/bin/ocamlopt.opt ocamlbuild: .......................................... /usr/bin/ocamlbuild Package name: ........................................ cohttp Package version: ..................................... 0.9.3 os_type: ............................................. Unix system: .............................................. linux architecture: ........................................ amd64 ccomp_type: .......................................... cc ocaml_version: ....................................... 3.12.1 standard_library_default: ............................ /usr/lib/ocaml standard_library: .................................... /usr/lib/ocaml standard_runtime: .................................... /usr/bin/ocamlrun bytecomp_c_compiler: ................................. gcc -D_FILE_OFFSET_BITS=64 -D_REENTRANT -fPIC native_c_compiler: ................................... gcc -D_FILE_OFFSET_BITS=64 -D_REENTRANT model: ............................................... default ext_obj: ............................................. .o ext_asm: ............................................. .s ext_lib: ............................................. .a ext_dll: ............................................. .so default_executable_name: ............................. a.out systhread_supported: ................................. true Install architecture-independent files dir: .......... /usr/local Install architecture-dependent files in dir: ......... $prefix User executables: .................................... $exec_prefix/bin System admin executables: ............................ $exec_prefix/sbin Program executables: ................................. $exec_prefix/libexec Read-only single-machine data: ....................... $prefix/etc Modifiable architecture-independent data: ............ $prefix/com Modifiable single-machine data: ...................... $prefix/var Object code libraries: ............................... $exec_prefix/lib Read-only arch-independent data root: ................ $prefix/share Read-only architecture-independent data: ............. $datarootdir Info documentation: .................................. $datarootdir/info Locale-dependent data: ............................... $datarootdir/locale Man documentation: ................................... $datarootdir/man Documentation root: .................................. $datarootdir/doc/$pkg_name HTML documentation: .................................. $docdir DVI documentation: ................................... $docdir PDF documentation: ................................... $docdir PS documentation: .................................... $docdir findlib_version: ..................................... 1.3.3 is_native: ........................................... true suffix_program: ...................................... Remove a file.: ...................................... rm -f Remove a directory.: ................................. rm -rf Turn ocaml debug flag on: ............................ true Turn ocaml profile flag on: .......................... false Compiler support generation of .cmxs.: ............... true OCamlbuild additional flags: ......................... build the Lwt library: ............................... false build the Core/Async library: ........................ false build the Mirage library: ............................ false run the internet-using tests: ........................ false Create documentations: ............................... true Compile tests executable and library and run them: ... true pkg_re: .............................................. /home/jon/.opam/system/lib/re pkg_uri: ............................................. /home/jon/.opam/system/lib/uri pkg_uri_services: .................................... /home/jon/.opam/system/lib/uri ocamldoc: ............................................ /usr/bin/ocamldoc pkg_ounit: ........................................... /home/jon/.opam/system/lib/oUnit ./setup.bin -build /usr/bin/ocamlopt.opt -I /usr/lib/ocaml/ocamlbuild unix.cmxa /usr/lib/ocaml/ocamlbuild/ocamlbuildlib.cmxa myocamlbuild.ml /usr/lib/ocaml/ocamlbuild/ocamlbuild.cmx -o myocamlbuild ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/code.mli > lib/code.mli.depends ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/code.cmi lib/code.mli ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/code.ml > lib/code.ml.depends ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/header.mli > lib/header.mli.depends ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/accept.mli > lib/accept.mli.depends ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/accept_types.ml > lib/accept_types.ml.depends ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/accept_types.cmo lib/accept_types.ml ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/auth.mli > lib/auth.mli.depends ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/make.ml > lib/make.ml.depends ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/transfer.mli > lib/transfer.mli.depends ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/make.cmo lib/make.ml ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/accept.cmi lib/accept.mli ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/auth.cmi lib/auth.mli ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/transfer.cmi lib/transfer.mli ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/header.cmi lib/header.mli ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/header.ml > lib/header.ml.depends ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/request.mli > lib/request.mli.depends ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/request.cmi lib/request.mli ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/request.ml > lib/request.ml.depends ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/response.mli > lib/response.mli.depends ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/response.cmi lib/response.mli ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/response.ml > lib/response.ml.depends ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/transfer.ml > lib/transfer.ml.depends ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/accept.ml > lib/accept.ml.depends /usr/bin/ocamllex.opt -q lib/accept_lexer.mll ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/accept_lexer.ml > lib/accept_lexer.ml.depends /usr/bin/ocamlyacc lib/accept_parser.mly ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/accept_parser.mli > lib/accept_parser.mli.depends ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/accept_parser.cmi lib/accept_parser.mli ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/accept_lexer.cmo lib/accept_lexer.ml ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/accept_parser.ml > lib/accept_parser.ml.depends ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/base64.mli > lib/base64.mli.depends ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/base64.cmi lib/base64.mli ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/base64.ml > lib/base64.ml.depends ocamlfind ocamldep -package uri.services -package uri -package re -modules lib/auth.ml > lib/auth.ml.depends ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/code.cmo lib/code.ml ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/header.cmo lib/header.ml ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/request.cmo lib/request.ml ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/response.cmo lib/response.ml ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/transfer.cmo lib/transfer.ml ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/accept.cmo lib/accept.ml ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/accept_parser.cmo lib/accept_parser.ml ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/base64.cmo lib/base64.ml ocamlfind ocamlc -c -g -annot -package uri.services -package uri -package re -I lib -o lib/auth.cmo lib/auth.ml ocamlfind ocamlc -pack -g lib/code.cmo lib/accept_parser.cmo lib/accept_lexer.cmo lib/accept.cmo lib/base64.cmo lib/auth.cmo lib/make.cmo lib/transfer.cmo lib/header.cmo lib/request.cmo lib/response.cmo -o lib/cohttp.cmo ocamlfind ocamlc -a lib/accept_types.cmo lib/cohttp.cmo -o lib/cohttp.cma ocamlfind ocamlopt -c -g -annot -package uri.services -package uri -package re -I lib -o lib/accept_types.cmx lib/accept_types.ml ocamlfind ocamlopt -c -g -annot -package uri.services -package uri -package re -for-pack Cohttp -I lib -o lib/accept_parser.cmx lib/accept_parser.ml ocamlfind ocamlopt -c -g -annot -package uri.services -package uri -package re -for-pack Cohttp -I lib -o lib/accept_lexer.cmx lib/accept_lexer.ml ocamlfind ocamlopt -c -g -annot -package uri.services -package uri -package re -for-pack Cohttp -I lib -o lib/base64.cmx lib/base64.ml ocamlfind ocamlopt -c -g -annot -package uri.services -package uri -package re -for-pack Cohttp -I lib -o lib/make.cmx lib/make.ml ocamlfind ocamlopt -c -g -annot -package uri.services -package uri -package re -for-pack Cohttp -I lib -o lib/accept.cmx lib/accept.ml ocamlfind ocamlopt -c -g -annot -package uri.services -package uri -package re -for-pack Cohttp -I lib -o lib/auth.cmx lib/auth.ml ocamlfind ocamlopt -c -g -annot -package uri.services -package uri -package re -for-pack Cohttp -I lib -o lib/transfer.cmx lib/transfer.ml ocamlfind ocamlopt -c -g -annot -package uri.services -package uri -package re -for-pack Cohttp -I lib -o lib/code.cmx lib/code.ml ocamlfind ocamlopt -c -g -annot -package uri.services -package uri -package re -for-pack Cohttp -I lib -o lib/header.cmx lib/header.ml ocamlfind ocamlopt -c -g -annot -package uri.services -package uri -package re -for-pack Cohttp -I lib -o lib/request.cmx lib/request.ml ocamlfind ocamlopt -c -g -annot -package uri.services -package uri -package re -for-pack Cohttp -I lib -o lib/response.cmx lib/response.ml touch lib/cohttp.mli ; if ocamlfind ocamlopt -pack -g -I lib lib/code.cmx lib/accept_parser.cmx lib/accept_lexer.cmx lib/accept.cmx lib/base64.cmx lib/auth.cmx lib/make.cmx lib/transfer.cmx lib/header.cmx lib/request.cmx lib/response.cmx -o lib/cohttp.cmx ; then rm -f lib/cohttp.mli ; else rm -f lib/cohttp.mli ; exit 1; fi ocamlfind ocamlopt -a lib/accept_types.cmx lib/cohttp.cmx -o lib/cohttp.cmxa ocamlfind ocamlopt -shared lib/cohttp.cmx -o lib/cohttp.cmxs ocamlfind ocamldep -package uri.services -package uri -package threads -package re -package oUnit -package lwt.unix -package lwt.syntax -package lwt -modules lib_test/test_accept.ml > lib_test/test_accept.ml.depends ocamlfind ocamlc -c -g -annot -I lib -I lwt -I lwt -I async -package uri.services -package uri -package threads -package re -package oUnit -package lwt.unix -package lwt.syntax -package lwt -thread -I lib_test -I async -I lib -I lwt -o lib_test/test_accept.cmo lib_test/test_accept.ml ocamlfind ocamlopt -c -g -annot -I lib -I lwt -I lwt -I async -package uri.services -package uri -package threads -package re -package oUnit -package lwt.unix -package lwt.syntax -package lwt -thread -I lib_test -I async -I lib -I lwt -o lib_test/test_accept.cmx lib_test/test_accept.ml ocamlfind ocamlopt -g -linkpkg -package uri.services -package uri -package re -package oUnit lib/cohttp.cmxa lib/accept_types.cmx lib/cohttp.cmx lib_test/test_accept.cmx -o lib_test/test_accept.native ocamlfind ocamldep -package uri.services -package uri -package threads -package re -package oUnit -package lwt.unix -package lwt.syntax -package lwt -modules lib_test/test_header.ml > lib_test/test_header.ml.depends ocamlfind ocamlc -c -g -annot -I lib -I lwt -I lwt -I async -package uri.services -package uri -package threads -package re -package oUnit -package lwt.unix -package lwt.syntax -package lwt -thread -I lib_test -I async -I lib -I lwt -o lib_test/test_header.cmo lib_test/test_header.ml ocamlfind ocamlopt -c -g -annot -I lib -I lwt -I lwt -I async -package uri.services -package uri -package threads -package re -package oUnit -package lwt.unix -package lwt.syntax -package lwt -thread -I lib_test -I async -I lib -I lwt -o lib_test/test_header.cmx lib_test/test_header.ml ocamlfind ocamlopt -g -linkpkg -package uri.services -package uri -package re -package oUnit lib/cohttp.cmxa lib/accept_types.cmx lib/cohttp.cmx lib_test/test_header.cmx -o lib_test/test_header.native ./setup.bin -test ............... Ran: 15 tests in: 0.00 seconds. OK.... Ran: 4 tests in: 0.00 seconds. OK..... Ran: 5 tests in: 0.00 seconds. OK....... Ran: 7 tests in: 0.00 seconds. OK. Ran: 1 tests in: 0.00 seconds. OK./setup.bin -doc ocamlfind ocamldoc -dump lib/code.odoc -package uri.services -package uri -package re -I lib lib/code.mli ocamlfind ocamldoc -dump lib/header.odoc -package uri.services -package uri -package re -I lib lib/header.mli ocamlfind ocamldoc -dump lib/request.odoc -package uri.services -package uri -package re -I lib lib/request.mli ocamlfind ocamldoc -dump lib/response.odoc -package uri.services -package uri -package re -I lib lib/response.mli ocamlfind ocamldoc -dump lib/transfer.odoc -package uri.services -package uri -package re -I lib lib/transfer.mli ocamlfind ocamldoc -dump lib/accept.odoc -package uri.services -package uri -package re -I lib lib/accept.mli ocamlfind ocamldoc -dump lib/accept_parser.odoc -package uri.services -package uri -package re -I lib lib/accept_parser.mli ocamlfind ocamldoc -dump lib/accept_lexer.odoc -package uri.services -package uri -package re -I lib lib/accept_lexer.ml ocamlfind ocamldoc -dump lib/base64.odoc -package uri.services -package uri -package re -I lib lib/base64.mli ocamlfind ocamldoc -dump lib/auth.odoc -package uri.services -package uri -package re -I lib lib/auth.mli ocamlfind ocamldoc -dump lib/make.odoc -package uri.services -package uri -package re -I lib lib/make.ml rm -rf lib/cohttp.docdir mkdir -p lib/cohttp.docdir ocamlfind ocamldoc -load lib/code.odoc -load lib/header.odoc -load lib/request.odoc -load lib/response.odoc -load lib/transfer.odoc -load lib/accept.odoc -load lib/accept_parser.odoc -load lib/accept_lexer.odoc -load lib/base64.odoc -load lib/auth.odoc -load lib/make.odoc -html -d lib/cohttp.docdir ./setup.bin -install W: Cannot find source header for module Accept_parser in library cohttp W: Cannot find source header for module Accept_lexer in library cohttp Installed /home/jon/.opam/system/lib/cohttp/code.mli Installed /home/jon/.opam/system/lib/cohttp/header.mli Installed /home/jon/.opam/system/lib/cohttp/request.mli Installed /home/jon/.opam/system/lib/cohttp/response.mli Installed /home/jon/.opam/system/lib/cohttp/transfer.mli Installed /home/jon/.opam/system/lib/cohttp/accept.mli Installed /home/jon/.opam/system/lib/cohttp/base64.mli Installed /home/jon/.opam/system/lib/cohttp/auth.mli Installed /home/jon/.opam/system/lib/cohttp/make.ml Installed /home/jon/.opam/system/lib/cohttp/cohttp.cmi Installed /home/jon/.opam/system/lib/cohttp/cohttp.cma Installed /home/jon/.opam/system/lib/cohttp/cohttp.cmxa Installed /home/jon/.opam/system/lib/cohttp/cohttp.a Installed /home/jon/.opam/system/lib/cohttp/cohttp.cmxs Installed /home/jon/.opam/system/lib/cohttp/META mkdir: cannot create directory `/usr/local/share/doc': Permission denied E: Failure("Command ''mkdir' '/usr/local/share/doc'' terminated with error code 1") make: *** [install] Error 1 Uninstalling cohttp.1.0.0 ... Removed /home/jon/.opam/system/lib/cohttp The archive for cohttp.1.0.0 is in the local cache. Extracting /home/jon/.opam/archives/cohttp.1.0.0+opam.tar.gz ... --Apple-Mail=_0454962E-2ADB-44C6-87ED-000103078D60 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 21 Sep 2012, at 22:34, Jon Ludlam wrote: > FYI I'm getting another failure to install 'crunch' into = /usr/local/bin/ - you pesky mac users keep breaking my opam! >=20 > I'm frustrated because I _love_ the idea of lwt-json-logger and want = to use it immediately! >=20 > Jon >=20 >=20 > On 21 Sep 2012, at 22:00, Anil Madhavapeddy wrote: >=20 >> I've released another two packages into OPAM today: >>=20 >> ocaml-crunch: http://github.com/mirage/ocaml-crunch >> ------------ >>=20 >> $ opam update >> $ opam install crunch >> $ man ocaml-crunch >> $ ocaml-crunch --help >>=20 >> This is a tidied up version of the mirage-fs `mir-crunch` utility, = which=20 >> converts a directory into a static OCaml module that has no = dependency on >> that filesystem. >>=20 >> It uses cmdliner to generate man pages and nice help, so I separated = it >> from the mirage-fs repository (which is where all the Block and Fat32 >> libs go, and this is useful independently.=20 >>=20 >> Secondly: >>=20 >> ocaml-lwt-json-logger: http://github.com/avsm/ocaml-lwt-json-logger >> --------------------- >>=20 >> $ opam install lwt-json-logger >>=20 >> This uses crunch to embed a debug web server within any Lwt/Mirage >> binary, and serves up the log messages using fancy AJAX. To try it >> out, opam install the library (so you have all the dependencies) and >> then clone the repo and run = ./_build/lib_test/lwt_json_logger_test.native >>=20 >> It will start a webserver on localhost:8080, which you can connect to >> with a web browser to see the debug messages. >>=20 >> The interface is still pretty beta, but I'm going to start using it >> in the Signpost code and see how useful it is. I hope to add more >> live progress bars using js_of_ocaml soon! >>=20 >> -anil >>=20 >=20 >=20 --Apple-Mail=_0454962E-2ADB-44C6-87ED-000103078D60-- From anil@recoil.org Fri Sep 21 23:30:07 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TFBjb-0007Gm-6D (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 23:30:07 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387911 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:26796 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1TFBja-0000ef-r7 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 23:30:07 +0100 Received: (qmail 23605 invoked by uid 634); 21 Sep 2012 22:30:06 -0000 Received: from c-68-36-200-153.hsd1.nj.comcast.net (HELO [192.168.1.16]) (68.36.200.153) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 21 Sep 2012 23:30:06 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: lwt-json-logger and crunch releases From: Anil Madhavapeddy In-Reply-To: Date: Fri, 21 Sep 2012 18:30:02 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <8B75D9F2-93A6-4D7B-A05F-2B4377165D2D@recoil.org> References: <79E2848F-FBC2-4998-AEF8-E4ED03B7E360@recoil.org> To: Jon Ludlam X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 22:30:07 -0000 X-Keywords: X-UID: 586 Status: O Content-Length: 2720 Lines: 96 You're like a human Jenkins! The OPAM file for the dev repo wasn't = synched with the release repo. I just did that, and it should install into the right = prefix now... Btw, I'm going with the convention of just calling the dev repo packages = (which=20 have git:// URLs and are refreshed on every opam update) as 1.0.0, and = the main opam-repository with tarballs will have lower release versions (0.9.2 = for cohttp at present). This means we don't have to keep updating the dev package on every = release, as OPAM sometimes gets unhappy and full of Not_founds when a package = directory disappears... -anil On 21 Sep 2012, at 18:19, Jon Ludlam wrote: > Hmm. The good news is that crunch 1.0.1 installs flawlessly! The bad = news is - well, let's just say I now anxiously await cohttp.1.0.1=85 >=20 > Logfile attached! >=20 > Jon >=20 > >=20 > On 21 Sep 2012, at 22:34, Jon Ludlam wrote: >=20 >> FYI I'm getting another failure to install 'crunch' into = /usr/local/bin/ - you pesky mac users keep breaking my opam! >>=20 >> I'm frustrated because I _love_ the idea of lwt-json-logger and want = to use it immediately! >>=20 >> Jon >>=20 >>=20 >> On 21 Sep 2012, at 22:00, Anil Madhavapeddy wrote: >>=20 >>> I've released another two packages into OPAM today: >>>=20 >>> ocaml-crunch: http://github.com/mirage/ocaml-crunch >>> ------------ >>>=20 >>> $ opam update >>> $ opam install crunch >>> $ man ocaml-crunch >>> $ ocaml-crunch --help >>>=20 >>> This is a tidied up version of the mirage-fs `mir-crunch` utility, = which=20 >>> converts a directory into a static OCaml module that has no = dependency on >>> that filesystem. >>>=20 >>> It uses cmdliner to generate man pages and nice help, so I separated = it >>> from the mirage-fs repository (which is where all the Block and = Fat32 >>> libs go, and this is useful independently.=20 >>>=20 >>> Secondly: >>>=20 >>> ocaml-lwt-json-logger: http://github.com/avsm/ocaml-lwt-json-logger >>> --------------------- >>>=20 >>> $ opam install lwt-json-logger >>>=20 >>> This uses crunch to embed a debug web server within any Lwt/Mirage >>> binary, and serves up the log messages using fancy AJAX. To try it >>> out, opam install the library (so you have all the dependencies) and >>> then clone the repo and run = ./_build/lib_test/lwt_json_logger_test.native >>>=20 >>> It will start a webserver on localhost:8080, which you can connect = to >>> with a web browser to see the debug messages. >>>=20 >>> The interface is still pretty beta, but I'm going to start using it >>> in the Signpost code and see how useful it is. I hope to add more >>> live progress bars using js_of_ocaml soon! >>>=20 >>> -anil >>>=20 >>=20 >>=20 >=20 From anil@recoil.org Fri Sep 21 23:54:10 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TFC6s-0007SU-GB (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 23:54:10 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387911 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:30834 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1TFC6s-0004s3-pf (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 21 Sep 2012 23:54:10 +0100 Received: (qmail 23479 invoked by uid 634); 21 Sep 2012 22:54:09 -0000 Received: from c-68-36-200-153.hsd1.nj.comcast.net (HELO [192.168.1.16]) (68.36.200.153) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 21 Sep 2012 23:54:09 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: correct cohttp opamconstraints Date: Fri, 21 Sep 2012 18:54:05 -0400 Message-Id: To: Thomas Gazagnaire Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 22:54:10 -0000 X-Keywords: X-UID: 587 Status: O Content-Length: 704 Lines: 23 Thomas, Mort just ran into an issue with installing cohttp, as you have to = remember to specify the ssl package too. Right now, the cohttp optional = dependencies are: depends: [ "ocamlfind" "re" "uri" {>=3D"1.3.2"} "ounit" ] depopts: [ "async" {=3D"108.00.02"} "lwt" {>=3D"2.4.1"} "mirage-net" ] However, cohttp requires Lwt AND SSL to be installed, or it fails (since = SSL is not yet optional). Is there any way to express this in the CNF? I tried: depopts: [ "async" {=3D"108.00.02"} ("lwt" {>=3D"2.4.1"} "ssl") = "mirage-net" ] but that gives me an error. We cant unconditionally add SSL as that = then breaks the Mirage/xen build (again, until supported via = cross-compilation). -anil= From jjl25@cam.ac.uk Sat Sep 22 00:06:38 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TFCIw-0007fM-Ct (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 22 Sep 2012 00:06:38 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from cpc2-cmbg14-2-0-cust622.5-4.cable.virginmedia.com ([86.26.2.111]:54203 helo=[192.168.1.140]) by ppsw-43.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.159]:587) with esmtpsa (PLAIN:jjl25) (TLSv1:AES128-SHA:128) id 1TFCIw-0007zQ-nG (Exim 4.72) (return-path ); Sat, 22 Sep 2012 00:06:38 +0100 Subject: Re: lwt-json-logger and crunch releases Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=windows-1252 From: Jon Ludlam In-Reply-To: <8B75D9F2-93A6-4D7B-A05F-2B4377165D2D@recoil.org> Date: Sat, 22 Sep 2012 00:06:37 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <79E2848F-FBC2-4998-AEF8-E4ED03B7E360@recoil.org> <8B75D9F2-93A6-4D7B-A05F-2B4377165D2D@recoil.org> To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1278) Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 23:06:38 -0000 X-Keywords: X-UID: 588 Status: O Content-Length: 3023 Lines: 107 Hurrah! Success! The logger is installed and running - very nice indeed. I'm sure I'll be = able to use this in a number of places. Jon On 21 Sep 2012, at 23:30, Anil Madhavapeddy wrote: > You're like a human Jenkins! The OPAM file for the dev repo wasn't = synched with > the release repo. I just did that, and it should install into the = right prefix > now... >=20 > Btw, I'm going with the convention of just calling the dev repo = packages (which=20 > have git:// URLs and are refreshed on every opam update) as 1.0.0, and = the main > opam-repository with tarballs will have lower release versions (0.9.2 = for cohttp > at present). >=20 > This means we don't have to keep updating the dev package on every = release, as > OPAM sometimes gets unhappy and full of Not_founds when a package = directory > disappears... >=20 > -anil >=20 > On 21 Sep 2012, at 18:19, Jon Ludlam wrote: >=20 >> Hmm. The good news is that crunch 1.0.1 installs flawlessly! The bad = news is - well, let's just say I now anxiously await cohttp.1.0.1=85 >>=20 >> Logfile attached! >>=20 >> Jon >>=20 >> >>=20 >> On 21 Sep 2012, at 22:34, Jon Ludlam wrote: >>=20 >>> FYI I'm getting another failure to install 'crunch' into = /usr/local/bin/ - you pesky mac users keep breaking my opam! >>>=20 >>> I'm frustrated because I _love_ the idea of lwt-json-logger and want = to use it immediately! >>>=20 >>> Jon >>>=20 >>>=20 >>> On 21 Sep 2012, at 22:00, Anil Madhavapeddy wrote: >>>=20 >>>> I've released another two packages into OPAM today: >>>>=20 >>>> ocaml-crunch: http://github.com/mirage/ocaml-crunch >>>> ------------ >>>>=20 >>>> $ opam update >>>> $ opam install crunch >>>> $ man ocaml-crunch >>>> $ ocaml-crunch --help >>>>=20 >>>> This is a tidied up version of the mirage-fs `mir-crunch` utility, = which=20 >>>> converts a directory into a static OCaml module that has no = dependency on >>>> that filesystem. >>>>=20 >>>> It uses cmdliner to generate man pages and nice help, so I = separated it >>>> from the mirage-fs repository (which is where all the Block and = Fat32 >>>> libs go, and this is useful independently.=20 >>>>=20 >>>> Secondly: >>>>=20 >>>> ocaml-lwt-json-logger: http://github.com/avsm/ocaml-lwt-json-logger >>>> --------------------- >>>>=20 >>>> $ opam install lwt-json-logger >>>>=20 >>>> This uses crunch to embed a debug web server within any Lwt/Mirage >>>> binary, and serves up the log messages using fancy AJAX. To try it >>>> out, opam install the library (so you have all the dependencies) = and >>>> then clone the repo and run = ./_build/lib_test/lwt_json_logger_test.native >>>>=20 >>>> It will start a webserver on localhost:8080, which you can connect = to >>>> with a web browser to see the debug messages. >>>>=20 >>>> The interface is still pretty beta, but I'm going to start using it >>>> in the Signpost code and see how useful it is. I hope to add more >>>> live progress bars using js_of_ocaml soon! >>>>=20 >>>> -anil >>>>=20 >>>=20 >>>=20 >>=20 >=20 From anil@recoil.org Sat Sep 22 00:11:51 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TFCNz-0007gK-Pz (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 22 Sep 2012 00:11:51 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1387911 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:21851 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1TFCNy-0007oY-si (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 22 Sep 2012 00:11:51 +0100 Received: (qmail 17592 invoked by uid 634); 21 Sep 2012 23:11:50 -0000 Received: from c-68-36-200-153.hsd1.nj.comcast.net (HELO [192.168.1.16]) (68.36.200.153) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sat, 22 Sep 2012 00:11:50 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Subject: Re: lwt-json-logger and crunch releases From: Anil Madhavapeddy In-Reply-To: Date: Fri, 21 Sep 2012 19:11:45 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <3093876F-4E86-450B-8E7E-DD4FE0B07863@recoil.org> References: <79E2848F-FBC2-4998-AEF8-E4ED03B7E360@recoil.org> <8B75D9F2-93A6-4D7B-A05F-2B4377165D2D@recoil.org> To: Jon Ludlam X-Mailer: Apple Mail (2.1486) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 23:11:51 -0000 X-Keywords: X-UID: 589 Status: O Content-Length: 4120 Lines: 140 Great! Just a word of warning that the Lwt_logger might not be the best = thing to use in production anger. The progress bar can only be implemented = via a=20 custom Lwt_section which leaks memory per bar, so this is definitely a = prototype. I'm wondering if we should just create a simple functorised Log module = around Async/Lwt and use that instead. It would also give us much more = flexibility for (e.g.) providing callbacks for more advanced plotting such as live = spark-graphs on values. The Async approach is to generate a lazy datastructure, and the Lwt = approach is a p4 extension that is compiled out of production binaries. I'm inclined = to go with the Async approach and leave logging in all the time. It's also easier = to turn a dynamic webserver on/off on demand. Balraj hacked one directly into the = TCP stack, and I'm thinking being able to toggle it there might be quite useful. -anil On 21 Sep 2012, at 19:06, Jon Ludlam wrote: > Hurrah! Success! >=20 > The logger is installed and running - very nice indeed. I'm sure I'll = be able to use this in a number of places. >=20 > Jon >=20 > On 21 Sep 2012, at 23:30, Anil Madhavapeddy wrote: >=20 >> You're like a human Jenkins! The OPAM file for the dev repo wasn't = synched with >> the release repo. I just did that, and it should install into the = right prefix >> now... >>=20 >> Btw, I'm going with the convention of just calling the dev repo = packages (which=20 >> have git:// URLs and are refreshed on every opam update) as 1.0.0, = and the main >> opam-repository with tarballs will have lower release versions (0.9.2 = for cohttp >> at present). >>=20 >> This means we don't have to keep updating the dev package on every = release, as >> OPAM sometimes gets unhappy and full of Not_founds when a package = directory >> disappears... >>=20 >> -anil >>=20 >> On 21 Sep 2012, at 18:19, Jon Ludlam wrote: >>=20 >>> Hmm. The good news is that crunch 1.0.1 installs flawlessly! The bad = news is - well, let's just say I now anxiously await cohttp.1.0.1=85 >>>=20 >>> Logfile attached! >>>=20 >>> Jon >>>=20 >>> >>>=20 >>> On 21 Sep 2012, at 22:34, Jon Ludlam wrote: >>>=20 >>>> FYI I'm getting another failure to install 'crunch' into = /usr/local/bin/ - you pesky mac users keep breaking my opam! >>>>=20 >>>> I'm frustrated because I _love_ the idea of lwt-json-logger and = want to use it immediately! >>>>=20 >>>> Jon >>>>=20 >>>>=20 >>>> On 21 Sep 2012, at 22:00, Anil Madhavapeddy wrote: >>>>=20 >>>>> I've released another two packages into OPAM today: >>>>>=20 >>>>> ocaml-crunch: http://github.com/mirage/ocaml-crunch >>>>> ------------ >>>>>=20 >>>>> $ opam update >>>>> $ opam install crunch >>>>> $ man ocaml-crunch >>>>> $ ocaml-crunch --help >>>>>=20 >>>>> This is a tidied up version of the mirage-fs `mir-crunch` utility, = which=20 >>>>> converts a directory into a static OCaml module that has no = dependency on >>>>> that filesystem. >>>>>=20 >>>>> It uses cmdliner to generate man pages and nice help, so I = separated it >>>>> from the mirage-fs repository (which is where all the Block and = Fat32 >>>>> libs go, and this is useful independently.=20 >>>>>=20 >>>>> Secondly: >>>>>=20 >>>>> ocaml-lwt-json-logger: = http://github.com/avsm/ocaml-lwt-json-logger >>>>> --------------------- >>>>>=20 >>>>> $ opam install lwt-json-logger >>>>>=20 >>>>> This uses crunch to embed a debug web server within any Lwt/Mirage >>>>> binary, and serves up the log messages using fancy AJAX. To try = it >>>>> out, opam install the library (so you have all the dependencies) = and >>>>> then clone the repo and run = ./_build/lib_test/lwt_json_logger_test.native >>>>>=20 >>>>> It will start a webserver on localhost:8080, which you can connect = to >>>>> with a web browser to see the debug messages. >>>>>=20 >>>>> The interface is still pretty beta, but I'm going to start using = it >>>>> in the Signpost code and see how useful it is. I hope to add more >>>>> live progress bars using js_of_ocaml soon! >>>>>=20 >>>>> -anil >>>>>=20 >>>>=20 >>>>=20 >>>=20 >>=20 >=20 From thomas.gazagnaire@gmail.com Sat Sep 22 08:12:55 2012 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TFJtX-0003w5-CS (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 22 Sep 2012 08:12:55 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.2 from SpamAssassin-3.3.2-1388377 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.179 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 0.9 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-we0-f179.google.com ([74.125.82.179]:37833) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1TFJtW-0003I0-sp (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 22 Sep 2012 08:12:55 +0100 Received: by weyx10 with SMTP id x10so2212695wey.38 for ; Sat, 22 Sep 2012 00:12:54 -0700 (PDT) Received: by 10.216.116.73 with SMTP id f51mr4203400weh.50.1348297974750; Sat, 22 Sep 2012 00:12:54 -0700 (PDT) Received: from [192.168.0.14] (gou06-3-88-170-165-56.fbx.proxad.net. [88.170.165.56]) by mx.google.com with ESMTPS id ct3sm1938894wib.5.2012.09.22.00.12.52 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 22 Sep 2012 00:12:53 -0700 (PDT) Subject: Re: correct cohttp opamconstraints Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: Date: Sat, 22 Sep 2012 09:12:51 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <01F62F9A-17A3-4A96-8DDE-25AAAC73422E@gmail.com> References: To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1084) Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Sep 2012 07:12:55 -0000 X-Keywords: X-UID: 590 Status: O Content-Length: 1046 Lines: 31 > depends: [ "ocamlfind" "re" "uri" {>=3D"1.3.2"} "ounit" ] > depopts: [ "async" {=3D"108.00.02"} "lwt" {>=3D"2.4.1"} "mirage-net" ] >=20 > However, cohttp requires Lwt AND SSL to be installed, or it fails = (since SSL is not yet optional). >=20 > Is there any way to express this in the CNF? I tried: Currently this not possible (as depopts are OR constraints only), but I = guess I could add that in 0.8 if you open an issue for that :-) > depopts: [ "async" {=3D"108.00.02"} ("lwt" {>=3D"2.4.1"} "ssl") = "mirage-net" ] I think I will go with the following syntax: depopts: [ "async" {=3D"108.00.02"} ("lwt" {>=3D"2.4.1"} & "ssl") = "mirage-net" ] as this is kind of dual with 'depends' where you express AND = constraints, and use '|' for OR. The other solution is to put the constraints in the 'depends' field = directly: depends: [ "ocamlfind" "re" "uri" {>=3D"1.3.2"} "ounit" ("async" = {=3D"108.00.02"} | ("lwt" {>=3D"2.4.1"} "ssl") | "mirage-net") ] (not sure that nested constraints currently work correctly though). -- Thomas From anil@recoil.org Thu Sep 27 04:11:23 2012 Received: from ppsw-43.csi.cam.ac.uk ([131.111.8.143]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TH4VX-0004XY-Hq (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 27 Sep 2012 04:11:23 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1389758 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:28403 helo=dark.recoil.org) by ppsw-43.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1TH4VW-0003qq-o5 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 27 Sep 2012 04:11:23 +0100 Received: (qmail 29387 invoked by uid 634); 27 Sep 2012 03:11:22 -0000 Received: from c-24-5-76-252.hsd1.ca.comcast.net (HELO [192.168.1.118]) (24.5.76.252) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 27 Sep 2012 04:11:22 +0100 From: Anil Madhavapeddy Content-Type: multipart/alternative; boundary="Apple-Mail=_3F642822-6F1A-4827-9321-1257C061ED62" Subject: Fwd: [mirage-platform] [xen] switch to using the bigarray's C finaliser to free() page-aligned ... (#6) Date: Wed, 26 Sep 2012 20:11:20 -0700 References: To: "cl-mirage@lists.cam.ac.uk List" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) X-Mailer: Apple Mail (2.1498) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Sep 2012 03:11:23 -0000 X-Keywords: X-UID: 591 Status: O Content-Length: 7852 Lines: 189 --Apple-Mail=_3F642822-6F1A-4827-9321-1257C061ED62 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Thanks to a spot of low-level detective work from Balraj and Dave, the = interaction between reference-counted and garbage-collected IO pages = should now actually work properly! Haris, you may also run into this if using the Xen backend (it wont = affect the UNIX backend), so be sure to merge it in. -anil Begin forwarded message: > From: Dave Scott > Subject: [mirage-platform] [xen] switch to using the bigarray's C = finaliser to free() page-aligned ... (#6) > Date: 25 September 2012 15:58:26 EDT > To: mirage/mirage-platform > Reply-To: mirage/mirage-platform = >=20 > ...buffers >=20 > Using an ocaml finaliser on the bigarray value doesn't work because > sub-bigarrays are distinct ocaml values which share the underlying = data. > When the parent ocaml value is finalised the child bigarray may still > be in scope somewhere and the data accidentally scared. >=20 > Now we directly call memalign() which calls _xmalloc() which calls > alloc_pages() to allocate individual blocks and rely on the fact that > sub-bigarrays all share a common "proxy" object and the underlying = data > is not free()d until the refcount within the proxy drops to zero i.e. > all related bigarrays are unreachable. This replaces the (broken) = ocaml-level > freelist implementation with a (hopefully working) C-level one. >=20 > If we run out of memory and memalign() fails, we return None and allow > the ocaml world to run a Gc.compact() which might call enough = finalizers > to allow the request to be satisfied. >=20 > This has only been superficially tested -- I need to give it more of a = hammering. >=20 > You can merge this Pull Request by running: >=20 > git pull https://github.com/djs55/mirage-platform fix-bigarray > Or view, comment on, or merge it at: >=20 > https://github.com/mirage/mirage-platform/pull/6 >=20 > Commit Summary >=20 > [xen] switch to using the bigarray's C finaliser to free() page-align=85= > File Changes >=20 > M xen/lib/io_page.ml (45) > M xen/runtime/kernel/page_stubs.c (35) > Patch Links >=20 > https://github.com/mirage/mirage-platform/pull/6.patch > https://github.com/mirage/mirage-platform/pull/6.diff > =97 > Reply to this email directly or view it on GitHub. >=20 >=20 --Apple-Mail=_3F642822-6F1A-4827-9321-1257C061ED62 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252
From: Dave Scott <notifications@github.com><= br>
Subject: [mirage-platform] [xen] switch to using the = bigarray's C finaliser to free() page-aligned ... = (#6)
Date: 25 September 2012 15:58:26 EDT
To: mirage/mirage-platform <mirage-platform@noreply= .github.com>

...buffers

<= p>Using an ocaml finaliser on the bigarray value doesn't work = because
sub-bigarrays are distinct ocaml values which share the underlying = data.
When the parent ocaml value is finalised the child bigarray may = still
be in scope somewhere and the data accidentally scared.

Now we = directly call memalign() which calls _xmalloc() which calls
alloc_pages() to allocate individual blocks and rely on the fact = that
sub-bigarrays all share a common "proxy" object and the underlying = data
is not free()d until the refcount within the proxy drops to zero = i.e.
all related bigarrays are unreachable. This replaces the (broken) = ocaml-level
freelist implementation with a (hopefully working) C-level one.

If = we run out of memory and memalign() fails, we return None and allow
the ocaml world to run a Gc.compact() which might call enough = finalizers
to allow the request to be satisfied.

This has only been = superficially tested -- I need to give it more of a hammering.

You can merge this Pull Request by running:

  git pull https://github.com/djs55=
/mirage-platform fix-bigarray

Or view, comment on, or merge = it at:

  https://github.c= om/mirage/mirage-platform/pull/6

Commit Summary

  • [xen] switch to using the bigarray's C finaliser to free() = page-align=85

File Changes

  • M xen/lib/io_page.ml (45)
  • M xen/runtime/kernel/page_stubs.c (35)

Patch Links

=97
Reply to this email directly or view it on = GitHub.


= --Apple-Mail=_3F642822-6F1A-4827-9321-1257C061ED62-- From sdiris@gmail.com Fri Sep 28 10:37:02 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1THX0I-00040I-P7 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 28 Sep 2012 10:37:02 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.2 from SpamAssassin-3.3.2-1390892 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.179 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (sdiris[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 0.9 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-wi0-f179.google.com ([209.85.212.179]:37855) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1THX0H-00022w-SQ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 28 Sep 2012 10:37:02 +0100 Received: by wibhq7 with SMTP id hq7so2504939wib.2 for ; Fri, 28 Sep 2012 02:37:01 -0700 (PDT) Received: by 10.180.87.34 with SMTP id u2mr2661362wiz.3.1348825021548; Fri, 28 Sep 2012 02:37:01 -0700 (PDT) Received: from YimingZhangPC (c072.al.cl.cam.ac.uk. [128.232.110.72]) by mx.google.com with ESMTPS id cn6sm9129635wib.9.2012.09.28.02.37.00 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 28 Sep 2012 02:37:00 -0700 (PDT) From: "Yiming Zhang" To: Subject: About Mirage Implementation Date: Fri, 28 Sep 2012 10:37:03 +0100 Message-ID: <001501cd9d5c$d1d37260$757a5720$@gmail.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0016_01CD9D65.339876A0" X-Mailer: Microsoft Outlook 14.0 Thread-Index: Ac2dWvujr2UDSy/WRBmhuD6p23RrYA== Content-Language: zh-cn X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Sep 2012 09:37:02 -0000 X-Keywords: X-UID: 592 Status: O Content-Length: 3825 Lines: 112 This is a multipart message in MIME format. ------=_NextPart_000_0016_01CD9D65.339876A0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit I want to contribute to the Mirage project by implementing a RAM-based KV library (like a better memcached) for it. However, it seems that it is very difficult to start. For the new modular version, the documents (in http://www.openmirage.org/wiki/hello-world) seem obsolete; for the old monolithic version, even the webpage (http://tutorial.openmirage.org/#1) cannot be opened! Can anybody help me figure out what to do in the first step? Regards, Yiming ------=_NextPart_000_0016_01CD9D65.339876A0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I want = to contribute to the Mirage project by implementing a RAM-based KV = library (like a better memcached) for it. However, it seems that it is = very difficult to start. For the new modular version, the documents (in = http://www.openmirage= .org/wiki/hello-world) seem obsolete; for the old monolithic = version, even the webpage (http://tutorial.openmirage.org= /#1) cannot be opened! Can anybody help me figure out what to do in = the first step?

 

Regards,

Yiming

------=_NextPart_000_0016_01CD9D65.339876A0-- From jac22@cl.cam.ac.uk Sat Sep 29 09:20:48 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1THsI4-0006NE-5F (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 29 Sep 2012 09:20:48 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1391368 * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP * -0.8 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mta0.cl.cam.ac.uk ([128.232.25.20]:49919) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1THsI3-0001te-YL (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 29 Sep 2012 09:20:48 +0100 Received: from ramsey.cl.cam.ac.uk ([128.232.1.219] helo=cl.cam.ac.uk) by mta0.cl.cam.ac.uk with esmtp (Exim 4.63) (envelope-from ) id 1THsI3-0005PX-G3 for cl-mirage@lists.cam.ac.uk; Sat, 29 Sep 2012 09:20:47 +0100 to: Mirage List Subject: Re: paper from nec/ucl along mirage lines In-reply-to: References: Comments: In-reply-to Jon Crowcroft message dated "Sat, 29 Sep 2012 08:52:33 +0100." MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <30024.1348906847.1@cl.cam.ac.uk> Date: Sat, 29 Sep 2012 09:20:47 +0100 From: Jon Crowcroft Message-Id: X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Sep 2012 08:20:48 -0000 X-Keywords: X-UID: 594 Status: O Content-Length: 639 Lines: 24 well derrrrrr- of course I had meant this to be local only distribution and now i've blown the authors anonimity w.r.t future submission and disclosed work in progress - please please delete/ignore/forget you saw my unethical dist of this and forgive me... for waste of bandwidth/disk too In missive , Jon Crowcroft typed: >> >>this paper didn't make it into hotnets....please dont fwd.. >> >>--0016e6dab04d8109b204cad274d5 >>Content-Type: text/html; charset=ISO-8859-1 >> >>this paper didn't make it into hotnets....please dont fwd.. cheers jon From anil@recoil.org Sat Sep 29 20:01:58 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TI2IY-00073f-S1 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 29 Sep 2012 20:01:58 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1391368 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:25321 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1TI2IY-0004PY-QK (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 29 Sep 2012 20:01:58 +0100 Received: (qmail 6417 invoked by uid 634); 29 Sep 2012 19:01:57 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from 202.sub-166-250-37.myvzw.com (HELO [192.168.1.118]) (166.250.37.202) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sat, 29 Sep 2012 20:01:56 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) Subject: Re: dns/crypto-keys/opam From: Anil Madhavapeddy In-Reply-To: <43B23F64-5332-4861-A6E3-4D10398C7D2F@recoil.org> Date: Sat, 29 Sep 2012 12:01:53 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <86CB2060-2D16-41E1-8CBE-08B7D37E45E7@recoil.org> <20120716154211.GA3121@cr409-ThinkPad-T60p> <43B23F64-5332-4861-A6E3-4D10398C7D2F@recoil.org> To: Haris Rotsos X-Mailer: Apple Mail (2.1498) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Sep 2012 19:01:58 -0000 X-Keywords: X-UID: 595 Status: O Content-Length: 1643 Lines: 38 On 16 Jul 2012, at 08:56, Anil Madhavapeddy wrote: >=20 > On 16 Jul 2012, at 16:42, Haris Rotsos wrote: >=20 >> On Mon, Jul 16, 2012 at 01:59:38PM +0100, Anil Madhavapeddy wrote: >>> This one's mainly for Mort and Haris. I'm working my way through = all the libraries required to compile Signposts, and put stable versions = up on OPAM. >>>=20 >>> I've taken all the forks of ocaml-dns and merged them (with the = latest version using cstruct) into http://github.com/mirage/ocaml-dns. = Mort, there seem to be a lot of commented out conversion functions in = the interfaces (packet.mli). Are these temporary or do you plan on = putting them back at some point? I've done some more hacking on the DNS library and released 0.5.0. Main = improvements are in the DNS resolver code. I resurrected some 2005 code that Dave wrote for DNS resolution, so the = Lwt Dns_resolver module can now parse resolv.conf files, or just take a = static server to point at. The interface makes fancy use of first-class = modules; I couldn't resist :-) https://github.com/avsm/ocaml-dns/blob/master/lwt/dns_resolver.ml https://github.com/avsm/ocaml-dns/blob/master/lwt/dns_resolver.mli Also, I put in an 'mldig' command which apes 'dig' in terms of output = format. It's complete enough to do most lookups, but still needs some = more work to get the more obscure bits of dig working. Note that you need latest cstruct-0.5.1 here, as I added string_to_enum = and enum_to_string to make it easier to do command line parsing of all = the various cenums used in the library. So Mort, how goes cstruct vs bitstring benchmarking? :-) -a= From anil@recoil.org Sat Sep 29 20:07:57 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TI2OL-00076K-CQ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 29 Sep 2012 20:07:57 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1391368 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:17346 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1TI2OK-0005HB-SL (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 29 Sep 2012 20:07:57 +0100 Received: (qmail 23857 invoked by uid 634); 29 Sep 2012 19:07:56 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from 202.sub-166-250-37.myvzw.com (HELO [192.168.1.118]) (166.250.37.202) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sat, 29 Sep 2012 20:07:56 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) Subject: Re: About Mirage Implementation From: Anil Madhavapeddy In-Reply-To: <001501cd9d5c$d1d37260$757a5720$@gmail.com> Date: Sat, 29 Sep 2012 12:07:52 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <001501cd9d5c$d1d37260$757a5720$@gmail.com> To: "Yiming Zhang" X-Mailer: Apple Mail (2.1498) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Sep 2012 19:07:57 -0000 X-Keywords: X-UID: 596 Status: O Content-Length: 1893 Lines: 43 Hi Yiming, The tutorial is indeed down (apparently due to an EC2 power cut that = took out the VM). However, the build tools are indeed quite different = now, due to us switching Mirage to a (significantly simpler and more = robust) package manager. This package manager works with normal OCaml = libraries, and so is generally a more sustainable long-term solution. So there are two options: - you can use the old monolithic repository, and clone = mirage/mirage-tutorial (but roll it back to last year by reverting the = last commit). This will give you a local copy of the tutorial, which = you can access on port 80. - I'm planning to port the hello world examples on mirage-www next week. = In the meanwhile, you can follow the OPAM instructions for mirage-www = to get it up and running, as the website uses the latest interfaces. You can also make a lot of progress without needing Mirage at all. I'd = suggest using Lwt, the Lwt_bytes module, and the cstruct library to = build a UNIX userspace version quickly. You can recompile this pretty = easily to a Mirage kernel if you keep the core logic fairly pure and = functional. Happy to advise about how that works (or you might want to = find someone locally in Cambridge this month until I return in November, = such as Balraj or Raphael who know who this works). -anil On 28 Sep 2012, at 02:37, "Yiming Zhang" wrote: > I want to contribute to the Mirage project by implementing a RAM-based = KV library (like a better memcached) for it. However, it seems that it = is very difficult to start. For the new modular version, the documents = (in http://www.openmirage.org/wiki/hello-world) seem obsolete; for the = old monolithic version, even the webpage = (http://tutorial.openmirage.org/#1) cannot be opened! Can anybody help = me figure out what to do in the first step? > =20 > Regards, > Yiming From Richard.Mortier@nottingham.ac.uk Sat Sep 29 20:09:58 2012 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TI2QI-00076w-8d (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 29 Sep 2012 20:09:58 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.6 from SpamAssassin-3.3.2-1391368 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from ixe-mta-19-tx.emailfiltering.com ([194.116.198.150]:56537 helo=ixe-mta-19.emailfiltering.com) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1TI2QC-0007x3-Y5 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 29 Sep 2012 20:09:58 +0100 Received: from smtp3.nottingham.ac.uk ([128.243.44.55]) by ixe-mta-19.emailfiltering.com with emfmta (version 4.8.5.104) by TLS id 2383372984 ;ddf4fbe67f60f567; Sat, 29 Sep 2012 20:09:06 +0100 Received: from uiwexhub01.ad.nottingham.ac.uk ([128.243.15.133]) by smtp3.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1TI2PR-0000pi-HC; Sat, 29 Sep 2012 20:09:05 +0100 Received: from EXCHANGE1.ad.nottingham.ac.uk ([fe80::7962:f868:e6ee:6267]) by UIWEXHUB01.ad.nottingham.ac.uk ([2002:80f3:f85::80f3:f85]) with mapi; Sat, 29 Sep 2012 20:08:45 +0100 From: Richard Mortier To: "anil@recoil.org" , "cr409@cl.cam.ac.uk" Date: Sat, 29 Sep 2012 20:08:45 +0100 Subject: Re: dns/crypto-keys/opam Thread-Topic: dns/crypto-keys/opam Thread-Index: Ac2edOiQ+Ep3TpbzSgO4vh0cl9WBbAAAPBIB Message-ID: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "richard.mortier@nottingham.ac.uk" , "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Sep 2012 19:09:58 -0000 X-Keywords: X-UID: 597 Status: O Content-Length: 3637 Lines: 74 Blocked at the moment on the illegal instruction exception I told you= about... :) proposals and marking done now though (bar one late submission= ) so should get to it monday.

Sent fro= m Samsung Mobile


Anil Madhavapeddy <anil@recoil.org&g= t; wrote:


=
=0D=0A

=0D=0AThis message and any attachment are intended solely= for the addressee and may=20=0D=0Acontain confidential information. If= you have received this message in error,=20=0D=0Aplease send it back t= o me, and immediately delete it. Please do not use,=20=0D=0Acopy or d= isclose the information contained in this message or in any attachment.= =20=0D=0AAny views or opinions expressed by the author of this email d= o not necessarily=20=0D=0Areflect the views of the University of Nottin= gham.=0D=0A

=0D=0A

=0D=0AThis message has been checked for viruses= but the contents of an attachment=0D=0Amay still contain software viru= ses which could damage your computer system:=0D=0Ayou are advised to pe= rform your own checks. Email communications with the=0D=0AUniversity of= Nottingham may be monitored as permitted by UK legislation.=0D=0A

=
On 16 Jul 2012, at 08:56, Anil Ma= dhavapeddy <anil@recoil.org> wrote:

>
> On 16 Jul 2012, at 16:42, Haris Rotsos wrote:
>
>> On Mon, Jul 16, 2012 at 01:59:38PM +0100, Anil Madhavapeddy wr= ote:
>>> This one's mainly for Mort and Haris.  I'm working my way= through all the libraries required to compile Signposts, and put stable ve= rsions up on OPAM.
>>>
>>> I've taken all the forks of ocaml-dns and merged them (with th= e latest version using cstruct) into http://github.com/mirage/ocaml-dns.   Mort, there se= em to be a lot of commented out conversion functions in the interfaces (pac= ket.mli).  Are these temporary or do you plan on putting them back at = some point?

I've done some more hacking on the DNS library and released 0.5.0.  Ma= in improvements are in the DNS resolver code.

I resurrected some 2005 code that Dave wrote for DNS resolution, so the Lwt= Dns_resolver module can now parse resolv.conf files, or just take a static= server to point at.  The interface makes fancy use of first-class mod= ules; I couldn't resist :-)
https://github.com/avsm/ocaml-dns/blob/master/lwt/dns_resolver.ml https://github.com/avsm/ocaml-dns/blob/master/lwt/dns_resolver.mli<= br>
Also, I put in an 'mldig' command which apes 'dig' in terms of output forma= t. It's complete enough to do most lookups, but still needs some more work = to get the more obscure bits of dig working.

Note that you need latest cstruct-0.5.1 here, as I added string_to_enum and= enum_to_string to make it easier to do command line parsing of all the var= ious cenums used in the library.

So Mort, how goes cstruct vs bitstring benchmarking? :-)

-a
 From anil@recoil.org Sun Sep 30 18:42:06 2012 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1TINWo-0001QG-5i (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 30 Sep 2012 18:42:06 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1391774 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:15601 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1TINWn-0004cO-RT (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 30 Sep 2012 18:42:06 +0100 Received: (qmail 17523 invoked by uid 634); 30 Sep 2012 17:42:05 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from c-24-5-76-252.hsd1.ca.comcast.net (HELO [192.168.1.118]) (24.5.76.252) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sun, 30 Sep 2012 18:42:04 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: ORM working in the toplevel Message-Id: Date: Sun, 30 Sep 2012 10:42:04 -0700 To: " List" Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) X-Mailer: Apple Mail (2.1498) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Sep 2012 17:42:06 -0000 X-Keywords: X-UID: 598 Status: O Content-Length: 1756 Lines: 47 I did a refresh of the SQL ORM type extension that Thomas and I wrote a = few years ago, and it's pretty cool on the toplevel! Once Thomas merges = the pull request for the new versions on OPAM [3], you can do "opam = install orm utop", and with this in your ~/.ocamlinit: #use "topfind" #camlp4o #thread Try this in "utop": utop $ #require "orm.syntax";; utop $ type t =3D { foo: int; bar: string } with orm;; utop $ let db =3D t_init "test.db";; val db : (t, [ `RW ]) Orm.Db.t =3D }=E2=94=80utop $ t_save db { foo=3D1; bar=3D"one" };; - : unit =3D () }=E2=94=80utop $ t_save db { foo=3D2; bar=3D"two" };; - : unit =3D () }=E2=94=80utop $ t_get db;; - : Cache_t.elt list =3D [{foo =3D 1; bar =3D "one"}; {foo =3D 2; bar =3D = "two"}] }=E2=94=80utop $ t_get ~bar:(`Contains "t") db;; - : Cache_t.elt list =3D [] }=E2=94=80utop $ t_get ~bar:(`Contains "two") db;; - : Cache_t.elt list =3D [] }=E2=94=80utop $ t_get ~bar:(`Eq "two") db;; - : Cache_t.elt list =3D [{foo =3D 2; bar =3D "two"}] utop $ t_get ~foo:(`Ge 0) db;; - : Cache_t.elt list =3D [{foo =3D 1; bar =3D "one"}; {foo =3D 2; bar =3D = "two"}] }=E2=94=80utop $ t_get ~foo:(`Ge 1) db;; - : Cache_t.elt list =3D [{foo =3D 2; bar =3D "two"}] I can see a few improvements to do, such as adding a signature generator = [1] to make the types more succinct (i.e. Cache_t.elt shouldnt be = exposed), and also new database backends (some NoSQL ones might be good, = perhaps to Arakoon). Also Thomas, I notice that the `Contains query isn't working as = expected. Is it meant to be a substring operation? See its use above.=20 [1] https://github.com/mirage/orm/issues/7 [2] https://github.com/mirage/orm/issues/6 [3] https://github.com/OCamlPro/opam-repository/pull/96=