From anil@recoil.org Mon Jul 18 17:20:28 2011 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1QiqYW-0006aG-NF (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Jul 2011 17:20:28 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1147555 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:31793 helo=dark.recoil.org) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp (csa=unknown) id 1QiqYV-0004vm-Dy (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Jul 2011 17:20:28 +0100 Received: (qmail 15237 invoked by uid 634); 18 Jul 2011 16:20:27 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.4 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.167]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.83) with ESMTPA; Mon, 18 Jul 2011 17:20:26 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Welcome to new Mirage developers Date: Mon, 18 Jul 2011 17:20:24 +0100 Message-Id: <7BDD9035-25D4-4EFD-8063-ECBC2334403E@recoil.org> To: cl-mirage@lists.cam.ac.uk Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jul 2011 16:20:28 -0000 X-IMAPbase: 1349086851 598 X-Keywords: X-UID: 1 Status: O Content-Length: 2344 Lines: 48 Hi folks, Mirage development has recently escaped from my little lab and is = including more people, so I just wanted to send out an introduction mail = to say who everyone is. - Richard Mortier is a lecturer at Nottingham, and he's been hacking on = the DNS stack and OpenFlow support, among other things. - Dave Scott is a developer of the Xen toolstack at Citrix, and is = interested in using Mirage for specialised Xen stub domains (e.g. a = block-level proxy) - Balraj Singh is a visitor in Cambridge for the next while, and will = look at various aspects (possibly Ensemble integration for distributed = communication) - =C9ireann Leverett is starting a PhD in Cambridge in the new year, and = is looking at DNS and DNSSEC integration=20 - Thomas Gazagnaire is the author of the dynamic typing extension, = Caml-on-the-Web and many other bits of Mirage. He's working at OCamlPro = doing consultancy at the moment. - Raphael Proust will join the Lab as an intern from next week, looking = at integration of a cool new linear-typing extension for the network = stack (called LinearML), and possibly ocsigen/js_of_ocaml support. The focus for this summer is to get a developer preview out with decent = documentation, as Richard, David and I will be doing a tutorial on = Mirage ("building a functional OS") at ICFP in late September [1], so = this gives us a nice relaxed deadline over the summer to tidy up and = document much of the existing code. I've just merged in the last big change to the core framework: switch = from the MPL domain specific language, over to Richard Jones' bitstring = syntax extension. This has pros and cons, but overall I'm much happier = that I don't have to maintain MPL any more, and we can use the native = OCaml 'string' type for packet parsing (old hands such as Thomas will = appreciate that Istrings have now disappeared). Anyway, we had a productive day with Balraj and Eireann both getting = Mirage compiled and running, and I've also got a couple of test boxes = running Xen all set up in the Lab now, for use to use as we desire. = This list is intended for day-to-day questions and discussion and is = open and archived; if the volume gets too high (which is doubtful!), we = can open up smaller sub-lists. cheers Anil [1] http://cufp.org/conference/sessions/2011/building-functional-os= From epl25@hermes.cam.ac.uk Tue Jul 26 11:23:06 2011 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Qlen4-00071j-Ew (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 11:23:06 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from hermes-2.csi.cam.ac.uk ([131.111.8.54]:52383) by ppsw-52.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.159]:25) with esmtpa (EXTERNAL:epl25) id 1Qlen4-0003LW-E7 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 11:23:06 +0100 Received: from prayer by hermes-2.csi.cam.ac.uk (hermes.cam.ac.uk) with local (PRAYER:epl25) id 1Qlen4-0004IG-Bs (Exim 4.67) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 11:23:06 +0100 Received: from [128.232.9.136] by webmail.hermes.cam.ac.uk with HTTP (Prayer-1.3.4); 26 Jul 2011 11:23:06 +0100 Date: 26 Jul 2011 11:23:06 +0100 From: =?ISO-8859-1?Q?=C9ireann_Leverett?= To: cl-mirage@lists.cam.ac.uk Subject: Simple DNS Record A Message-ID: X-Mailer: Prayer v1.3.4 Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset=ISO-8859-1 Sender: "E.P. Leverett" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 10:23:06 -0000 X-Keywords: X-UID: 2 Status: O Content-Length: 964 Lines: 29 Hello all, I'm looking into DNS resolution in Mirage, and learning OCAML at the same time. So forgive me if I'm a bit slow finding my way around in the code base. I (like most people) have the 'magic version' of dns in my head, where I ask for a record and get a reply. Time to change that! So to get started here's my model of an iterative DNS A record request: localhost sends a udp request on port 53 to Then one of two things happens: 1) localhost gets a response on port 53 from with: IP address/s | not found 2) Timeout So a basic resolver needs a timeout, a nameserver (initially hardcoded, but later from configuration), and to be able to build the structure of the request and parse the response structure. I'm looking at the bitstring-v-native code for inspiration, here and functions I can use. If this is the model for the A record then I'll flesh this into more meaningful pseudo code. Thoughts? Eireann From avsm@dark.recoil.org Tue Jul 26 11:44:15 2011 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Qlf7X-0007XG-T8 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 11:44:15 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1150583 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:35955 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp (csa=unknown) id 1Qlf7X-00053A-Qe (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 11:44:15 +0100 Received: (qmail 7298 invoked by uid 10000); 26 Jul 2011 10:44:15 -0000 Date: Tue, 26 Jul 2011 11:44:15 +0100 From: Anil Madhavapeddy To: ?ireann Leverett Subject: Re: Simple DNS Record A Message-ID: <20110726104414.GA13896@dark.recoil.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 10:44:16 -0000 X-Keywords: X-UID: 3 Status: O Content-Length: 2603 Lines: 67 On Tue, Jul 26, 2011 at 11:23:06AM +0100, ?ireann Leverett wrote: > Hello all, > > I'm looking into DNS resolution in Mirage, and learning OCAML at the same > time. So forgive me if I'm a bit slow finding my way around in the code > base. > > I (like most people) have the 'magic version' of dns in my head, where I > ask for a record and get a reply. Time to change that! > > So to get started here's my model of an iterative DNS A record request: > > localhost sends a udp request on port 53 to > Then one of two things happens: > 1) localhost gets a response on port 53 from > with: IP address/s | not found > 2) Timeout Almost right. You're describing a 'recursive resolver', since it takes a client request, does the full resolution, and returns back the response. An iterative resolver, on the other hand, would only point to the next server in the DNS chain and return just that next result. So a recursive resolver is what responds to a typical client request, by performing iterative resolution itself (or pointing to yet another resolver). Here's a way to get familiar with this concept, using the 'dig' tool: First, determine the root DNS servers on the Internet by: $ dig This will return all the root name servers as NS records. These point to the top-level domains such as org and com. $ dig +norecurse @a.root-servers.net dark.recoil.org At this point, do an iterative request to the root name server asking for 'dark.recoil.org'. It will not give an immediate reply, but will instead point to the next name server for .org You can continue this exercise by then requesting the .org name server: $ dig +norecurse @a0.org.afilias-nst.info dark.recoil.org ...and so on until you get the IP address for dark.recoil.org The recursive resolver for clients will perform this iteration process itself, and possibly caching results (depending on the TTL for responses). > > So a basic resolver needs a timeout, a nameserver (initially hardcoded, but > later from configuration), and to be able to build the structure of the > request and parse the response structure. I'm looking at the > bitstring-v-native code for inspiration, here and functions I can use. > > If this is the model for the A record then I'll flesh this into more > meaningful pseudo code. Yeah, keep fleshing it out and take notes on precisely what the process is. Also see: http://www.umlgraph.org/doc/dns.html For a message sequence chart of the various queries from a system level. -- Anil Madhavapeddy http://anil.recoil.org From epl25@hermes.cam.ac.uk Tue Jul 26 12:48:06 2011 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Qlg7K-0001jY-Qu (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 12:48:06 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from hermes-2.csi.cam.ac.uk ([131.111.8.54]:51483) by ppsw-52.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.159]:25) with esmtpa (EXTERNAL:epl25) id 1Qlg7K-0003bG-F3 (Exim 4.72) (return-path ); Tue, 26 Jul 2011 12:48:06 +0100 Received: from prayer by hermes-2.csi.cam.ac.uk (hermes.cam.ac.uk) with local (PRAYER:epl25) id 1Qlg7K-0000i5-L9 (Exim 4.67) (return-path ); Tue, 26 Jul 2011 12:48:06 +0100 Received: from [128.232.9.136] by webmail.hermes.cam.ac.uk with HTTP (Prayer-1.3.4); 26 Jul 2011 12:48:06 +0100 Date: 26 Jul 2011 12:48:06 +0100 From: =?ISO-8859-1?Q?=C9ireann_Leverett?= To: Anil Madhavapeddy Subject: Re: Simple DNS Record A Message-ID: In-Reply-To: <20110726104414.GA13896@dark.recoil.org> References: <20110726104414.GA13896@dark.recoil.org> X-Mailer: Prayer v1.3.4 Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset=ISO-8859-1 Sender: "E.P. Leverett" Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 11:48:06 -0000 X-Keywords: X-UID: 4 Status: O Content-Length: 418 Lines: 14 Ah! I had that 'recursive' model wrong. I thought each server contacted the next up the chain, now I can see all those little inefficient loops in a recursive resolver...and hence motivating the Main Name Service thesis. OK, I'll get a socket opening and closing on a timeout for some coding practice, but keep reading the DNS queries message diagrams to understand the resolver pseudo-code mainly. Eireann From robert.watson@cl.cam.ac.uk Tue Jul 26 14:37:30 2011 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1QlhpC-00057A-UO (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 14:37:30 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1150583 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from cyrus.watson.org ([65.122.17.42]:61676) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp (csa=unknown) id 1QlhpC-0000DU-Y3 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 14:37:30 +0100 Received: from c0143.aw.cl.cam.ac.uk (c0143.aw.cl.cam.ac.uk [128.232.100.143]) by cyrus.watson.org (Postfix) with ESMTPSA id 52DE246B03; Tue, 26 Jul 2011 09:37:26 -0400 (EDT) Subject: Re: Simple DNS Record A Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=iso-8859-1 From: "Robert N. M. Watson" In-Reply-To: Date: Tue, 26 Jul 2011 14:37:24 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <24E6F503-F7A6-4A47-AEB6-12A904EEDDC2@cl.cam.ac.uk> References: <20110726104414.GA13896@dark.recoil.org> To: =?iso-8859-1?Q?=C9ireann_Leverett?= X-Mailer: Apple Mail (2.1084) Cc: cl-mirage@lists.cam.ac.uk, Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 13:37:31 -0000 X-Keywords: X-UID: 5 Status: O Content-Length: 1536 Lines: 38 On 26 Jul 2011, at 12:48, =C9ireann Leverett wrote: > Ah! I had that 'recursive' model wrong. I thought each server = contacted the next up the chain, now I can see all those little = inefficient loops in a recursive resolver...and hence motivating the = Main Name Service thesis. >=20 > OK, I'll get a socket opening and closing on a timeout for some coding = practice, but keep reading the DNS queries message diagrams to = understand the resolver pseudo-code mainly. Can I suggest some practical experimentation here? There's nothing like = getting your hands dirty :-). The first tool you should know about is dig(1), which is a command line = DNS query tool. You can ask it to do recursive/non-recursive lookups, = get traces of communication, etc. The second is tcpdump, which I feel certain you've used before, but = quite useful for seeing what actually goes out. Wireshark might have = better packet parsing, and you can also write out dumps with tcpdump and = read them with Wireshark, etc. The third is bind -- if you start up a local name server on your box, = you can query its cache, trace it's transactions, and so on. (If for some reason this isn't possible on your box, perhaps some = argument about security or something, we can work out an alternative = system for you to play with). All of these will be useful when you're experimenting with DNS in = Mirage, since you'll want to trace its interactions with the name = server, compare queries you construct with what it comes up with, and so = on. Robert= From crowcroft@gmail.com Tue Jul 26 14:39:38 2011 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1QlhrG-00059b-TN (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 14:39:38 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.6 from SpamAssassin-3.3.2-1150583 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.218.51 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (crowcroft[at]gmail.com) * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-yi0-f51.google.com ([209.85.218.51]:35590) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp (csa=unknown) id 1QlhrF-0000Xt-SK (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 14:39:38 +0100 Received: by yib2 with SMTP id 2so249032yib.38 for ; Tue, 26 Jul 2011 06:39:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.101.98.14 with SMTP id a14mr5181511anm.104.1311687576909; Tue, 26 Jul 2011 06:39:36 -0700 (PDT) Sender: crowcroft@gmail.com Received: by 10.101.45.5 with HTTP; Tue, 26 Jul 2011 06:39:36 -0700 (PDT) Received: by 10.101.45.5 with HTTP; Tue, 26 Jul 2011 06:39:36 -0700 (PDT) In-Reply-To: <24E6F503-F7A6-4A47-AEB6-12A904EEDDC2@cl.cam.ac.uk> References: <20110726104414.GA13896@dark.recoil.org> <24E6F503-F7A6-4A47-AEB6-12A904EEDDC2@cl.cam.ac.uk> Date: Tue, 26 Jul 2011 14:39:36 +0100 X-Google-Sender-Auth: jQ3_oUeM3W6vhC-qSR_RVoUp8Bs Message-ID: Subject: Re: Simple DNS Record A From: Jon Crowcroft To: "Robert N. M. Watson" Content-Type: multipart/alternative; boundary=001636c934e013f75704a8f910e1 Cc: =?ISO-8859-1?Q?=C9ireann_Leverett?= , cl-mirage@lists.cam.ac.uk, Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 13:39:39 -0000 X-Keywords: X-UID: 6 Status: O Content-Length: 3965 Lines: 84 --001636c934e013f75704a8f910e1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable And see also rich sterns tcp IP illustrated volume 1&2 On 26 Jul 2011 15:37, "Robert N. M. Watson" wrote: > > On 26 Jul 2011, at 12:48, =C9ireann Leverett wrote: > >> Ah! I had that 'recursive' model wrong. I thought each server contacted the next up the chain, now I can see all those little inefficient loops in = a recursive resolver...and hence motivating the Main Name Service thesis. >> >> OK, I'll get a socket opening and closing on a timeout for some coding practice, but keep reading the DNS queries message diagrams to understand the resolver pseudo-code mainly. > > Can I suggest some practical experimentation here? There's nothing like getting your hands dirty :-). > > The first tool you should know about is dig(1), which is a command line DNS query tool. You can ask it to do recursive/non-recursive lookups, get traces of communication, etc. > > The second is tcpdump, which I feel certain you've used before, but quite useful for seeing what actually goes out. Wireshark might have better packe= t parsing, and you can also write out dumps with tcpdump and read them with Wireshark, etc. > > The third is bind -- if you start up a local name server on your box, you can query its cache, trace it's transactions, and so on. > > (If for some reason this isn't possible on your box, perhaps some argumen= t about security or something, we can work out an alternative system for you to play with). > > All of these will be useful when you're experimenting with DNS in Mirage, since you'll want to trace its interactions with the name server, compare queries you construct with what it comes up with, and so on. > > Robert --001636c934e013f75704a8f910e1 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

And see also rich sterns tcp IP illustrated volume 1&2

On 26 Jul 2011 15:37, "Robert N. M. Watson&= quot; <robert.watson@cl.ca= m.ac.uk> wrote:
>
> On 26 Jul 2011= , at 12:48, =C9ireann Leverett wrote:
>
>> Ah! I had that 'recursive' model wrong. I thought= each server contacted the next up the chain, now I can see all those littl= e inefficient loops in a recursive resolver...and hence motivating the Main= Name Service thesis.
>>
>> OK, I'll get a socket opening and closing on a ti= meout for some coding practice, but keep reading the DNS queries message di= agrams to understand the resolver pseudo-code mainly.
>
> Can = I suggest some practical experimentation here? There's nothing like get= ting your hands dirty :-).
>
> The first tool you should know about is dig(1), which is a co= mmand line DNS query tool. You can ask it to do recursive/non-recursive loo= kups, get traces of communication, etc.
>
> The second is tcpd= ump, which I feel certain you've used before, but quite useful for seei= ng what actually goes out. Wireshark might have better packet parsing, and = you can also write out dumps with tcpdump and read them with Wireshark, etc= .
>
> The third is bind -- if you start up a local name server on y= our box, you can query its cache, trace it's transactions, and so on.>
> (If for some reason this isn't possible on your box, pe= rhaps some argument about security or something, we can work out an alterna= tive system for you to play with).
>
> All of these will be useful when you're experimenting wit= h DNS in Mirage, since you'll want to trace its interactions with the n= ame server, compare queries you construct with what it comes up with, and s= o on.
>
> Robert
--001636c934e013f75704a8f910e1-- From epl25@hermes.cam.ac.uk Tue Jul 26 17:17:09 2011 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1QlkJh-00019w-1A (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 17:17:09 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from hermes-2.csi.cam.ac.uk ([131.111.8.54]:52502) by ppsw-41.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.156]:25) with esmtpa (EXTERNAL:epl25) id 1QlkJh-0002YQ-Pp (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 17:17:09 +0100 Received: from prayer by hermes-2.csi.cam.ac.uk (hermes.cam.ac.uk) with local (PRAYER:epl25) id 1QlkJg-0005T1-W7 (Exim 4.67) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 17:17:08 +0100 Received: from [128.232.9.136] by webmail.hermes.cam.ac.uk with HTTP (Prayer-1.3.4); 26 Jul 2011 17:17:08 +0100 Date: 26 Jul 2011 17:17:08 +0100 From: =?ISO-8859-1?Q?=C9ireann_Leverett?= To: cl-mirage@lists.cam.ac.uk Subject: dnspacket.ml Message-ID: X-Mailer: Prayer v1.3.4 Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset=ISO-8859-1 Sender: "E.P. Leverett" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 16:17:09 -0000 X-Keywords: X-UID: 7 Status: O Content-Length: 361 Lines: 14 DNS/OCaml gods, So for example one of the files I might be interested in linking in for a resolver would be dnspacket.ml which would help me parse dns replies. Do I do this using a statement within my resolver file like: Open Dnspacket;; and then link to the location of the Dnspacket.cma at compile time? Or is there a 'Mirage way' to do this? Eireann From thomas.gazagnaire@gmail.com Tue Jul 26 17:42:37 2011 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1QlkiL-0001Vk-2c (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 17:42:37 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.7 from SpamAssassin-3.3.2-1150583 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.51 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-ww0-f51.google.com ([74.125.82.51]:35683) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp (csa=unknown) id 1QlkiK-0001WX-rt (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 26 Jul 2011 17:42:37 +0100 Received: by wwj26 with SMTP id 26so543407wwj.20 for ; Tue, 26 Jul 2011 09:42:36 -0700 (PDT) Received: by 10.227.11.137 with SMTP id t9mr5458992wbt.9.1311698556455; Tue, 26 Jul 2011 09:42:36 -0700 (PDT) Received: from [192.168.0.10] (gou06-3-88-170-165-56.fbx.proxad.net [88.170.165.56]) by mx.google.com with ESMTPS id em16sm580619wbb.16.2011.07.26.09.42.33 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 26 Jul 2011 09:42:34 -0700 (PDT) Subject: Re: dnspacket.ml Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=iso-8859-1 From: Thomas Gazagnaire In-Reply-To: Date: Tue, 26 Jul 2011 18:42:32 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <9BD2559A-2856-4D3E-90E7-0DD32B5423DE@gmail.com> References: To: =?iso-8859-1?Q?=C9ireann_Leverett?= X-Mailer: Apple Mail (2.1084) Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 16:42:37 -0000 X-Keywords: X-UID: 8 Status: O Content-Length: 645 Lines: 34 I guess this file is packed into the Dns pack. So to access function = inside it, you can use either : --- open Dns.Dnspacket --- or simply: --- Dns.Dnspacket. --- On Jul 26, 2011, at 6:17 PM, =C9ireann Leverett wrote: > DNS/OCaml gods, >=20 > So for example one of the files I might be interested in linking in = for a resolver would be dnspacket.ml which would help me parse dns = replies. Do I do this using a statement within my resolver file like: >=20 > Open Dnspacket;; >=20 > and then link to the location of the Dnspacket.cma at compile time? >=20 > Or is there a 'Mirage way' to do this? >=20 > Eireann >=20 From epl25@hermes.cam.ac.uk Wed Jul 27 14:51:28 2011 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Qm4WG-0001Hc-Mz (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Jul 2011 14:51:28 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from hermes-2.csi.cam.ac.uk ([131.111.8.54]:52464) by ppsw-52.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.159]:25) with esmtpa (EXTERNAL:epl25) id 1Qm4WG-0004KU-F4 (Exim 4.72) (return-path ); Wed, 27 Jul 2011 14:51:28 +0100 Received: from prayer by hermes-2.csi.cam.ac.uk (hermes.cam.ac.uk) with local (PRAYER:epl25) id 1Qm4WG-0002Ix-LG (Exim 4.67) (return-path ); Wed, 27 Jul 2011 14:51:28 +0100 Received: from [128.232.9.136] by webmail.hermes.cam.ac.uk with HTTP (Prayer-1.3.4); 27 Jul 2011 14:51:28 +0100 Date: 27 Jul 2011 14:51:28 +0100 From: =?ISO-8859-1?Q?=C9ireann_Leverett?= To: Thomas Gazagnaire Subject: Re: dnspacket.ml Message-ID: In-Reply-To: <9BD2559A-2856-4D3E-90E7-0DD32B5423DE@gmail.com> References: <9BD2559A-2856-4D3E-90E7-0DD32B5423DE@gmail.com> X-Mailer: Prayer v1.3.4 Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Sender: "E.P. Leverett" Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 13:51:28 -0000 X-Keywords: X-UID: 9 Status: O Content-Length: 1313 Lines: 64 I'm guessing I should be using the interfaces instead of linking all files= =20 together (perhaps this is what you meant). So for DNS I can see eight=20 currently defined: server hashcons dnstrie dnsserver dnsrr dnsquery dnsloader dnslexer Which is now starting to make a lot more sense. However, can anyone tell me= =20 what .mly files are for? Also, does mirage generally use the ocamlc or ocamlopt compilation? The=20 book establishes the difference as primarily easilydebuggable&portable=20 -v-performance enhanced&native code. Is there any more to that story? Eireann On Jul 26 2011, Thomas Gazagnaire wrote: > I guess this file is packed into the Dns pack. So to access function=20 > inside it, you can use either : > >--- >open Dns.Dnspacket > > >--- > >or simply: > >--- >Dns.Dnspacket. >--- > > >On Jul 26, 2011, at 6:17 PM, =C9ireann Leverett wrote: > >> DNS/OCaml gods, >>=20 >> So for example one of the files I might be interested in linking in for= =20 >> a resolver would be dnspacket.ml which would help me parse dns replies.= =20 >> Do I do this using a statement within my resolver file like: >>=20 >> Open Dnspacket;; >>=20 >> and then link to the location of the Dnspacket.cma at compile time? >>=20 >> Or is there a 'Mirage way' to do this? >>=20 >> Eireann >>=20 > > From anil@recoil.org Wed Jul 27 15:00:39 2011 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Qm4f9-0001c3-MS (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Jul 2011 15:00:39 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1151020 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:9358 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp (csa=unknown) id 1Qm4f9-0003WL-ql (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Jul 2011 15:00:39 +0100 Received: (qmail 6799 invoked by uid 634); 27 Jul 2011 14:00:39 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.4 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from dyn898-202.eduroam.ic.ac.uk (HELO dyn898-202.eduroam.ic.ac.uk) (146.179.198.202) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.83) with ESMTPA; Wed, 27 Jul 2011 15:00:38 +0100 Subject: Re: dnspacket.ml Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=windows-1252 From: Anil Madhavapeddy In-Reply-To: Date: Wed, 27 Jul 2011 15:00:39 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <31CEACFA-1992-4477-99C1-D13FC503F5AF@recoil.org> References: <9BD2559A-2856-4D3E-90E7-0DD32B5423DE@gmail.com> To: =?iso-8859-1?Q?=C9ireann_Leverett?= X-Mailer: Apple Mail (2.1244.3) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk, Thomas Gazagnaire X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 14:00:39 -0000 X-Keywords: X-UID: 10 Status: O Content-Length: 1780 Lines: 54 [am in Imperial in meetings all day, so excuse delayed replies] On 27 Jul 2011, at 14:51, =C9ireann Leverett wrote: >=20 > I'm guessing I should be using the interfaces instead of linking all = files together (perhaps this is what you meant). So for DNS I can see = eight currently defined: >=20 > server > hashcons > dnstrie > dnsserver > dnsrr > dnsquery > dnsloader > dnslexer Right=85 these are packed together into the Dns module via the = 'dns.mlpack' file, which indicates that the modules within should be = packed under the Dns namespace (so Dns.Dnstrie for example). However, to start with, you could also add your resolver code into = client.ml, add Client to the dns.mlpack file, and not have to prepend = everything with Dns. See server.ml for an example of this. >=20 > Which is now starting to make a lot more sense. However, can anyone = tell me what .mly files are for? .mly files are ocamlyacc files for defining grammars. The ocamlyacc tool = is similar to the C yacc, except it outputs OCaml parser tables instead = of C ones. This is used in DNS to parse zone files, and the build system = automatically generates OCaml modules by calling ocamlyacc on the .mly = file. >=20 > Also, does mirage generally use the ocamlc or ocamlopt compilation? = The book establishes the difference as primarily = easilydebuggable&portable -v-performance enhanced&native code. Is there = any more to that story? >=20 It actually generates both. The programmer shouldn't be able to observe = the difference from within the OCaml program (it only makes a difference = for speed, in practise). Some backends use the byte code files to do = further code generation (e.g. to Javascript), but the Xen backend = exclusively uses the ocamlopt native code backend. -anil= From raphlalou@gmail.com Wed Jul 27 15:01:25 2011 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Qm4ft-0001ei-T9 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Jul 2011 15:01:25 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.7 from SpamAssassin-3.3.2-1151020 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.213.179 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (raphlalou[at]gmail.com) * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-yx0-f179.google.com ([209.85.213.179]:59468) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp (csa=unknown) id 1Qm4ft-0007Fo-Xc (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Jul 2011 15:01:25 +0100 Received: by yxm34 with SMTP id 34so815428yxm.38 for ; Wed, 27 Jul 2011 07:01:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.42.147.8 with SMTP id l8mr45628icv.496.1311775284559; Wed, 27 Jul 2011 07:01:24 -0700 (PDT) Received: by 10.42.169.134 with HTTP; Wed, 27 Jul 2011 07:01:24 -0700 (PDT) In-Reply-To: References: <9BD2559A-2856-4D3E-90E7-0DD32B5423DE@gmail.com> Date: Wed, 27 Jul 2011 15:01:24 +0100 Message-ID: Subject: Re: dnspacket.ml From: Raphael Proust To: =?UTF-8?Q?=C3=89ireann_Leverett?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 14:01:26 -0000 X-Keywords: X-UID: 11 Status: O Content-Length: 1786 Lines: 85 2011/7/27 =C3=89ireann Leverett : > > I'm guessing I should be using the interfaces instead of linking all file= s > together (perhaps this is what you meant). So for DNS I can see eight > currently defined: > > server > hashcons > dnstrie > dnsserver > dnsrr > dnsquery > dnsloader > dnslexer > > Which is now starting to make a lot more sense. However, can anyone tell = me > what .mly files are for? mly files are intended to generate parsers through the use of ocamlyacc. Similarly mll files can be processed by ocamllex to obtain lexers. > > Also, does mirage generally use the ocamlc or ocamlopt compilation? The b= ook > establishes the difference as primarily easilydebuggable&portable > -v-performance enhanced&native code. Is there any more to that story? AFAIK thee are no big differences besides those you mentioned. I know that the linking step can differ in certain cases. Also, bytecode is used as an intermediate step in the js_of_ocaml compiler. > > Eireann > > On Jul 26 2011, Thomas Gazagnaire wrote: > >> I guess this file is packed into the Dns pack. So to access function >> inside it, you can use either : >> >> --- >> open Dns.Dnspacket >> >> >> --- >> >> or simply: >> >> --- >> Dns.Dnspacket. >> --- >> >> >> On Jul 26, 2011, at 6:17 PM, =C3=89ireann Leverett wrote: >> >>> DNS/OCaml gods, >>> >>> So for example one of the files I might be interested in linking in for= a >>> resolver would be dnspacket.ml which would help me parse dns replies. D= o I >>> do this using a statement within my resolver file like: >>> >>> Open Dnspacket;; >>> >>> and then link to the location of the Dnspacket.cma at compile time? >>> >>> Or is there a 'Mirage way' to do this? >>> >>> Eireann >>> >> >> > > --=20 _______ Raphael From anil@recoil.org Wed Jul 27 16:36:44 2011 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Qm6A8-0004Yo-68 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Jul 2011 16:36:44 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1151020 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:25164 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp (csa=unknown) id 1Qm6A7-0007zu-sD (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Jul 2011 16:36:44 +0100 Received: (qmail 11658 invoked by uid 634); 27 Jul 2011 15:36:43 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.4 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from dyn898-202.eduroam.ic.ac.uk (HELO dyn898-202.eduroam.ic.ac.uk) (146.179.198.202) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.83) with ESMTPA; Wed, 27 Jul 2011 16:36:43 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: ocamldoc generation and packed files Date: Wed, 27 Jul 2011 16:36:43 +0100 Message-Id: <0296729D-1BEC-40D2-B491-F7098A07FEB4@recoil.org> To: cl-mirage@lists.cam.ac.uk Mime-Version: 1.0 (Apple Message framework v1244.3) X-Mailer: Apple Mail (2.1244.3) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 15:36:44 -0000 X-Keywords: X-UID: 12 Status: O Content-Length: 1372 Lines: 30 I'd really like to generate ocamldoc (HTML/PDF) of all the various = libraries so that it's easier to learn Mirage (and support editor = auto-completion, etc). The big problem is that ocamldoc doesn't support packed modules, and we = use packing quite extensively (in Net, Http, Block, etc). So I've hacked up an ocamlbuild target that concats together the = *source* ML files from an .mlpack and uses that to generate the ocamldoc = for the standard library, with one set of HTML files generated per = backend (Xen, Net-Direct, Net-Socket, and so on). The issue is that these rules are a little grim: everywhere where we = have a .mlpack file at the moment, we need to override that rule to = generate a concatenated ML file that is used for ocamldoc (but not for = actual compilation, since line numbers get lost since those aren't = preserved when converting from ML files into a single big one). Does anyone have a better solution for ocamldoc and packed files? All of = the grimness here is hidden away in the ocamlbuild rules, so I'm = inclined to just commit this patch, and perhaps see about adding -pack = support into ocamldoc at a later stage (there's an open bug in Mantis = somewhere).=20 Dave, do you use pack in XAPI, and/or ocamldoc? I wonder if everyone = else (like Core) also have their own swanky 'cat ML files into one' = script too... -anil= From thomas.gazagnaire@gmail.com Wed Jul 27 16:57:03 2011 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Qm6Tn-0004qd-2s (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Jul 2011 16:57:03 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.7 from SpamAssassin-3.3.2-1151020 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.51 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mail-ww0-f51.google.com ([74.125.82.51]:64010) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp (csa=unknown) id 1Qm6Tl-0005eP-RV (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Jul 2011 16:57:03 +0100 Received: by wwj26 with SMTP id 26so1149226wwj.20 for ; Wed, 27 Jul 2011 08:57:01 -0700 (PDT) Received: by 10.227.62.194 with SMTP id y2mr202341wbh.43.1311782221305; Wed, 27 Jul 2011 08:57:01 -0700 (PDT) Received: from [192.168.0.10] (gou06-3-88-170-165-56.fbx.proxad.net [88.170.165.56]) by mx.google.com with ESMTPS id fr7sm8473wbb.39.2011.07.27.08.56.59 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 27 Jul 2011 08:57:00 -0700 (PDT) Subject: Re: ocamldoc generation and packed files Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: <0296729D-1BEC-40D2-B491-F7098A07FEB4@recoil.org> Date: Wed, 27 Jul 2011 17:56:58 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <29701FD7-6824-40F1-8244-5D2A06F827B2@gmail.com> References: <0296729D-1BEC-40D2-B491-F7098A07FEB4@recoil.org> To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1084) Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 15:57:03 -0000 X-Keywords: X-UID: 13 Status: O Content-Length: 1780 Lines: 42 We have as well a program which pack together ML files (and is able to = functorize packs as well...) it is not released yet, but I guess we can = open-source it shortly. But I think overriding the default rule for ocamldoc+pack in ocamlbuild = is sufficient for now on so you should push your patch :-) Thomas On Jul 27, 2011, at 5:36 PM, Anil Madhavapeddy wrote: > I'd really like to generate ocamldoc (HTML/PDF) of all the various = libraries so that it's easier to learn Mirage (and support editor = auto-completion, etc). >=20 > The big problem is that ocamldoc doesn't support packed modules, and = we use packing quite extensively (in Net, Http, Block, etc). >=20 > So I've hacked up an ocamlbuild target that concats together the = *source* ML files from an .mlpack and uses that to generate the ocamldoc = for the standard library, with one set of HTML files generated per = backend (Xen, Net-Direct, Net-Socket, and so on). >=20 > The issue is that these rules are a little grim: everywhere where we = have a .mlpack file at the moment, we need to override that rule to = generate a concatenated ML file that is used for ocamldoc (but not for = actual compilation, since line numbers get lost since those aren't = preserved when converting from ML files into a single big one). >=20 > Does anyone have a better solution for ocamldoc and packed files? All = of the grimness here is hidden away in the ocamlbuild rules, so I'm = inclined to just commit this patch, and perhaps see about adding -pack = support into ocamldoc at a later stage (there's an open bug in Mantis = somewhere).=20 >=20 > Dave, do you use pack in XAPI, and/or ocamldoc? I wonder if everyone = else (like Core) also have their own swanky 'cat ML files into one' = script too... >=20 > -anil From anil@recoil.org Thu Jul 28 07:55:57 2011 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1QmKVg-0005Uz-W9 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Jul 2011 07:55:57 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1151385 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:16240 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp (csa=unknown) id 1QmKVg-00073i-Rh (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Jul 2011 07:55:56 +0100 Received: (qmail 13974 invoked by uid 634); 28 Jul 2011 06:55:56 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.4 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc2-cmbg9-0-0-cust410.5-4.cable.virginmedia.com (HELO [192.168.1.111]) (82.16.101.155) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.83) with ESMTPA; Thu, 28 Jul 2011 07:55:55 +0100 Subject: Re: ocamldoc generation and packed files Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=us-ascii From: Anil Madhavapeddy In-Reply-To: <29701FD7-6824-40F1-8244-5D2A06F827B2@gmail.com> Date: Thu, 28 Jul 2011 07:55:54 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <977F1FF7-4400-415E-9A00-C69715488676@recoil.org> References: <0296729D-1BEC-40D2-B491-F7098A07FEB4@recoil.org> <29701FD7-6824-40F1-8244-5D2A06F827B2@gmail.com> To: Thomas Gazagnaire X-Mailer: Apple Mail (2.1244.3) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 06:55:57 -0000 X-Keywords: X-UID: 14 Status: O Content-Length: 2420 Lines: 57 That tool will be very useful! I've committed the 'make doc' target. The = output is still pretty dirty as we don't use the documentation tags = properly, but I'll go through adding .mli files and adding proper = documentation on the more stable interfaces. It would be quite nice to eventually have a single documentation output = for all the backends, with an addition section saying 'only present in = Xen' or 'only present in Node'. I think that should be possible by = parsing the ocamldoc dump outputs, but something for the future! Anil On 27 Jul 2011, at 16:56, Thomas Gazagnaire wrote: > We have as well a program which pack together ML files (and is able to = functorize packs as well...) it is not released yet, but I guess we can = open-source it shortly. >=20 > But I think overriding the default rule for ocamldoc+pack in = ocamlbuild is sufficient for now on so you should push your patch :-) >=20 > Thomas >=20 > On Jul 27, 2011, at 5:36 PM, Anil Madhavapeddy wrote: >=20 >> I'd really like to generate ocamldoc (HTML/PDF) of all the various = libraries so that it's easier to learn Mirage (and support editor = auto-completion, etc). >>=20 >> The big problem is that ocamldoc doesn't support packed modules, and = we use packing quite extensively (in Net, Http, Block, etc). >>=20 >> So I've hacked up an ocamlbuild target that concats together the = *source* ML files from an .mlpack and uses that to generate the ocamldoc = for the standard library, with one set of HTML files generated per = backend (Xen, Net-Direct, Net-Socket, and so on). >>=20 >> The issue is that these rules are a little grim: everywhere where we = have a .mlpack file at the moment, we need to override that rule to = generate a concatenated ML file that is used for ocamldoc (but not for = actual compilation, since line numbers get lost since those aren't = preserved when converting from ML files into a single big one). >>=20 >> Does anyone have a better solution for ocamldoc and packed files? All = of the grimness here is hidden away in the ocamlbuild rules, so I'm = inclined to just commit this patch, and perhaps see about adding -pack = support into ocamldoc at a later stage (there's an open bug in Mantis = somewhere).=20 >>=20 >> Dave, do you use pack in XAPI, and/or ocamldoc? I wonder if everyone = else (like Core) also have their own swanky 'cat ML files into one' = script too... >>=20 >> -anil >=20 From anil@recoil.org Sun Jul 31 23:30:46 2011 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-1.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1QneX0-00071L-Go (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 31 Jul 2011 23:30:46 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.9 from SpamAssassin-3.3.2-1152433 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:28009 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp (csa=unknown) id 1QneX0-0008Qe-QQ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 31 Jul 2011 23:30:46 +0100 Received: (qmail 15765 invoked by uid 634); 31 Jul 2011 22:30:45 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.4 required=5.0 tests=ALL_TRUSTED,FB_WORD1_END_DOLLAR X-Spam-Check-By: dark.recoil.org Received: from cpc2-cmbg9-0-0-cust410.5-4.cable.virginmedia.com (HELO [192.168.1.111]) (82.16.101.155) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.83) with ESMTPA; Sun, 31 Jul 2011 23:30:45 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: node.js backend resurrected Date: Sun, 31 Jul 2011 23:30:45 +0100 Message-Id: <0E4268DF-2769-4EDB-8AAE-FBCD018FE843@recoil.org> To: cl-mirage@lists.cam.ac.uk Mime-Version: 1.0 (Apple Message framework v1244.3) X-Mailer: Apple Mail (2.1244.3) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 22:30:46 -0000 X-Keywords: X-UID: 15 Status: O Content-Length: 879 Lines: 33 I've refreshed Thomas' node.js backend for Mirage to the latest = ocamlbuild framework, and it works for very basic stuff such as the = thread sleeping test in tests/basic/sleep, when compiled with mir-node = and run with "node _build/sleep.js" (see below). Next step: will take a look at re-binding the OS.Netif (via the pcap = interface in Node) and seeing how well the TCP stack performs when = compiled under Javascript. If anyone's interested in playing with the = JS backend, let me know. -anil click:sleep avsm$ node _build/sleep.js=20 31 Jul 23:27:56 - start: four 31 Jul 23:27:56 -=20 31 Jul 23:27:56 - iteration: four 1 31 Jul 23:27:56 -=20 31 Jul 23:27:56 - start: three 31 Jul 23:27:56 -=20 31 Jul 23:27:56 - iteration: three 1 31 Jul 23:27:56 -=20 31 Jul 23:27:56 - start: two 31 Jul 23:27:56 -=20 31 Jul 23:27:56 - iteration: two 1 31 Jul 23:27:56 -=20