From mshinwell@janestreet.com Fri Mar 01 07:02:05 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UBJyn-000588-6q (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 01 Mar 2013 07:02:05 +0000 X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1451140 * -0.7 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mx5.janestreet.com ([38.105.200.229]:48292 helo=tot-dmz-mxout1.janestreet.com) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UBJyf-0000br-7j (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 01 Mar 2013 07:02:05 +0000 Received: from [172.27.22.15] (helo=tot-smtp) by tot-dmz-mxout1.janestreet.com with esmtp (Exim 4.76) (envelope-from ) id 1UBJyL-0000wm-Ix for cl-mirage@lists.cam.ac.uk; Fri, 01 Mar 2013 02:01:37 -0500 Received: from tot-dmz-mxgoog1.delacy.com ([172.27.224.14] helo=mxgoog2.janestreet.com) by tot-smtp with esmtps (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1UBJyK-0002LV-1Z for cl-mirage@lists.cam.ac.uk; Fri, 01 Mar 2013 02:01:36 -0500 Received: from mail-qe0-f70.google.com ([209.85.128.70]) by mxgoog2.janestreet.com with esmtp (Exim 4.76) (envelope-from ) id 1UBJyJ-0007sT-WE for cl-mirage@lists.cam.ac.uk; Fri, 01 Mar 2013 02:01:36 -0500 Received: by mail-qe0-f70.google.com with SMTP id a11so4847331qen.1 for ; Thu, 28 Feb 2013 23:01:35 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:x-received:in-reply-to:references:date :message-id:subject:from:to:cc:content-type:x-gm-message-state; bh=K+ktEMrDZMjeS77XmHPdOQ+CYIvG+ny43Ccllq7pcy4=; b=SlSCu7PJCcVTPcVgCaP8W2Q4vSVnGRqmg2msM3rgxW/aQc4x7znR6v2a66XkLZTWem WancYq+tbmmSxCWA0nVORt//aLWA1o8/76sv+mRxFilRzYlQAHZOHrtIducdHxqf6GuK ufcX35lcOrfY2Inr6z6NzxTFCw9rkbhca8U0eray8UZuIajAzgS3KVDWnaRlvtkl2qXt 2NtsNcKKrCmePdVPVXJoIFa5eH21kFfeDwD481KqpYt4E5HNEIHBBMrCHgevvBuBl0GC yKuuahUThTlNfNT7715UgVNlgFX8BsTiEgeXTUWbGUaSzGuj2l+Jj/9Z3j8vM60utaR4 r9jQ== X-Received: by 10.58.84.164 with SMTP id a4mr3783521vez.9.1362121295732; Thu, 28 Feb 2013 23:01:35 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.58.84.164 with SMTP id a4mr3783513vez.9.1362121295452; Thu, 28 Feb 2013 23:01:35 -0800 (PST) Received: by 10.58.244.164 with HTTP; Thu, 28 Feb 2013 23:01:35 -0800 (PST) In-Reply-To: References: <9022E254-4804-4630-9CF8-EFA7C213F58C@recoil.org> Date: Fri, 1 Mar 2013 07:01:35 +0000 Message-ID: Subject: Re: the profiling patch From: Mark Shinwell To: Anil Madhavapeddy Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQlpYqwQm2n0wlwpQRg3ezSyrP5hPrVMwTuuzv8KClujsHwzRHYqZSP/dj6fgjPXCf2N+R57NJ7j+Lyem99dD/Kr91Br/hcs/lQ7+rFesNJ0HEYSddHAfaWMu/BFO1kTzDzrPfxtKx9k0PD/qyeOmsyYv/M5kA== Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 07:02:05 -0000 Content-Length: 615 Lines: 28 On 28 February 2013 18:05, Anil Madhavapeddy wrote: > However, when I run it on a toy program, I get the expect > > $ ./a.out > minor,0x404368,303 > minor,0x405bd0,5 > minor,0x405e48,2 > minor,0x4063c0,9 > minor,0x40ba48,12 > major,0x4151a8,72 > $ ./a.out > x > $ ./parse.sh ./a.out x > /tmp/tmp.fJkFjf9obV:1: Error in sourced command file: > No symbol "minor" in current context. > 303 > 72 > 12 > 9 > 5 > 2 > > Where does this 'minor' symbol crop up from? You need to split the output into two files and delete the first column from each ("minor," or "major,"). Did you miss that step? Mark From raphlalou@gmail.com Fri Mar 01 10:02:54 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UBMnm-0005we-Cg (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 01 Mar 2013 10:02:54 +0000 X-Cam-SpamDetails: score 0.2 from SpamAssassin-3.3.2-1451140 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.223.179 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (raphlalou[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 0.9 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ie0-f179.google.com ([209.85.223.179]:45335) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UBMnl-00047t-7P (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 01 Mar 2013 10:02:54 +0000 Received: by mail-ie0-f179.google.com with SMTP id k11so3289879iea.24 for ; Fri, 01 Mar 2013 02:02:52 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.50.190.130 with SMTP id gq2mr6358411igc.77.1362132172509; Fri, 01 Mar 2013 02:02:52 -0800 (PST) Received: by 10.42.18.9 with HTTP; Fri, 1 Mar 2013 02:02:52 -0800 (PST) In-Reply-To: References: <9022E254-4804-4630-9CF8-EFA7C213F58C@recoil.org> Date: Fri, 1 Mar 2013 10:02:52 +0000 Message-ID: Subject: Re: the profiling patch From: Raphael Proust To: Mark Shinwell Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "cl-mirage@lists.cam.ac.uk List" , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 10:02:54 -0000 Content-Length: 918 Lines: 38 On Fri, Mar 1, 2013 at 7:01 AM, Mark Shinwell wr= ote: > On 28 February 2013 18:05, Anil Madhavapeddy wrote: >> However, when I run it on a toy program, I get the expect >> >> $ ./a.out >> minor,0x404368,303 >> minor,0x405bd0,5 >> minor,0x405e48,2 >> minor,0x4063c0,9 >> minor,0x40ba48,12 >> major,0x4151a8,72 >> $ ./a.out > x >> $ ./parse.sh ./a.out x >> /tmp/tmp.fJkFjf9obV:1: Error in sourced command file: >> No symbol "minor" in current context. >> 303 >> 72 >> 12 >> 9 >> 5 >> 2 >> >> Where does this 'minor' symbol crop up from? > > You need to split the output into two files and delete the first > column from each ("minor," or "major,"). Did you miss that step? Can you add the following to your processing pipeline =E2=80=A6 | grep "^minor" | cut -b 7- | =E2=80=A6 =E2=80=A6 | grep "^major" | cut -b 7- | =E2=80=A6 --=20 ______________ Rapha=C3=ABl Proust From raphlalou@gmail.com Fri Mar 01 10:05:11 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UBMpz-00063K-Bh (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 01 Mar 2013 10:05:11 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1451140 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.223.182 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (raphlalou[at]gmail.com) * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature * 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. * See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block * for more information. * [URIs: parse.sh] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ie0-f182.google.com ([209.85.223.182]:36060) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UBMpy-00021b-sS (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 01 Mar 2013 10:05:11 +0000 Received: by mail-ie0-f182.google.com with SMTP id k14so3321470iea.27 for ; Fri, 01 Mar 2013 02:05:10 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.50.150.228 with SMTP id ul4mr6273091igb.9.1362132310192; Fri, 01 Mar 2013 02:05:10 -0800 (PST) Received: by 10.42.18.9 with HTTP; Fri, 1 Mar 2013 02:05:10 -0800 (PST) In-Reply-To: References: <9022E254-4804-4630-9CF8-EFA7C213F58C@recoil.org> Date: Fri, 1 Mar 2013 10:05:10 +0000 Message-ID: Subject: Re: the profiling patch From: Raphael Proust To: Mark Shinwell Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "cl-mirage@lists.cam.ac.uk List" , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 10:05:11 -0000 Content-Length: 1288 Lines: 51 On Fri, Mar 1, 2013 at 10:02 AM, Raphael Proust wrote= : > On Fri, Mar 1, 2013 at 7:01 AM, Mark Shinwell = wrote: >> On 28 February 2013 18:05, Anil Madhavapeddy wrote: >>> However, when I run it on a toy program, I get the expect >>> >>> $ ./a.out >>> minor,0x404368,303 >>> minor,0x405bd0,5 >>> minor,0x405e48,2 >>> minor,0x4063c0,9 >>> minor,0x40ba48,12 >>> major,0x4151a8,72 >>> $ ./a.out > x >>> $ ./parse.sh ./a.out x >>> /tmp/tmp.fJkFjf9obV:1: Error in sourced command file: >>> No symbol "minor" in current context. >>> 303 >>> 72 >>> 12 >>> 9 >>> 5 >>> 2 >>> >>> Where does this 'minor' symbol crop up from? >> >> You need to split the output into two files and delete the first >> column from each ("minor," or "major,"). Did you miss that step? > > Can you add the following to your processing pipeline > > =E2=80=A6 | grep "^minor" | cut -b 7- | =E2=80=A6 > =E2=80=A6 | grep "^major" | cut -b 7- | =E2=80=A6 In fact, depending on support by your shell, you might do something like: in zsh: ./parse.sh ./a.out <( cat x | grep "^minor" | cut -b 7- ) in rc: ./parse.sh ./a.out <{ cat x | grep "^minor" | cut -b 7- } > > > -- > ______________ > Rapha=C3=ABl Proust --=20 ______________ Rapha=C3=ABl Proust From mshinwell@janestreet.com Fri Mar 01 12:49:00 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UBPOW-0007W1-SU (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 01 Mar 2013 12:49:00 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1451140 * 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. * See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block * for more information. * [URIs: janestreet.com] * -0.7 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mx5.janestreet.com ([38.105.200.229]:59718 helo=tot-dmz-mxout1.janestreet.com) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UBPOR-0004dG-qv (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 01 Mar 2013 12:49:00 +0000 Received: from [172.27.22.15] (helo=tot-smtp) by tot-dmz-mxout1.janestreet.com with esmtp (Exim 4.76) (envelope-from ) id 1UBPO8-0003OL-IO for cl-mirage@lists.cam.ac.uk; Fri, 01 Mar 2013 07:48:36 -0500 Received: from tot-dmz-mxgoog1.delacy.com ([172.27.224.14] helo=mxgoog2.janestreet.com) by tot-smtp with esmtps (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1UBPO6-0007my-W8 for cl-mirage@lists.cam.ac.uk; Fri, 01 Mar 2013 07:48:34 -0500 Received: from mail-gh0-f198.google.com ([209.85.160.198]) by mxgoog2.janestreet.com with esmtp (Exim 4.76) (envelope-from ) id 1UBPO6-0007CM-U9 for cl-mirage@lists.cam.ac.uk; Fri, 01 Mar 2013 07:48:34 -0500 Received: by mail-gh0-f198.google.com with SMTP id r11so4317024ghr.9 for ; Fri, 01 Mar 2013 04:48:34 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:x-received:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding:x-gm-message-state; bh=Sm4nQO5/A0m9eGSz8/SRpZiWJ8ab67IkEZZg1v8iM5E=; b=J82iaMAiizz+GN1Z8/fh0xKJG3tY9+e6L/eVuTwU/xc6wyseiyMD64ejIPR7Cz1EAG NGxtWfSAX5W44IGOgoquy+B6PkH87xdGJJFWPPp00jZY6FdMAH6uU5sasAmG4EqbrziR iyt44sKhtKruMwVb1WTY8nIVf8hQGRdyHqsGSweOKBb7LahTOX5Ud8H72AL12yI9BCda x+ZOD7/8OizGN+u2eJMio1+ht1Xm4jFIcyznGYHnVKtuYtFbJAug+LtXnCIKvs98T3bM ZcJthg5rQ0v9G6Xi02eDCLfdI+A6IGQ0IDsurYZlu+WcD85JfyquxstYLzfw7MUhD0VJ GRWQ== X-Received: by 10.229.203.78 with SMTP id fh14mr3602826qcb.143.1362142114708; Fri, 01 Mar 2013 04:48:34 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.229.203.78 with SMTP id fh14mr3602823qcb.143.1362142114619; Fri, 01 Mar 2013 04:48:34 -0800 (PST) Received: by 10.49.132.9 with HTTP; Fri, 1 Mar 2013 04:48:34 -0800 (PST) In-Reply-To: References: <9022E254-4804-4630-9CF8-EFA7C213F58C@recoil.org> Date: Fri, 1 Mar 2013 12:48:34 +0000 Message-ID: Subject: Re: the profiling patch From: Mark Shinwell To: Raphael Proust Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQk8N3F14/scP62vfpJ70Cu8r4Dlv08YPhoSP9n3p1XgQr2onGiuH+XPDHgo2utz5D1JhyvTO6nnsjFpPCqZMCVtOMvzT+k7o/7zhq+/HdywGTUPo6LpHbNH0CoMsdpLH7tk9LchlauReANabAsgs0/MP780vQ== Cc: "cl-mirage@lists.cam.ac.uk List" , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 12:49:00 -0000 Content-Length: 824 Lines: 25 On 1 March 2013 10:02, Raphael Proust wrote: >>> Where does this 'minor' symbol crop up from? >> >> You need to split the output into two files and delete the first >> column from each ("minor," or "major,"). Did you miss that step? > > Can you add the following to your processing pipeline > > =85 | grep "^minor" | cut -b 7- | =85 > =85 | grep "^major" | cut -b 7- | =85 Creature comforts! Once this is proven to work, then yeah, I'll tidy it all up. I haven't really thought much about a "proper" interface for extracting all of this information. More information is coming too. The next version of the patch, in testing internally at JS at the moment, also enables one---without using any extra memory---to determine which source file and line number allocated each block on the heap... Mark From amc79@cam.ac.uk Mon Mar 04 16:35:13 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UCYM5-00035Q-4o (Exim 4.70) (return-path ); Mon, 04 Mar 2013 16:35:13 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from dhcp-128-232-141-191.eduroam.csx.cam.ac.uk ([128.232.141.191]:52574) by ppsw-51.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.158]:587) with esmtpsa (PLAIN:amc79) (TLSv1:AES128-SHA:128) id 1UCYM5-00008W-WU (Exim 4.72) (return-path ); Mon, 04 Mar 2013 16:35:13 +0000 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: OCaml Labs Meeting - 8th March at 2pm in the Computer Lab From: Amir Chaudhry In-Reply-To: <9C034B82-D570-4443-80CD-AFC9EAF25417@cam.ac.uk> Date: Mon, 4 Mar 2013 16:35:15 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <4736721C-FA72-4258-9991-A05A03BC150D@cam.ac.uk> References: <9C034B82-D570-4443-80CD-AFC9EAF25417@cam.ac.uk> To: "cl-ocamllabs@lists.cam.ac.uk" X-Mailer: Apple Mail (2.1499) Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Mar 2013 16:35:13 -0000 Content-Length: 1337 Lines: 60 Dear all, The Agenda for the meeting this Friday is below. The meeting will be in = FW26 in the Computer Lab at 2pm. If you haven't confirmed you = can/cannot make it, please do let me know. --- Agenda --- OCL Updates (Anil - 5-10 mins) - New joiners (Jeremy Yallop, Philippe Wang) - Networking as a Service - 3yr EPSRC Grant awarded - (other grants in progress) Platform projects - Website design (Amir - 5mins) - CI system (Amir - 5min) Systems projects - Mirage and xen.org (Amir - 2mins) - Mirari (TBC - 10mins) Compiler projects - camlp4 working group (Leo - 5 mins) Open discussion Close ----- Best wishes, Amir On 25 Feb 2013, at 17:33, Amir Chaudhry wrote: > Dear all, >=20 > The next OCaml Labs meeting will take place on the 8th of March.=20 > Location information is below and the meeting agenda will follow later = this week. =20 >=20 > Please do let me know if you will be attending. >=20 > -- Details -- > OCaml Labs Meeting > 8th March 2013 > 2pm =96 3:30pm (but aiming for 3pm finish) > Room FW26 - Cambridge Computer Laboratory > William Gates Building > JJ Thomson Avenue > Cambridge CB3 0FD >=20 > Best wishes, > Amir >=20 >=20 > PS If you'd like to subscribe to an iCal feed of OCaml Labs events, = please use the following link: webcal://talks.cam.ac.uk/show/ics/41277 From am21@cl.cam.ac.uk Mon Mar 04 18:05:49 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UCZll-0007GV-5q (Exim 4.70) (return-path ); Mon, 04 Mar 2013 18:05:49 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.6 from SpamAssassin-3.3.2-1452019 * 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. * See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block * for more information. * [URIs: xen.org] * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP * -0.6 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mta0.cl.cam.ac.uk ([128.232.25.20]:47968) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UCZlk-00083g-SR (Exim 4.72) (return-path ); Mon, 04 Mar 2013 18:05:49 +0000 Received: from ramsey.cl.cam.ac.uk ([128.232.1.219] helo=cl.cam.ac.uk) by mta0.cl.cam.ac.uk with esmtp (Exim 4.63) (envelope-from ) id 1UCZlk-0007zM-Hw; Mon, 04 Mar 2013 18:05:48 +0000 To: Amir Chaudhry Subject: Re: OCaml Labs Meeting - 8th March at 2pm in the Computer Lab In-Reply-To: Message from Amir Chaudhry of "Mon, 04 Mar 2013 16:35:15 GMT." <4736721C-FA72-4258-9991-A05A03BC150D@cam.ac.uk> Date: Mon, 04 Mar 2013 18:05:48 +0000 From: Alan Mycroft Message-Id: Cc: Mirage List , "cl-ocamllabs@lists.cam.ac.uk" , Alan.Mycroft@cl.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Mar 2013 18:05:49 -0000 Content-Length: 780 Lines: 33 > The Agenda for the meeting this Friday is below. The meeting will be in = > FW26 in the Computer Lab at 2pm. If you haven't confirmed you = > can/cannot make it, please do let me know. It might be nice to try to get the SML source-level interface to OCaml working in the near future. [LCP has some big SML programs.] Sorry I cannot be there. Alan --- Agenda --- OCL Updates (Anil - 5-10 mins) - New joiners (Jeremy Yallop, Philippe Wang) - Networking as a Service - 3yr EPSRC Grant awarded - (other grants in progress) Platform projects - Website design (Amir - 5mins) - CI system (Amir - 5min) Systems projects - Mirage and xen.org (Amir - 2mins) - Mirari (TBC - 10mins) Compiler projects - camlp4 working group (Leo - 5 mins) Open discussion Close From sdiris@gmail.com Tue Mar 05 10:51:25 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UCpSv-0002v0-Gn (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 05 Mar 2013 10:51:25 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1452211 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.175 listed in list.dnswl.dnsbl.ja.net] * 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. * See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block * for more information. * [URIs: cam.ac.uk] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (sdiris[at]gmail.com) * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-we0-f175.google.com ([74.125.82.175]:61691) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UCpSu-0004bZ-px (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 05 Mar 2013 10:51:25 +0000 Received: by mail-we0-f175.google.com with SMTP id x8so5999733wey.34 for ; Tue, 05 Mar 2013 02:51:23 -0800 (PST) X-Received: by 10.194.87.229 with SMTP id bb5mr37888218wjb.32.1362480683819; Tue, 05 Mar 2013 02:51:23 -0800 (PST) Received: from yimingwin7 (c155.al.cl.cam.ac.uk. [128.232.110.155]) by mx.google.com with ESMTPS id fv2sm21560414wib.6.2013.03.05.02.51.21 (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 05 Mar 2013 02:51:22 -0800 (PST) From: "Yiming Zhang" To: Subject: re: Cl-mirage Digest, Vol 14, Issue 3 Date: Tue, 5 Mar 2013 10:51:23 -0000 Message-ID: <002801ce198f$61bb0fb0$25312f10$@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: Ac4ZjpPCciu4EkZfRRuFz85k0J96IQ== Content-Language: zh-cn X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 10:51:25 -0000 Content-Length: 4145 Lines: 163 Great! I want to attend the meeting. Cheers, Yiming -----=D3=CA=BC=FE=D4=AD=BC=FE----- =B7=A2=BC=FE=C8=CB: cl-mirage-bounces@lists.cam.ac.uk [mailto:cl-mirage-bounces@lists.cam.ac.uk] =B4=FA=B1=ED = cl-mirage-request@lists.cam. ac.uk =B7=A2=CB=CD=CA=B1=BC=E4: 2013=C4=EA3=D4=C25=C8=D5 4:01 =CA=D5=BC=FE=C8=CB: cl-mirage@lists.cam.ac.uk =D6=F7=CC=E2: Cl-mirage Digest, Vol 14, Issue 3 Send Cl-mirage mailing list submissions to cl-mirage@lists.cam.ac.uk To subscribe or unsubscribe via the World Wide Web, visit https://lists.cam.ac.uk/mailman/listinfo/cl-mirage or, via email, send a message with subject or body 'help' to cl-mirage-request@lists.cam.ac.uk You can reach the person managing the list at cl-mirage-owner@lists.cam.ac.uk When replying, please edit your Subject line so it is more specific than "Re: Contents of Cl-mirage digest..." Today's Topics: 1. Re: OCaml Labs Meeting - 8th March at 2pm in the Computer Lab (Amir Chaudhry) 2. Re: OCaml Labs Meeting - 8th March at 2pm in the Computer Lab (Alan Mycroft) ---------------------------------------------------------------------- Message: 1 Date: Mon, 4 Mar 2013 16:35:15 +0000 From: Amir Chaudhry Subject: Re: OCaml Labs Meeting - 8th March at 2pm in the Computer Lab To: "cl-ocamllabs@lists.cam.ac.uk" Cc: Mirage List Message-ID: <4736721C-FA72-4258-9991-A05A03BC150D@cam.ac.uk> Content-Type: text/plain; charset=3Dwindows-1252 Dear all, The Agenda for the meeting this Friday is below. The meeting will be in FW26 in the Computer Lab at 2pm. If you haven't confirmed you = can/cannot make it, please do let me know. --- Agenda --- OCL Updates (Anil - 5-10 mins) - New joiners (Jeremy Yallop, Philippe Wang) - Networking as a Service - 3yr EPSRC Grant awarded - (other grants in progress) Platform projects - Website design (Amir - 5mins) - CI system (Amir - 5min) Systems projects - Mirage and xen.org (Amir - 2mins) - Mirari (TBC - 10mins) Compiler projects - camlp4 working group (Leo - 5 mins) Open discussion Close ----- Best wishes, Amir On 25 Feb 2013, at 17:33, Amir Chaudhry wrote: > Dear all, >=20 > The next OCaml Labs meeting will take place on the 8th of March.=20 > Location information is below and the meeting agenda will follow later this week. =20 >=20 > Please do let me know if you will be attending. >=20 > -- Details -- > OCaml Labs Meeting > 8th March 2013 > 2pm ? 3:30pm (but aiming for 3pm finish) Room FW26 - Cambridge=20 > Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge = > CB3 0FD >=20 > Best wishes, > Amir >=20 >=20 > PS If you'd like to subscribe to an iCal feed of OCaml Labs events,=20 > please use the following link: webcal://talks.cam.ac.uk/show/ics/41277 ------------------------------ Message: 2 Date: Mon, 04 Mar 2013 18:05:48 +0000 From: Alan Mycroft Subject: Re: OCaml Labs Meeting - 8th March at 2pm in the Computer Lab To: Amir Chaudhry Cc: Mirage List , "cl-ocamllabs@lists.cam.ac.uk" , Alan.Mycroft@cl.cam.ac.uk Message-ID: > The Agenda for the meeting this Friday is below. The meeting will be=20 > in =3D > FW26 in the Computer Lab at 2pm. If you haven't confirmed you =3D=20 > can/cannot make it, please do let me know. It might be nice to try to get the SML source-level interface to OCaml working in the near future. [LCP has some big SML programs.] Sorry I cannot be there. Alan --- Agenda --- OCL Updates (Anil - 5-10 mins) - New joiners (Jeremy Yallop, Philippe Wang) - Networking as a Service - 3yr EPSRC Grant awarded - (other grants in progress) Platform projects - Website design (Amir - 5mins) - CI system (Amir - 5min) Systems projects - Mirage and xen.org (Amir - 2mins) - Mirari (TBC - 10mins) Compiler projects - camlp4 working group (Leo - 5 mins) Open discussion Close End of Cl-mirage Digest, Vol 14, Issue 3 **************************************** From vb@luminar.eu.org Tue Mar 05 17:56:04 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UCw5s-0002AY-Ja (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 05 Mar 2013 17:56:04 +0000 X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1452211 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [66.165.176.63 listed in list.dnswl.dnsbl.ja.net] * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from smtp02.citrix.com ([66.165.176.63]:7953) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UCw5r-0004Eb-8L (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 05 Mar 2013 17:56:04 +0000 X-IronPort-AV: E=Sophos;i="4.84,788,1355097600"; d="scan'208";a="10726636" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP/TLS/AES128-SHA; 05 Mar 2013 17:56:01 +0000 Received: from ukmail1.uk.xensource.com (10.80.16.128) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.2.318.1; Tue, 5 Mar 2013 12:56:00 -0500 Received: from serenity.uk.xensource.com ([10.80.2.33]) by ukmail1.uk.xensource.com with esmtp (Exim 4.69) (envelope-from ) id 1UCw5o-0007ot-UE for cl-mirage@lists.cam.ac.uk; Tue, 05 Mar 2013 17:56:00 +0000 Message-ID: <513631B0.3070202@luminar.eu.org> Date: Tue, 5 Mar 2013 17:56:00 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130221 Thunderbird/17.0.3 MIME-Version: 1.0 To: Subject: Re: mirari updates References: <20130227215220.GA2255@dark.recoil.org> In-Reply-To: <20130227215220.GA2255@dark.recoil.org> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 17:56:04 -0000 Content-Length: 1207 Lines: 31 On 27/02/2013 21:52, Anil Madhavapeddy wrote: > - In the UNIX backend, in mirage-platform/unix/runtime/tap_stubs_*.c, > we currently hardcode an `ifconfig 10.0.0.1` to give the tap device > a static IP address. This has made it easy so far to get networking > up and running, but the tap device setup really ought to be done by > Mirari-run instead of the Mirage-platform libraries (which should just > open a tun passed to them). I don’t think mirari (and obviously even less Netif) should setup the tap since it requires root access. Mirari should give instructions on how to do so, but not issue root commands itself. I understand the benefit of simplifing the process of running mirage applications but in my opinion, it is not acceptable that mirari run as root. Please tell me what you think about it. About the Xen backend, we should decide on a standard way to execute kernels (as Anil said). At the moment, I only tested the xl create -c config.cfg kernel.xen method, which works well. If I understand well, it is better to use libvirt as it is a library that could be better integrated with mirari. I did not test it yet, but plan to do it asap. Cheers ! Vincent From anil@recoil.org Tue Mar 05 20:54:54 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UCysw-0005sd-Ol (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 05 Mar 2013 20:54:54 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1452211 * 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. * See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block * for more information. * [URIs: luminar.eu.org] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:48080 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UCysw-00032L-pw (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 05 Mar 2013 20:54:54 +0000 Received: (qmail 7294 invoked by uid 634); 5 Mar 2013 20:54:53 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 05 Mar 2013 20:54:53 +0000 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: mirari updates From: Anil Madhavapeddy In-Reply-To: <513631B0.3070202@luminar.eu.org> Date: Tue, 5 Mar 2013 20:54:53 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <5D5E9F35-FC42-4E73-9045-7C86E18ACB44@recoil.org> References: <20130227215220.GA2255@dark.recoil.org> <513631B0.3070202@luminar.eu.org> To: Vincent Bernardoff X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Jon Ludlam , cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 20:54:54 -0000 Content-Length: 2170 Lines: 51 On 5 Mar 2013, at 17:56, Vincent Bernardoff wrote: > On 27/02/2013 21:52, Anil Madhavapeddy wrote: >> - In the UNIX backend, in mirage-platform/unix/runtime/tap_stubs_*.c, >> we currently hardcode an `ifconfig 10.0.0.1` to give the tap device >> a static IP address. This has made it easy so far to get = networking >> up and running, but the tap device setup really ought to be done by >> Mirari-run instead of the Mirage-platform libraries (which should = just >> open a tun passed to them). >=20 > I don=92t think mirari (and obviously even less Netif) should setup = the tap since it requires root access. Mirari should give instructions = on how to do so, but not issue root commands itself. It doesn't necessarily need root: there are a bunch of platform-specific = ways. For example, on Linux, you can set CAP_NET_ADMIN to the Mirari = binary to gain access to tunctl. There's a good summary of all the ways = in the Erlang tuntap bindings here: https://github.com/msantos/tunctl (it might be worth creating a robust ocaml-tuntap that wraps these = mechanisms) >=20 > About the Xen backend, we should decide on a standard way to execute = kernels (as Anil said). At the moment, I only tested the xl create -c = config.cfg kernel.xen method, which works well. If I understand well, it = is better to use libvirt as it is a library that could be better = integrated with mirari. I did not test it yet, but plan to do it asap. Great. I know Dave has been experimenting with libvirt recently, but = he's travelling this week. Jon, Mike, do you have any opinions about = this? A (very) simple first start would be to just generate the minimal .conf = file (as mir-run does at the moment) and directly call 'xl/xm create = -c'. The more interesting second backend is automating Amazon EC2, for which = bindings exist on Github (but I haven't tried). This requires signing = the Xen kernel as an AKI image, and calling the right HTTP calls to = upload it and register it, before spinning it up. I can give you = delegate access to my Amazon account, Vincent, if you don't have access = to an EC2 account within Citrix. -anil From thomas.gazagnaire@gmail.com Tue Mar 05 21:00:19 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UCyyB-00060W-7w (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 05 Mar 2013 21:00:19 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1452211 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.46 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-wg0-f46.google.com ([74.125.82.46]:52411) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UCyyA-0006on-Yo (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 05 Mar 2013 21:00:19 +0000 Received: by mail-wg0-f46.google.com with SMTP id fg15so6311517wgb.13 for ; Tue, 05 Mar 2013 13:00:18 -0800 (PST) X-Received: by 10.194.77.129 with SMTP id s1mr42488998wjw.17.1362517218423; Tue, 05 Mar 2013 13:00:18 -0800 (PST) Received: from [192.168.0.14] (gou06-3-88-170-165-56.fbx.proxad.net. [88.170.165.56]) by mx.google.com with ESMTPS id ek4sm22886814wib.11.2013.03.05.13.00.16 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 05 Mar 2013 13:00:17 -0800 (PST) Subject: Re: mirari updates Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: <5D5E9F35-FC42-4E73-9045-7C86E18ACB44@recoil.org> Date: Tue, 5 Mar 2013 22:00:14 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20130227215220.GA2255@dark.recoil.org> <513631B0.3070202@luminar.eu.org> <5D5E9F35-FC42-4E73-9045-7C86E18ACB44@recoil.org> To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1085) Cc: Jon Ludlam , cl-mirage@lists.cam.ac.uk, Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 21:00:19 -0000 Content-Length: 606 Lines: 12 > The more interesting second backend is automating Amazon EC2, for = which bindings exist on Github (but I haven't tried). This requires = signing the Xen kernel as an AKI image, and calling the right HTTP calls = to upload it and register it, before spinning it up. I can give you = delegate access to my Amazon account, Vincent, if you don't have access = to an EC2 account within Citrix. Actually this would be *very* helpful. I haven't add the chance yet to = send an OCaml microkernel to EC2, that would be quite cool to be able to = do it with a single command-line option (and some magic) :-) From scott.dj@gmail.com Tue Mar 05 21:02:56 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UCz0i-00062Q-1i (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 05 Mar 2013 21:02:56 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1452211 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.52 listed in list.dnswl.dnsbl.ja.net] * 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. * See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block * for more information. * [URIs: luminar.eu.org] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-vb0-f52.google.com ([209.85.212.52]:34190) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UCz0h-0007XX-Wl (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 05 Mar 2013 21:02:56 +0000 Received: by mail-vb0-f52.google.com with SMTP id fa15so1442529vbb.11 for ; Tue, 05 Mar 2013 13:02:54 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.58.90.66 with SMTP id bu2mr4607766veb.29.1362517374507; Tue, 05 Mar 2013 13:02:54 -0800 (PST) Received: by 10.220.56.200 with HTTP; Tue, 5 Mar 2013 13:02:54 -0800 (PST) In-Reply-To: <5D5E9F35-FC42-4E73-9045-7C86E18ACB44@recoil.org> References: <20130227215220.GA2255@dark.recoil.org> <513631B0.3070202@luminar.eu.org> <5D5E9F35-FC42-4E73-9045-7C86E18ACB44@recoil.org> Date: Tue, 5 Mar 2013 21:02:54 +0000 Message-ID: Subject: Re: mirari updates From: David Scott To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=089e013cc3861bad4504d733cc74 Cc: Jon Ludlam , Mirage List , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 21:02:56 -0000 Content-Length: 6958 Lines: 153 --089e013cc3861bad4504d733cc74 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On Tue, Mar 5, 2013 at 8:54 PM, Anil Madhavapeddy wrote: > On 5 Mar 2013, at 17:56, Vincent Bernardoff wrote: > > > On 27/02/2013 21:52, Anil Madhavapeddy wrote: > >> - In the UNIX backend, in mirage-platform/unix/runtime/tap_stubs_*.c, > >> we currently hardcode an `ifconfig 10.0.0.1` to give the tap device > >> a static IP address. This has made it easy so far to get networking > >> up and running, but the tap device setup really ought to be done by > >> Mirari-run instead of the Mirage-platform libraries (which should ju= st > >> open a tun passed to them). > > > > I don=92t think mirari (and obviously even less Netif) should setup the > tap since it requires root access. Mirari should give instructions on how > to do so, but not issue root commands itself. > > It doesn't necessarily need root: there are a bunch of platform-specific > ways. For example, on Linux, you can set CAP_NET_ADMIN to the Mirari bina= ry > to gain access to tunctl. There's a good summary of all the ways in the > Erlang tuntap bindings here: > > https://github.com/msantos/tunctl > > (it might be worth creating a robust ocaml-tuntap that wraps these > mechanisms) > An ocaml-tuntap package sounds like a very nice thing. > > > About the Xen backend, we should decide on a standard way to execute > kernels (as Anil said). At the moment, I only tested the xl create -c > config.cfg kernel.xen method, which works well. If I understand well, it = is > better to use libvirt as it is a library that could be better integrated > with mirari. I did not test it yet, but plan to do it asap. > > Great. I know Dave has been experimenting with libvirt recently, but he's > travelling this week. Jon, Mike, do you have any opinions about this? > So far libvirt (via Richard Jones' ocaml bindings) is working well for me, although I've been focusing on the storage interface rather than the VM ("domain") interface. I haven't tried any of the xen drivers -- there are 3: xend, libxl and (old) xen-api. I would recommend figuring out how to start a simple VM using KVM+libvirt first to become familiar with the libvirt API and then switching over to xen. A (very) simple first start would be to just generate the minimal .conf > file (as mir-run does at the moment) and directly call 'xl/xm create -c'. > > The more interesting second backend is automating Amazon EC2, for which > bindings exist on Github (but I haven't tried). This requires signing the > Xen kernel as an AKI image, and calling the right HTTP calls to upload it > and register it, before spinning it up. I can give you delegate access t= o > my Amazon account, Vincent, if you don't have access to an EC2 account > within Citrix. > Support for EC2 would be fantastic! Cheers, Dave --089e013cc3861bad4504d733cc74 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable



On Tue, Mar 5, 2013 at 8:54 PM, Anil Madhavapeddy <anil@recoil.org> wrote:
On 5 Mar 2013, at 17:56, V= incent Bernardoff <vb@luminar.eu.or= g> wrote:

> On 27/02/2013 21:52, Anil Madhavapeddy wrote:
>> - In the UNIX backend, in mirage-platform/unix/runtime/tap_stubs_*= .c,
>> =A0 we currently hardcode an `ifconfig 10.0.0.1` to give the tap d= evice
>> =A0 a static IP address. =A0This has made it easy so far to get ne= tworking
>> =A0 up and running, but the tap device setup really ought to be do= ne by
>> =A0 Mirari-run instead of the Mirage-platform libraries (which sho= uld just
>> =A0 open a tun passed to them).
>
> I don=92t think mirari (and obviously even less Netif) should setup th= e tap since it requires root access. Mirari should give instructions on how= to do so, but not issue root commands itself.

It doesn't necessarily need root: there are a bunch of platform-s= pecific ways. For example, on Linux, you can set CAP_NET_ADMIN to the Mirar= i binary to gain access to tunctl. =A0There's a good summary of all the= ways in the Erlang tuntap bindings here:

https://git= hub.com/msantos/tunctl

(it might be worth creating a robust ocaml-tuntap that wraps these mechanis= ms)

An ocaml-tuntap package sounds like= a very nice thing.
=A0

>
> About the Xen backend, we should decide on a standard way to execute k= ernels (as Anil said). At the moment, I only tested the xl create -c config= .cfg kernel.xen method, which works well. If I understand well, it is bette= r to use libvirt as it is a library that could be better integrated with mi= rari. I did not test it yet, but plan to do it asap.

Great. I know Dave has been experimenting with libvirt recently, but = he's travelling this week. =A0Jon, Mike, do you have any opinions about= this?

So far libvirt (via Richard Jone= s' ocaml bindings) is working well for me, although I've been focus= ing on the storage interface rather than the VM ("domain") interf= ace. I haven't tried any of the xen drivers -- there are 3: xend, libxl= and (old) xen-api. I would recommend figuring out how to start a simple VM= using KVM+libvirt first to become familiar with the libvirt API and then s= witching over to xen.

A (very) simple first start would be to just generate the minimal .conf fil= e (as mir-run does at the moment) and directly call 'xl/xm create -c= 9;.

The more interesting second backend is automating Amazon EC2, for which bin= dings exist on Github (but I haven't tried). This requires signing the = Xen kernel as an AKI image, and calling the right HTTP calls to upload it a= nd register it, before spinning it up. =A0I can give you delegate access to= my Amazon account, Vincent, if you don't have access to an EC2 account= within Citrix.

Support for EC2 would be fantastic!
<= div>
Cheers,
Dave

<= /div> --089e013cc3861bad4504d733cc74-- From vb@vb.fdn.fr Tue Mar 05 22:15:09 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UD08b-0007dJ-Bz (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 05 Mar 2013 22:15:09 +0000 X-Cam-SpamDetails: score -5.1 from SpamAssassin-3.3.2-1452211 * -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high * trust * [66.165.176.89 listed in list.dnswl.dnsbl.ja.net] * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from smtp.citrix.com ([66.165.176.89]:2181) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UD08a-0006XG-7L (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 05 Mar 2013 22:15:09 +0000 X-IronPort-AV: E=Sophos;i="4.84,791,1355097600"; d="scan'208";a="11346645" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP/TLS/AES128-SHA; 05 Mar 2013 22:14:56 +0000 Received: from ukmail1.uk.xensource.com (10.80.16.128) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.2.318.1; Tue, 5 Mar 2013 17:14:56 -0500 Received: from gateway-1.uk.xensource.com ([10.80.16.66]) by ukmail1.uk.xensource.com with esmtp (Exim 4.69) (envelope-from ) id 1UD08O-0003Ir-3a; Tue, 05 Mar 2013 22:14:56 +0000 Received: from localhost ([127.0.0.1] helo=[IPv6:::1]) by gateway-1.uk.xensource.com with esmtp (Exim 4.72) (envelope-from ) id 1UD08N-0002s2-OX; Tue, 05 Mar 2013 22:14:55 +0000 Message-ID: <51366E5F.3030406@vb.fdn.fr> Date: Tue, 5 Mar 2013 22:14:55 +0000 From: Vincent B. User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130109 Thunderbird/17.0.2 MIME-Version: 1.0 To: Subject: Re: mirari updates References: <20130227215220.GA2255@dark.recoil.org> <513631B0.3070202@luminar.eu.org> <5D5E9F35-FC42-4E73-9045-7C86E18ACB44@recoil.org> In-Reply-To: <5D5E9F35-FC42-4E73-9045-7C86E18ACB44@recoil.org> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 8bit Cc: Jon Ludlam X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 22:15:09 -0000 Content-Length: 4679 Lines: 83 On 05/03/2013 20:54, Anil Madhavapeddy wrote: > On 5 Mar 2013, at 17:56, Vincent Bernardoff wrote: > >> On 27/02/2013 21:52, Anil Madhavapeddy wrote: >>> - In the UNIX backend, in mirage-platform/unix/runtime/tap_stubs_*.c, >>> we currently hardcode an `ifconfig 10.0.0.1` to give the tap device >>> a static IP address. This has made it easy so far to get networking >>> up and running, but the tap device setup really ought to be done by >>> Mirari-run instead of the Mirage-platform libraries (which should just >>> open a tun passed to them). >> >> I don’t think mirari (and obviously even less Netif) should setup the tap since it requires root access. Mirari should give instructions on how to do so, but not issue root commands itself. > > It doesn't necessarily need root: there are a bunch of platform-specific ways. For example, on Linux, you can set CAP_NET_ADMIN to the Mirari binary to gain access to tunctl. There's a good summary of all the ways in the Erlang tuntap bindings here: > > https://github.com/msantos/tunctl > > (it might be worth creating a robust ocaml-tuntap that wraps these mechanisms) Ok. I’ve read about the CAP_NET_ADMIN but thought at first that it was not a very practical possiblity. It would work well with Mirari on Linux, Mirari would have to have that cap and it could create tun/tap interfaces. For other OS, it seems that sudo is needed. But this looks really feasible indeed. And an ocaml-tuntap library is probably the best way to implement those functions, as they will probably be needed by other ocaml programs in the future. As everybody seems to agree on that I’ll do my best to write such small library, if we were going to port mirage-platform code into Mirari, it should not be much more work to put it into such library instead. The other possibility is to _not_ bother to setup tun/tap with mirari, but let the use issue the right commands instead (ip tuntap ... or whatever command on other unixes). This is probably a bit less convenient for the user, but it removes some complexity from mirage/mirari, and it makes the user more aware of what is under the hood. My opinion on that is that automatic setup is better ONLY if it works perfectly for everybody. If implementing it properly for all platforms is too tricky such that it does not work equally well for everyone and frustrates users, then it is seems better to me to explain users that they have to setup a tuntap interface, and how to do it. Just tell me what you think about that. >> About the Xen backend, we should decide on a standard way to execute kernels (as Anil said). At the moment, I only tested the xl create -c config.cfg kernel.xen method, which works well. If I understand well, it is better to use libvirt as it is a library that could be better integrated with mirari. I did not test it yet, but plan to do it asap. > > Great. I know Dave has been experimenting with libvirt recently, but he's travelling this week. Jon, Mike, do you have any opinions about this? > > A (very) simple first start would be to just generate the minimal .conf file (as mir-run does at the moment) and directly call 'xl/xm create -c'. Finally I think this is a better first solution (and maybe a better last solution as well). Because it does not require the dependency to libvirt, and that will always work. I’ll definitely implement that first, and I’ll have a look at libvirt later if needed. > The more interesting second backend is automating Amazon EC2, for which bindings exist on Github (but I haven't tried). This requires signing the Xen kernel as an AKI image, and calling the right HTTP calls to upload it and register it, before spinning it up. I can give you delegate access to my Amazon account, Vincent, if you don't have access to an EC2 account within Citrix. Agree that it is very cool. So, basically, here is what I’m going to do for mirari run, in order (if everyones agree): 1. mirari run for UNIX backend: Basically figure out / fix the tuntap business, and just run the executable. 2. mirari run for Xen: Create a xl config file, then run the kernel (using xl exe/ (or libxl if available ?) 3. mirari run for EC2: Implement the automation of sending kernels on EC2. As for making mirari do the opam switch itself (and thus specify the desired backend in mirari config file), I have no opinion about it. To me personally I don’t perceive that as very important (I don’t mind opam switching manually, because I don’t think it is required to do it often). But I can certainly implement it if needed (as you explained in former emails). Good evening, Vincent From thomas.gazagnaire@gmail.com Wed Mar 06 11:10:44 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UDCFA-0002bk-Dh (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 06 Mar 2013 11:10:44 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1452698 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.41 listed in list.dnswl.dnsbl.ja.net] * 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. * See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block * for more information. * [URIs: github.com] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-wg0-f41.google.com ([74.125.82.41]:62114) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UDCF9-0002XJ-qT (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 06 Mar 2013 11:10:44 +0000 Received: by mail-wg0-f41.google.com with SMTP id ds1so4614545wgb.0 for ; Wed, 06 Mar 2013 03:10:43 -0800 (PST) X-Received: by 10.194.178.9 with SMTP id cu9mr25421379wjc.39.1362568243102; Wed, 06 Mar 2013 03:10:43 -0800 (PST) Received: from saorge.home (ANice-551-1-330-9.w86-205.abo.wanadoo.fr. [86.205.202.9]) by mx.google.com with ESMTPS id q2sm22659557wiz.8.2013.03.06.03.10.41 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 06 Mar 2013 03:10:41 -0800 (PST) Subject: Re: mirari updates Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: <239ED0CD-CA7E-479C-8559-A66821A75AC1@recoil.org> Date: Wed, 6 Mar 2013 12:10:39 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <49735BEC-D85A-4CA3-83CB-0CAC5904FFDD@gmail.com> References: <20130227215220.GA2255@dark.recoil.org> <239ED0CD-CA7E-479C-8559-A66821A75AC1@recoil.org> To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1085) Cc: "Vincent B." , Mike McClurg , "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Mar 2013 11:10:44 -0000 Content-Length: 1044 Lines: 30 >> That's a really nice feature! I think what Vincent was looking for = (correct me if I'm wrong, Vincent) is the way to run arbitrary commands = without switching back and forth between compilers. I'm thinking of: >>=20 >> $ opam config env --switch 4.01.0+mirage-xen -- obuild build >>=20 >> So that I can do a mirage build in my current dir, but still use the = system compiler, for instance. This would be a useful feature outside of = mirari. >=20 > That's correct: we currently don't have a way to spawn a subshell with = the environment variables for a different switch. Note that the = subshell does *not* need to call OPAM, only obuild. >=20 > Thomas, I (think) this is what our friend in = https://github.com/OCamlPro/opam/issues/482 wanted :-) I've just added to OPAM this feature. You can now do: $ opam config exec "obuild build" --switch=3D 4.01.0+mirage-xen And this will execute 'obuild build' with the correct environment = variables to use the compiler and tools installed in the = 4.01.0+mirage-xen switch. -- Thomas= From anil@recoil.org Wed Mar 06 11:33:28 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UDCbA-0003ju-Ku (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 06 Mar 2013 11:33:28 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1452698 * 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. * See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block * for more information. * [URIs: github.com] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:41049 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UDCb9-0001A4-Sn (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 06 Mar 2013 11:33:28 +0000 Received: (qmail 7414 invoked by uid 634); 6 Mar 2013 11:33:27 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.38]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 06 Mar 2013 11:33:27 +0000 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: mirari updates From: Anil Madhavapeddy In-Reply-To: <51366E5F.3030406@vb.fdn.fr> Date: Wed, 6 Mar 2013 11:32:24 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <9A44D300-40C4-4F93-B70A-0AEACD3F73C0@recoil.org> References: <20130227215220.GA2255@dark.recoil.org> <513631B0.3070202@luminar.eu.org> <5D5E9F35-FC42-4E73-9045-7C86E18ACB44@recoil.org> <51366E5F.3030406@vb.fdn.fr> To: Vincent B. X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Jon Ludlam , cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Mar 2013 11:33:28 -0000 Content-Length: 4163 Lines: 93 On 5 Mar 2013, at 22:14, Vincent B. wrote: >=20 > Ok. I=92ve read about the CAP_NET_ADMIN but thought at first that it = was not a very practical possiblity. It would work well with Mirari on = Linux, Mirari would have to have that cap and it could create tun/tap = interfaces. For other OS, it seems that sudo is needed. But this looks = really feasible indeed. And an ocaml-tuntap library is probably the best = way to implement those functions, as they will probably be needed by = other ocaml programs in the future. As everybody seems to agree on that = I=92ll do my best to write such small library, if we were going to port = mirage-platform code into Mirari, it should not be much more work to put = it into such library instead. Agreed. The goal for Mirage-platform is to strip it down into a set of = smaller, reusable libraries. Dave has already removed xenstore from = there, for example, and the Blk/Net device drivers are also now in their = own repositories. Doing the same with the UNIX backends makes it much easier to re-use the = code in other contexts, such as Haris' NS3 work, and makes testing and = portability easier. > The other possibility is to _not_ bother to setup tun/tap with mirari, = but let the use issue the right commands instead (ip tuntap ... or = whatever command on other unixes). This is probably a bit less = convenient for the user, but it removes some complexity from = mirage/mirari, and it makes the user more aware of what is under the = hood. My opinion on that is that automatic setup is better ONLY if it = works perfectly for everybody. If implementing it properly for all = platforms is too tricky such that it does not work equally well for = everyone and frustrates users, then it is seems better to me to explain = users that they have to setup a tuntap interface, and how to do it. I agree with this too. I only possible issue in on MacOS X, where there = is no tunctl command, and so something has to: - open() /dev/tap0 - configure it with ifconfig - pass the fd to the application to read()/write() - close() it which destroys the interface. This is an unfortunate API, as it mandates fd passing as far as I can = tell. Unfortunately, most of the users of unix-direct in Mirage will be = on a Mac! >>=20 >> A (very) simple first start would be to just generate the minimal = .conf file (as mir-run does at the moment) and directly call 'xl/xm = create -c'. >=20 > Finally I think this is a better first solution (and maybe a better = last solution as well). Because it does not require the dependency to = libvirt, and that will always work. I=92ll definitely implement that = first, and I=92ll have a look at libvirt later if needed. Agreed. >=20 > So, basically, here is what I=92m going to do for mirari run, in order = (if everyones agree): >=20 > 1. mirari run for UNIX backend: Basically figure out / fix the tuntap = business, and just run the executable. >=20 > 2. mirari run for Xen: Create a xl config file, then run the kernel = (using xl exe/ (or libxl if available ?) >=20 > 3. mirari run for EC2: Implement the automation of sending kernels on = EC2. Agreed! There's a script to wrap the kernel here: https://github.com/avsm/mirage/blob/master/scripts/ec2.sh ...but give me a shout if you need a hand with it. >=20 > As for making mirari do the opam switch itself (and thus specify the = desired backend in mirari config file), I have no opinion about it. To = me personally I don=92t perceive that as very important (I don=92t mind = opam switching manually, because I don=92t think it is required to do it = often). But I can certainly implement it if needed (as you explained in = former emails). I think this would be very useful to put in, since it would let us = remove the Unix module entirely from the Xen OPAM switch. It's = currently in there since some of the host tools are compiled in that = switch (i.e. UNIX binaries), and it leads to confusing error messages. A clean separation of host tools (in a normal OPAM switch, where Mirari = is compiled for example) and the weirder backends such as Xen make = future backends easier too. -anil= From amc79@cam.ac.uk Wed Mar 06 17:17:15 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UDHxr-0004s8-NN (Exim 4.70) (return-path ); Wed, 06 Mar 2013 17:17:15 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from dhcp-128-232-140-31.eduroam.csx.cam.ac.uk ([128.232.140.31]:56939) by ppsw-50.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.157]:587) with esmtpsa (PLAIN:amc79) (TLSv1:AES128-SHA:128) id 1UDHxr-0000oD-rV (Exim 4.72) (return-path ); Wed, 06 Mar 2013 17:17:15 +0000 From: Amir Chaudhry Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Reminder of OCaml Labs-related mailing lists Date: Wed, 6 Mar 2013 17:17:18 +0000 Message-Id: <7B2B1F92-FA0B-4D82-A0CC-7C9064031608@cam.ac.uk> To: "cl-ocamllabs@lists.cam.ac.uk" Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) Cc: wg-camlp4@lists.ocaml.org, "platform@lists.ocaml.org" , infrastructure , Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Mar 2013 17:17:15 -0000 Content-Length: 1460 Lines: 41 Dear all, I thought it would be useful to point out the range of mailing lists = that OCaml Labs is involved in. Most of these have public archives and = if you're interested in any of the topics, please feel free to sign-up. = I'm sending this message to each list, so apologies if you receive it = multiple times. cl-ocamllabs@lists.cam.ac.uk Mostly used for announcements and updates. Discussions take place = elsewhere. Join: https://lists.cam.ac.uk/mailman/listinfo/cl-ocamllabs cl-mirage@lists.cam.ac.uk Day to day development talk on Mirage [1] and related projects. Archive: https://lists.cam.ac.uk/pipermail/cl-mirage/ Join: https://lists.cam.ac.uk/mailman/listinfo/cl-mirage platform@lists.ocaml.org Discussions on aspects of the OCaml Platform and things that pertain to = it. Archive: http://lists.ocaml.org/pipermail/platform/ Join: http://lists.ocaml.org/listinfo/platform infrastructure@lists.ocaml.org Discussions on continuous integration infrastructure, ocaml.org and = other things supporting OCaml. Archive: http://lists.ocaml.org/pipermail/infrastructure Join: http://lists.ocaml.org/listinfo/infrastructure wg-camlp4@lists.ocaml.org Community working group on the future of syntax extensions to OCaml. Archive: http://lists.ocaml.org/pipermail/wg-camlp4/ Join: http://lists.ocaml.org/listinfo/wg-camlp4 [1] mirage - http://openmirage.org and = http://www.cl.cam.ac.uk/projects/ocamllabs/projects/mirage.html Best wishes, Amir= From anil@recoil.org Fri Mar 08 16:39:57 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UE0Kr-0004qD-L9 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 08 Mar 2013 16:39:57 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1453735 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:31836 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UE0Kq-0006Kw-Wq (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 08 Mar 2013 16:39:57 +0000 Received: (qmail 30548 invoked by uid 634); 8 Mar 2013 16:39:56 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.15.235]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 08 Mar 2013 16:39:55 +0000 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Fwd: [ocaml-platform] OPAM 0.9.5 released Date: Fri, 8 Mar 2013 16:39:54 +0000 References: <5A92BC9D-0A67-49BA-9CB8-CD8E76F210D1@ocamlpro.com> To: " List" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Mar 2013 16:39:57 -0000 Content-Length: 2368 Lines: 64 Begin forwarded message: > From: Thomas Gazagnaire > Subject: [ocaml-platform] OPAM 0.9.5 released > Date: 8 March 2013 16:30:29 GMT > To: opam-devel , platform = >=20 > Hi, >=20 > I'm pleased to announce the release 0.9.5 of OPAM. All the features = that I though are important for the first releases are in, so you can = consider 0.9.5 to be something like 1.0-rc1. >=20 > For this release, I'm particularly keen on getting people feebdack on = the new auto-setup thing: OPAM now asks the user if he wants his local = configuration upgraded to fully use OPAM (including installation = auto-completion scripts). Use `opam init --no-setup` if you want to = continue using OPAM as before. >=20 > One notable change which will please packagers of OPAM for OS = distributions: I've added a `make with-ocamlbuild` and `make = install-with-ocamlbuild` targets to not depend on the unreleased (and = thus unpackaged) ocp-build. Otherwise, when using the development = version of OPAM, it is still recommended to use ocp-build (as it is much = faster).=20 >=20 > The changelog: > * If necessary, apply patches and substitute files before removing a = package > * Fix `opam remove --keep-build-dir` keeps the folder if a = source archive is extracted > * Add build and install rules using ocamlbuild to help distro = packagers > * Support arbitrary level of nested subdirectories in packages = repositories > * Add `opam config exec "CMD ARG1 ... ARGn" --switch=3DSWITCH` to = execute a command in a subshell > * Improve the behaviour of `opam update` wrt. pinned packages > * Change the default external solver criteria (only useful if you have = aspcud installed on your machine) > * Add support for global and user configuration for OPAM (`opam config = setup`) > * Stop yelling when OPAM is not up-to-date > * Update or generate `~/.ocamlinit` when running `opam init` > * Fix tests on *BSD (thx Arnaud Degroote) > * Fix compilation for the source archive >=20 > The files: > [OPAM] https://github.com/OCamlPro/opam/archive/latest.tar.gz > [OPAM+depends] http://www.ocamlpro.com/pub/opam-latest-full.tar.gz >=20 > Best, > Thomas >=20 > _______________________________________________ > Platform mailing list > Platform@lists.ocaml.org > http://lists.ocaml.org/listinfo/platform >=20 From avsm@dark.recoil.org Sun Mar 10 11:03:29 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UEe2L-0003fI-9q (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 10 Mar 2013 11:03:29 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1454674 * 0.0 FAKE_REPLY_C FAKE_REPLY_C X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:43411 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UEe2K-00059d-rR (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 10 Mar 2013 11:03:29 +0000 Received: (qmail 25972 invoked by uid 10000); 10 Mar 2013 11:03:28 -0000 Date: Sun, 10 Mar 2013 11:03:28 +0000 From: Anil Madhavapeddy To: Brian Anderson Subject: Re: [rust-dev] Rust on Xen Message-ID: <20130310110328.GI17918@dark.recoil.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <513121D2.1030603@mozilla.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: rust-dev@mozilla.org, Tim Ford , cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 11:03:29 -0000 Content-Length: 4252 Lines: 89 On Fri, Mar 01, 2013 at 01:46:58PM -0800, Brian Anderson wrote: > On 03/01/2013 08:37 AM, Tim Ford wrote: > >The OCaml-on-Xen OpenMirage (http://openmirage.org/) project is > >very interesting to me. The idea is that your OCaml application > >is compiled directly into a bootable Xen microkernel image. > >Rather than running inside an OS, your OCaml links to a "libOS", a > >library that provides all the OS services in terms of Xen > >hypercalls. > > > >What are your thoughts regarding whether the same could be > >accomplished with Rust? You'd have to reimplement the Rust > >runtime and many OS services, though perhaps the work done for > >OpenMirage could be of some help there. > > > >It seems like Rust would be a great language for this. > > I am interested in this. What they are doing does sound a lot like > Rust and I would like us to get to the point where we can write > kernel code in Rust. As Graydon mentioned though, our runtime code > isn't structured to allow this yet, but we are inching closer. We > probably shouldn't try to do it until after the scheduler rewrite > because `core` depends on a bunch of runtime code that doesn't have > a future and isn't worth porting to kernelspace. Afterword though, I > expect the relationship between `core` and the scheduler to be > [reversed], so that all the runtime machinery will be optional. At > that point it should take just a very small platform abstraction > layer to implement `core`, and we can think about the multithreading > separately. > > [reversed]: https://github.com/mozilla/rust/issues/5157 > > I talked briefly with Anil (from openmirage) about this subject last > year, but it would be nice to touch base again sometime. I've copied > him on this thread. We've been doing a bunch of work to make it easier to share unikernel code between different languages. Most of this is in the Xen control stack, where having a low-latency VM interface makes it easier to spawn virtual machines on-demand (instead of the seconds it currently takes in many cases). As for the actual runtimes, we've taken an extreme position in Mirage to strip away much of the C code: - cooperative threading in OCaml, with everything boiling down to a single Xen domain poll. No preemptive threading at all means that the runtime is extremely simple, but the only way to get parallelism is via multiple virtual machines. - support for 1 vCPU only. For many services such as HTTP, it's easier to spawn multiple single-vCPU VMs than it is to deal with multi-vCPU scheduling. - compile-time specialisation to let configuration directives be compiled directly into the application. Most of the Mirage libraries are 'policy-free', and the application glues it all together. This giives us a lot of flexibility when building applications, but makes the build system much more complicated. We're building a system called 'Mirari' which hides much of the build-time complexity via a config DSL, though: http://github.com/mirage/mirari - C bindings are mostly banned, which means that we've had to reimplement most protocols in pure OCaml (including device drivers). The huge upside to this is that we can get rid of almost all of the usual embedded libC. We only require a malloc and printf implementation at the moment, and should be able to get rid of the latter quite soon with a smarter Printf library. All of these restrictions can be relaxed, at the expense of greater complexity in the runtime. I imagine that's where a lot of the interoperability will come in: start with a non-preemptive boot library, add on threading support, and then bits of libC. It should be quite easy to build a multi-language unikernel that calls back from the OCaml boot+TCP stack into the Rust runtime. See this GSoC proposal: http://wiki.xen.org/wiki/GSoC_2013#Towards_a_multi-language_unikernel_substrate_for_Xen Also Xenopsd, which is the simple toolstack that Dave Scott is working on at Citrix: http://wiki.xen.org/wiki/Xenopsd You can also read more information about Mirage in the upcoming ASPLOS 2013 paper: http://anil.recoil.org/papers/2013-asplos-mirage.pdf cheers, -- Anil Madhavapeddy http://anil.recoil.org From vb@vb.fdn.fr Sun Mar 10 12:33:48 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UEfRk-00056K-Qz (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 10 Mar 2013 12:33:48 +0000 X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1454674 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:47578) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UEfRk-00051R-7c (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 10 Mar 2013 12:33:48 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id CAFE5C0A6A for ; Sun, 10 Mar 2013 13:33:47 +0100 (CET) Message-ID: <513C7DAB.2080700@vb.fdn.fr> Date: Sun, 10 Mar 2013 12:33:47 +0000 From: "Vincent B." User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130221 Thunderbird/17.0.3 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Notes on Mirari Content-Type: multipart/mixed; boundary="------------030407070803020302000107" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 12:33:48 -0000 Content-Length: 4955 Lines: 143 This is a multi-part message in MIME format. --------------030407070803020302000107 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Hi everybody, As per Anil's request, here are some notes I used while doing the talk about Mirari last friday at OCamlLabs. I will try to write a blog post at openmirage.org about Mirari as well, and then hopefully be able to start developping the planned features…! Cheers, Vincent NB: On the current hello world page at openmirage.org, there is mention of a "ping server" that I cannot find anywhere (absent from the old @avsm/mirage and @mirage/mirage-skeleton as well). Is there somewhere where I could find this code ? --------------030407070803020302000107 Content-Type: text/plain; charset=UTF-8; name="mirari_points.txt" Content-Transfer-Encoding: 8bit Content-Disposition: attachment; filename="mirari_points.txt" # Mirari, what is it, and current state: ---------------------------------------- Basically a tool "wrap the complexity of developping a Mirage unikernel". Written by Thomas Gazagnaire during its stay at the CL in the context of an OCamlPro/OcamlLabs partnership. A reminder: ----------- There are 3 possible compilers for Mirage: Unix-socket (normal compiler), Unix-direct (for use with tap interfaces to emulate ethernet), and Xen. * Unix-socket enables you to develop with the normal compiler but you don’t have access to the fancy network stack. * Unix-direct enable the full network stack but you need to have a system that supports the tun/tap virtual interface. * Xen-direct compiles into the real Xen unikernel but it do not allow debugging using UNIX tools. So basically you probably develop under Unix-direct, and deploy with Xen-direct, requiring to do frequent opam switch. ----------------------------------------------------- Now let’s see the workflow of developping unikernels: ----------------------------------------------------- 0. Install opam dependencies 1. write the code logic 2. write the boilerplate code that embed the unikernel config. This config is different according to where / in what mode you want to run the unikernel, so you have to modify that according to where you want to run it. 4. opam switch in the right compiler 5. specific steps to run/deploy it (Unix -> just run | Xen -> do additionnal steps in order to run it on your laptop/server/EC2) ------ -> All the above is tedious but for 1. -> Mirari already automatise steps 0 and 2, will do 4 and 5 in the future. ------------- How it works: ------------- With Mirari, you write the configuration in a config file, and Mirari will create boilerplate code to embed the config into the Unikernel. That’s essentially the main functionality of Mirari right now. The steps to deploy a Mirage app: - Write the Mirari config file that embed the unikernel config and information that Mirari needs in order to build and configure the app. - Use "mirari configure", that will generate an obuild config file, and boilerplate code to embed the config into the unikernel, it will also install the unikernel dependencies with OPAM. - Use "mirari build", it just build the project for now, and creates a link to the resulting exe or xen image. -------------------------------------------------------------- The missing piece: Mirari to autoswitch opam, and "mirari run" -------------------------------------------------------------- - The autoswitch idea: You put the desired mode of compilation in the mirari config file and it uses the fancy new feature of opam to launch a command a la "with_compiler_X make...". You always work under your standard (non mirage) compiler. No need to switch anymore. That has been taken care of on the opam side and require minor modification on mirari which will be done shortly. - mirari run: * Specify deployment parameters in mirari config file : * For unix-socket, just run the exe * For unix-direct, setup the tap virtual interface. This is currently done by code residing in mirage-platform, with the configuration parameters hardcoded. This will now be made by mirari using the config file parameters. * For xen, the kernel can be run locally (via xl/libvirt), on a testserver (ssh commands?), or amazon EC2. ---------------- Work to be done: ---------------- - Extend the config file syntax to take account of deployements parameters. - Isolate the tap creation code of mirage-platform into an ocaml-tuntap library. Apparently Macs need the kernel to setup its tap itself. For Linux, mirari could have taken care of creating/destroying the tunnel, which is cleaner. - Make mirari use the new opam feature to run commands under a particular compiler. - Generate config files for running xen kernels (xl config file, EC2 wrapping and config), and maybe issue the command to start it. --------------030407070803020302000107-- From Richard.Mortier@nottingham.ac.uk Sun Mar 10 14:08:27 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UEgvL-0006KZ-Ho (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 10 Mar 2013 14:08:27 +0000 X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1454674 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine01-20433-1.icritical.com ([93.95.15.169]:33875) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1UEgvL-0001Mq-78 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 10 Mar 2013 14:08:27 +0000 Received: (qmail 30707 invoked from network); 10 Mar 2013 14:03:13 -0000 Received: from localhost (127.0.0.1) by engine01-20433-1.icritical.com with SMTP; 10 Mar 2013 14:03:13 -0000 Received: from engine01-20433-1.icritical.com ([127.0.0.1]) by localhost (engine01-20433-1.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 30501-01 for ; Sun, 10 Mar 2013 14:03:12 +0000 (GMT) Received: (qmail 30453 invoked by uid 599); 10 Mar 2013 14:03:04 -0000 Received: from unknown (HELO smtp4.nottingham.ac.uk) (128.243.220.65) by engine01-20433-1.icritical.com (qpsmtpd/0.28) with ESMTP; Sun, 10 Mar 2013 14:03:04 +0000 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp4.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1UEgqI-0004zs-VF for cl-mirage@lists.cam.ac.uk; Sun, 10 Mar 2013 14:03:14 +0000 From: Richard Mortier To: Vincent B. Date: Sun, 10 Mar 2013 14:03:14 +0000 Subject: Re: Notes on Mirari Thread-Topic: Notes on Mirari Thread-Index: Ac4dmAGuRREFcCNNReuYlctL12ZVDQ== Message-ID: <6A95AC65-39C6-4B4B-A33C-37741421E131@nottingham.ac.uk> References: <513C7DAB.2080700@vb.fdn.fr> In-Reply-To: <513C7DAB.2080700@vb.fdn.fr> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine01-20433-1.icritical.com Cc: "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 14:08:27 -0000 Content-Length: 1431 Lines: 33 On 10 Mar 2013, at 12:33, Vincent B. wrote: > As per Anil's request, here are some notes I used while doing the talk=20 > about Mirari last friday at OCamlLabs. I will try to write a blog post=20 > at openmirage.org about Mirari as well, and then hopefully be able to=20 > start developping the planned features=85! cool! :) > NB: On the current hello world page at openmirage.org, there is mention=20 > of a "ping server" that I cannot find anywhere (absent from the old=20 > @avsm/mirage and @mirage/mirage-skeleton as well). Is there somewhere=20 > where I could find this code ? probably in the old mirage-perf repo, under net/ -- try @avsm/mirage-perf --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From Richard.Mortier@nottingham.ac.uk Sun Mar 10 14:11:22 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UEgyA-0006Ms-B9 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 10 Mar 2013 14:11:22 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1454674 * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine01-20433-1.icritical.com ([93.95.15.169]:34266) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UEgy9-00023Q-Sg (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 10 Mar 2013 14:11:22 +0000 Received: (qmail 2631 invoked from network); 10 Mar 2013 14:11:10 -0000 Received: from localhost (127.0.0.1) by engine01-20433-1.icritical.com with SMTP; 10 Mar 2013 14:11:10 -0000 Received: from engine01-20433-1.icritical.com ([127.0.0.1]) by localhost (engine01-20433-1.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 02266-05 for ; Sun, 10 Mar 2013 14:11:09 +0000 (GMT) Received: (qmail 30453 invoked by uid 599); 10 Mar 2013 14:03:04 -0000 Received: from unknown (HELO smtp4.nottingham.ac.uk) (128.243.220.65) by engine01-20433-1.icritical.com (qpsmtpd/0.28) with ESMTP; Sun, 10 Mar 2013 14:03:04 +0000 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp4.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1UEgqI-0004zs-VF for cl-mirage@lists.cam.ac.uk; Sun, 10 Mar 2013 14:03:14 +0000 From: Richard Mortier To: Vincent B. Date: Sun, 10 Mar 2013 14:03:14 +0000 Subject: Re: Notes on Mirari Thread-Topic: Notes on Mirari Thread-Index: Ac4dmAGuRREFcCNNReuYlctL12ZVDQ== Message-ID: <6A95AC65-39C6-4B4B-A33C-37741421E131@nottingham.ac.uk> References: <513C7DAB.2080700@vb.fdn.fr> In-Reply-To: <513C7DAB.2080700@vb.fdn.fr> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine01-20433-1.icritical.com Cc: "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 14:11:22 -0000 Content-Length: 1431 Lines: 33 On 10 Mar 2013, at 12:33, Vincent B. wrote: > As per Anil's request, here are some notes I used while doing the talk=20 > about Mirari last friday at OCamlLabs. I will try to write a blog post=20 > at openmirage.org about Mirari as well, and then hopefully be able to=20 > start developping the planned features=85! cool! :) > NB: On the current hello world page at openmirage.org, there is mention=20 > of a "ping server" that I cannot find anywhere (absent from the old=20 > @avsm/mirage and @mirage/mirage-skeleton as well). Is there somewhere=20 > where I could find this code ? probably in the old mirage-perf repo, under net/ -- try @avsm/mirage-perf --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From anil@recoil.org Sun Mar 10 14:20:52 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UEh7M-0006Ud-Jp (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 10 Mar 2013 14:20:52 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1454674 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:28939 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UEh7L-0002Y9-qW (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 10 Mar 2013 14:20:52 +0000 Received: (qmail 751 invoked by uid 634); 10 Mar 2013 14:20:51 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.38]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sun, 10 Mar 2013 14:20:50 +0000 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: Notes on Mirari From: Anil Madhavapeddy In-Reply-To: <6A95AC65-39C6-4B4B-A33C-37741421E131@nottingham.ac.uk> Date: Sun, 10 Mar 2013 14:20:55 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <0D8B0199-B9D2-419D-A980-2893E3065AC2@recoil.org> References: <513C7DAB.2080700@vb.fdn.fr> <6A95AC65-39C6-4B4B-A33C-37741421E131@nottingham.ac.uk> To: Richard Mortier X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "Vincent B." , "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Mar 2013 14:20:52 -0000 Content-Length: 959 Lines: 34 On 10 Mar 2013, at 14:03, Richard Mortier = wrote: >=20 >> NB: On the current hello world page at openmirage.org, there is = mention=20 >> of a "ping server" that I cannot find anywhere (absent from the old=20= >> @avsm/mirage and @mirage/mirage-skeleton as well). Is there somewhere=20= >> where I could find this code ? >=20 > probably in the old mirage-perf repo, under net/ -- try = @avsm/mirage-perf >=20 Here: https://github.com/mirage/mirage-skeleton/blob/master/ping/ping.ml Note that there's no actual network code in there. That's because the Mirari configuration includes: ip-use-dhcp: true which should bind a network manager to any VIFs that are plugged into = the VM. Each network thread will respond to ping requests automatically, = without the application needing to do anything special. The ICMP response code = is in; https://github.com/mirage/mirage-net/blob/master/direct/lib/icmp.ml -anil= From bogdanmatican@gmail.com Wed Mar 13 12:34:11 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UFksl-0003oM-SB (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 13 Mar 2013 12:34:11 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1455441 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.223.180 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (bogdanmatican[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ie0-f180.google.com ([209.85.223.180]:53505) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UFksl-0000NK-WI (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 13 Mar 2013 12:34:11 +0000 Received: by mail-ie0-f180.google.com with SMTP id bn7so1289963ieb.25 for ; Wed, 13 Mar 2013 05:34:10 -0700 (PDT) X-Received: by 10.50.222.167 with SMTP id qn7mr15948035igc.42.1363178050297; Wed, 13 Mar 2013 05:34:10 -0700 (PDT) MIME-Version: 1.0 Received: by 10.231.156.69 with HTTP; Wed, 13 Mar 2013 05:33:50 -0700 (PDT) From: Bogdan Alexandru Matican Date: Wed, 13 Mar 2013 12:33:50 +0000 Message-ID: Subject: Questions on using orm To: cl-mirage@lists.cam.ac.uk Content-Type: multipart/alternative; boundary=14dae9340f877427d504d7cd9f95 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 12:34:11 -0000 Content-Length: 5287 Lines: 135 --14dae9340f877427d504d7cd9f95 Content-Type: text/plain; charset=ISO-8859-1 Hello, I've integrated using orm (and thus sqlite3) for persistent storage in the project I am working on, but I have a couple of questions. I am not sure if I have misunderstood something, or what I have noticed is by design... So let's say I have the following: type book = { book_name : string; } and chapter = { chapter_fk_book : book; chapter_name : string; } with orm let db_name = "book.db" let book_db = book_init db_name let chapter_db = chapter_init db_name let some_book = { book_name = "My book"; } let some_chapter = { chapter_fk_book = some_book; chapter_name = "Introduction"; } --- Question 1: is there any way to actually use foreign keys to query? I would like to do basically: SELECT c.* from Chapter c, Book b where b.book_name = "My book"; At the moment, I am forced to scrap foreign keys all together and hash my objects by feature combinations that make the hashes unique. This also brings me to my second questions... --- Question 2: is there any way to do cascading deletes? chapter_delete ~recursive:true ~db:chapter_db some_chapter This does something I cannot really envision ever needing...It allows me to delete the chapter _and_ the book (doesn't really seem like something too natural), where as I would like the reverse to actually happen: book_delete ~recursive:true ~db:book_db some_book I would expect this to delete the book and everything that actually keeps foreign keys to the book item, much like cascading deletes in SQL. Instead, at the moment, I have to query for the book object, then use it to query for all chapters underneath, use each to call chapter_delete individually, and afterwards delete the book. There is something here that feels it won't just be optimized away for me :) --- If anyone can share some insight on this, that would be great. I have a working prototype, as is, but it feels slightly contrived to potentially hack around certain things... Best, Bogdan --14dae9340f877427d504d7cd9f95 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hello,

I've integrated using = orm (and thus sqlite3) for persistent storage in the project I am working o= n, but I have a couple of questions. I am not sure if I have misunderstood = something, or what I have noticed is by design...

So let's say I have the following:

type book =3D {
=A0 book_n= ame : string;
} and chapter =3D {
=A0 chapt= er_fk_book : book;
=A0 chapter_name : string;
} with orm

let db_name =3D "book.db"
let book_db =3D book_init db_name
let chapter_db =3D ch= apter_init db_name

let some_book =3D {
=A0 boo= k_name =3D "My book";
}

let some_chapter =3D {
=A0 chapter_fk_book =3D = some_book;
=A0 chapter_name =3D "Introduction";
}=

---
Question 1: is th= ere any way to actually use foreign keys to query?

I would like to do basically:

SELECT c.* from Chapter c, Book b where b.book_name =3D "My book&qu= ot;;

At the moment, I am forced to= scrap foreign keys all together and hash my objects by feature combination= s that make the hashes unique.

This also brings me to my second questions.= ..
---
Question 2: is there any way to do c= ascading deletes?

chapter_delete ~= recursive:true ~db:chapter_db some_chapter

This does something I cannot really envisio= n ever needing...It allows me to delete the chapter _and_ the book (doesn&#= 39;t really seem like something too natural), where as I would like the rev= erse to actually happen:

book_delete ~recursive:true ~db:book_db som= e_book

I would expect this to delete t= he book and everything that actually keeps foreign keys to the book item, m= uch like cascading deletes in SQL.

Instead, at the moment, I have to query for= the book object, then use it to query for all chapters underneath, use eac= h to call chapter_delete individually, and afterwards delete the book. Ther= e is something here that feels it won't just be optimized away for me := )
---

If anyone can share som= e insight on this, that would be great. I have a working prototype, as is, = but it feels slightly contrived to potentially hack around certain things..= .

Best,
Bogdan

--14dae9340f877427d504d7cd9f95-- From anil@recoil.org Thu Mar 14 10:55:05 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UG5oP-0004f4-Hq (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 14 Mar 2013 10:55:05 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1455841 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:19363 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UG5oO-0002w8-pk (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 14 Mar 2013 10:55:05 +0000 Received: (qmail 4029 invoked by uid 634); 14 Mar 2013 10:55:03 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.164]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 14 Mar 2013 10:55:03 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: possible cstruct bug? From: Anil Madhavapeddy In-Reply-To: Date: Thu, 14 Mar 2013 10:54:58 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <074FD150-B3D0-469E-A9E4-4E844F48B74F@recoil.org> References: To: Arjun Guha X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2013 10:55:05 -0000 Content-Length: 1897 Lines: 56 [cc cl-mirage: this references bug: = https://github.com/mirage/ocaml-cstruct/issues/3 ] On 11 Mar 2013, at 21:09, Arjun Guha wrote: >=20 > I finally had a change to look at this, and I think I see what's = needed. >=20 > IMO, for a 32-bit enum, cstruct should also generate the functions = int32_to_foo and foo_to_int32. (Similarly for 64-bit enums). >=20 > Let me know if you agree and I'll do ahead. >=20 > It is a straightforward fix. I think the cleanest solution is to = create three copies of the output_enum function, each specialized for = each field-width. It is theoretically possible to still have just one = output_enum func, but the dependency between function names, types, and = the arguments to output_enum would be really messy. Thanks for looking at this; I think that having a specialised field = width function is easiest. It's a little annoying on 64-bit = architectures though, since an unboxed native integer is sufficient to = encode a 32-bit integer without any boxing (which is why we didn't = notice this bug before). However, it's probably not worth all the configure script trouble to = make this change, so generating specialised int{32/64}_to_foo functions = is best. cheers, Anil >=20 > On 20 Feb 2013, at 14:32, Arjun Guha wrote: >=20 > > Anil, > > > > Where do you take bug reports on cstruct and other libraries? It = looks like you've disabled issues in mirage/ocaml-cstruct. > > > > Anyway, I've installed cstruct from HEAD. With the following = definition, I get an error saying "Integer literal exceeds the range of = representable integers of type int32": > > > > cenum ofp_port_no { > > OFPP_ANY =3D 0xffffffff > > } as int32_t > > > > I think that ought to work. I can't postfix an "l" or "L" either. = I'd be happy (actually, would enjoy) fixing if you can confirm. > > > > Arjun >=20 >=20 From thomas.gazagnaire@gmail.com Thu Mar 14 11:45:30 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UG6bC-0007kb-RW (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 14 Mar 2013 11:45:30 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1455841 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.42 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-wg0-f42.google.com ([74.125.82.42]:56986) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UG6bB-0001jE-sA (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 14 Mar 2013 11:45:30 +0000 Received: by mail-wg0-f42.google.com with SMTP id 12so931842wgh.3 for ; Thu, 14 Mar 2013 04:45:29 -0700 (PDT) X-Received: by 10.180.78.168 with SMTP id c8mr3257262wix.27.1363261529633; Thu, 14 Mar 2013 04:45:29 -0700 (PDT) Received: from [192.168.0.12] (gou06-3-88-170-165-56.fbx.proxad.net. [88.170.165.56]) by mx.google.com with ESMTPS id r7sm3492085wiz.2.2013.03.14.04.45.27 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 14 Mar 2013 04:45:28 -0700 (PDT) Subject: Re: Questions on using orm Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: Date: Thu, 14 Mar 2013 12:45:24 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Bogdan Alexandru Matican X-Mailer: Apple Mail (2.1085) Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2013 11:45:30 -0000 Content-Length: 2087 Lines: 72 Hi, Two answers to your questions: =3D=3D=3D Answer 1 (the direct answer) =3D=3D=3D > Question 1: is there any way to actually use foreign keys to query? >=20 > I would like to do basically: >=20 > SELECT c.* from Chapter c, Book b where b.book_name =3D "My book"; You can use custom functions: let get_chapters_by_book_name db book_name =3D chapter_get db ~custom:(fun c -> c.chapter_fk_book.book_name =3D = book_name) > --- > Question 2: is there any way to do cascading deletes? >=20 > chapter_delete ~recursive:true ~db:chapter_db some_chapter >=20 > This does something I cannot really envision ever needing...It allows = me to delete the chapter _and_ the book (doesn't really seem like = something too natural), where as I would like the reverse to actually = happen: >=20 > book_delete ~recursive:true ~db:book_db some_book >=20 > I would expect this to delete the book and everything that actually = keeps foreign keys to the book item, much like cascading deletes in SQL. >=20 > Instead, at the moment, I have to query for the book object, then use = it to query for all chapters underneath, use each to call chapter_delete = individually, and afterwards delete the book. There is something here = that feels it won't just be optimized away for me :) This is not possible (as there is no notion of foreign key in ORM). For = a better solution, see answer 2. =3D=3D Answer 2 (the indirect one) =3D=3D > type book =3D { > book_name : string; > } and chapter =3D { > chapter_fk_book : book; > chapter_name : string; > } with orm This is not the most natural way to define book/chapters if you would do = it without persistence in mind. For instance, I would do: type book =3D { book_name : string; book_chapters: chapter list; } and chapter =3D { chapter_name =3D string; } with orm let get_chapters_by_book_name db book_name =3D match get_book db ~book_name:(`Eq book_name) with | [] -> [] | [b] -> b.chapters | _ -> failwith "too many books with the same name" and then recursive deletion works as expected. Hope this helps, Thomas From amc79@cam.ac.uk Thu Mar 14 16:53:39 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGBPP-0005Iz-M2 (Exim 4.70) (return-path ); Thu, 14 Mar 2013 16:53:39 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from dhcp-128-232-140-85.eduroam.csx.cam.ac.uk ([128.232.140.85]:53743) by ppsw-41.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.156]:587) with esmtpsa (PLAIN:amc79) (TLSv1:AES128-SHA:128) id 1UGBPP-00019o-RY (Exim 4.72) (return-path ); Thu, 14 Mar 2013 16:53:39 +0000 From: Amir Chaudhry Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Wireframe demo of OCaml.og Date: Thu, 14 Mar 2013 16:53:47 +0000 Message-Id: To: infrastructure Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) Cc: "platform@lists.ocaml.org" , Mirage List , "cl-ocamllabs@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2013 16:53:39 -0000 Content-Length: 517 Lines: 16 Dear all, I've been working on putting together wireframes to demonstrate aspects = of the design and functionality we'd like to introduce into the = OCaml.org website. Today, I published a blog post summarising this = work, which includes screencasts and a link to a clickable demo you can = play with. http://amirchaudhry.com/wireframe-demos-for-ocamlorg I'd be keen to know what you think so please send any feedback via the = infrastructure mailing list (infrastructure@lists.ocaml.org). Best wishes, Amir= From sebastien.mondet@gmail.com Thu Mar 14 18:45:33 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGD9h-0001Eh-Do (Exim 4.70) (return-path ); Thu, 14 Mar 2013 18:45:33 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1455841 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.169 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (sebastien.mondet[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-wi0-f169.google.com ([209.85.212.169]:42107) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UGD9d-0000cA-XO (Exim 4.72) (return-path ); Thu, 14 Mar 2013 18:45:33 +0000 Received: by mail-wi0-f169.google.com with SMTP id l13so4516398wie.2 for ; Thu, 14 Mar 2013 11:45:29 -0700 (PDT) X-Received: by 10.180.100.10 with SMTP id eu10mr36219643wib.4.1363286729120; Thu, 14 Mar 2013 11:45:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.9.167 with HTTP; Thu, 14 Mar 2013 11:45:08 -0700 (PDT) In-Reply-To: References: From: Sebastien Mondet Date: Thu, 14 Mar 2013 14:45:08 -0400 Message-ID: Subject: Re: [ocaml-infra] Wireframe demo of OCaml.og To: Amir Chaudhry Content-Type: multipart/alternative; boundary=f46d044280c837640b04d7e6ed20 Cc: "platform@lists.ocaml.org" , Mirage List , infrastructure , "cl-ocamllabs@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2013 18:45:33 -0000 Content-Length: 3423 Lines: 98 --f46d044280c837640b04d7e6ed20 Content-Type: text/plain; charset=UTF-8 Hi Very impressive work!! One question: even without fonts and colors, the wireframes are supposed to be about page layout, right? On the videos, the pseudo-browser is higher than wider while most people seem to have nowadays w > h. Is there a way to view how the layout is intended to adapt to other "page geometries" (or to mobile-browsers by the way)? or is it too early in the design? Thanks & Cheers Seb On Thu, Mar 14, 2013 at 12:53 PM, Amir Chaudhry wrote: > Dear all, > > I've been working on putting together wireframes to demonstrate aspects of > the design and functionality we'd like to introduce into the OCaml.org > website. Today, I published a blog post summarising this work, which > includes screencasts and a link to a clickable demo you can play with. > > http://amirchaudhry.com/wireframe-demos-for-ocamlorg > > I'd be keen to know what you think so please send any feedback via the > infrastructure mailing list (infrastructure@lists.ocaml.org). > > Best wishes, > Amir > _______________________________________________ > Infrastructure mailing list > Infrastructure@lists.ocaml.org > http://lists.ocaml.org/listinfo/infrastructure > --f46d044280c837640b04d7e6ed20 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Hi

Very im= pressive work!!

One question:
even without fonts and color= s, the wireframes are supposed to be about page layout, right?
On the v= ideos, the pseudo-browser is higher than wider while most people seem to ha= ve nowadays w > h.
Is there a way to view how the layout is intended to adapt to other "p= age geometries" (or to mobile-browsers by the way)? or is it too early= in the design?


Thanks & Cheers
Seb






<= br>
On Thu, Mar 14, 2013 at 12:53 PM, Amir Ch= audhry <amc79@cam.ac.uk> wrote:
Dear all,

I've been working on putting together wireframes to demonstrate aspects= of the design and functionality we'd like to introduce into the OCaml.= org website. =C2=A0Today, I published a blog post summarising this work, wh= ich includes screencasts and a link to a clickable demo you can play with.<= br>
http://amirchaudhry.com/wireframe-demos-for-ocamlorg

I'd be keen to know what you think so please send any feedback via the = infrastructure mailing list (infrastructure@lists.ocaml.org).

Best wishes,
Amir
_______________________________________________
Infrastructure mailing list
Infrastructure@lists.ocam= l.org
http://lists.ocaml.org/listinfo/infrastructure

--f46d044280c837640b04d7e6ed20-- From thomas.gazagnaire@gmail.com Thu Mar 14 20:28:36 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGElQ-0002qG-OP (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 14 Mar 2013 20:28:36 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1455841 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.169 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-wi0-f169.google.com ([209.85.212.169]:59920) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UGElP-0004qG-sL (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 14 Mar 2013 20:28:36 +0000 Received: by mail-wi0-f169.google.com with SMTP id l13so4610470wie.4 for ; Thu, 14 Mar 2013 13:28:35 -0700 (PDT) X-Received: by 10.180.83.39 with SMTP id n7mr6641209wiy.8.1363292915641; Thu, 14 Mar 2013 13:28:35 -0700 (PDT) Received: from [192.168.0.12] (gou06-3-88-170-165-56.fbx.proxad.net. [88.170.165.56]) by mx.google.com with ESMTPS id fg6sm6355419wib.10.2013.03.14.13.28.33 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 14 Mar 2013 13:28:34 -0700 (PDT) Sender: Thomas Gazagnaire From: Thomas Gazagnaire Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: OPAM 1.0.0 released ! Date: Thu, 14 Mar 2013 21:28:32 +0100 Message-Id: <999F571F-57A2-46F0-9DA8-83DC7026363E@ocamlpro.com> To: OCaml List Mime-Version: 1.0 (Apple Message framework v1085) X-Mailer: Apple Mail (2.1085) Cc: platform , opam-devel , Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Mar 2013 20:28:36 -0000 Content-Length: 5924 Lines: 148 I nm *very* happy to announce the first official release of OPAM! Many of you already know and use OPAM so I won't be long. Please read = http://www.ocamlpro.com/blog/2013/01/17/opam-beta.html for a longer = description. 1.0.0 fixes many bugs and add few new features to the previously = announced beta-release. The most visible new feature, which should be useful for beginners with = OCaml and OPAM, is an auto-configuration tool. This tool easily enables = all the features of OPAM (auto-completion, fix the loading of scripts = for the toplevel, opam-switch-eval alias, etc). This tool runs = interactively on each `opam init` invocation. If you don't like OPAM to = change your configuration files, use `opam init --no-setup`. If you = trust the tool blindly, use `opam init --auto-setup`. You can later = review the setup by doing `opam config setup --list` and call the tool = again using `opam config setup` (and you can of course manually edit = your ~/.profile (or ~/.zshrc for zsh users), ~/.ocamlinit and = ~/.opam/opam-init/*). Please report: - Bug reports and feature requests for the OPAM tool: = http://github.com/OCamlPro/opam/issues - Packaging issues or requests for a new package: = http://github.com/OCamlPro/opam-repository/issues - General queries to: http://lists.ocaml.org/listinfo/platform - More specific queries about the internals of OPAM to: = http://lists.ocaml.org/listinfo/opam-devel On behalf on the OPAM team, Thomas =3D=3D=3D Install =3D=3D=3D Packages for Debian and OSX (at least homebrew) should follow shortly = and I'm looking for volunteers to create and maintain rpm packages. The = binary installer is up-to-date for Linux and Darwin 64-bit = architectures, the 32-bit version for Linux should arrive shortly. If you want to build from sources, the full archive (including = dependencies) is available here: =20 http://www.ocamlpro.com/pub/opam-full-latest.tar.gz =3D=3D=3D Upgrade =3D=3D=3D If you are upgrading from 0.9.* you won't have anything special to do = apart installing the new binary. You can then update your package = metadata by running `opam update`. If you want to use the auto-setup = feature, remove the "eval `opam config env` line you have previously = added in your ~/.profile and run `opam config setup --all`. So everything should be fine. But you never know ... so if something = goes horribly wrong in the upgrade process (of if your are upgrading = from an old version of OPAM) you can still trash your ~/.opam, manually = remove what OPAM added in your ~/.profile (~/.zshrc for zsh users) and = ~/.ocamlinit, and start again from scratch.=20 =3D=3D=3D Random stats =3D=3D=3D Great success on github. Thanks everybody for the great contributions! https://github.com/OCamlPro/opam: +2000 commits, 26 contributors https://github.com/OCamlPro/opam-repository: +1700 commits, 75 = contributors, 370+ packages on http://opam.ocamlpro.com/ +400 unique visitor per week, 15k 'opam update' per week +1300 unique visitor per month, 55k 'opam update' per month 3815 unique visitor since the alpha release =3D=3D=3D Changelog =3D=3D=3D The full change-log since the beta release in January: 1.0.0 [Mar 2013] * Improve the lexer performance (thx to @oandrieu) * Fix various typos (thx to @chaudhuri) * Fix build issue (thx to @avsm) 0.9.6 [Mar 2013] * Fix installation of pinned packages on BSD (thx to @smondet) * Fix configuration for zsh users (thx to @AltGr) * Fix loading of `~/.profile` when using dash (eg. in Debian/Ubuntu) * Fix installation of packages with symbolic links (regression = introduced in 0.9.5) 0.9.5 [Mar 2013] * If necessary, apply patches and substitute files before removing a = package * Fix `opam remove --keep-build-dir` keeps the folder if a source = archive is extracted * Add build and install rules using ocamlbuild to help distro packagers * Support arbitrary level of nested subdirectories in packages = repositories * Add `opam config exec "CMD ARG1 ... ARGn" --switch=3DSWITCH` to = execute a command in a subshell * Improve the behaviour of `opam update` wrt. pinned packages * Change the default external solver criteria (only useful if you have = aspcud installed on your machine) * Add support for global and user configuration for OPAM (`opam config = setup`) * Stop yelling when OPAM is not up-to-date * Update or generate `~/.ocamlinit` when running `opam init` * Fix tests on *BSD (thx Arnaud Degroote) * Fix compilation for the source archive 0.9.4 [Feb 2013] * Disable auto-removal of unused dependencies. This can now be enabled = on-demand using `-a` * Fix compilation and basic usage on Cygwin * Fix BSD support (use `type` instead of `which` to detect existing = commands) * Add a way to tag external dependencies in OPAM files * Better error messages when trying to upgrade pinned packages * Display `depends` and `depopts` fields in `opam info` * `opam info pkg.version` shows the metadata for this given package = version * Add missing `doc` fields in `.install` files * `opam list` now only shows installable packages 0.9.3 [Feb 2013] * Add system compiler constraints in OPAM files * Better error messages in case of conflicts * Cleaner API to install/uninstall packages * On upgrade, OPAM now perform all the remove action first * Use a cache for main storing OPAM metadata: this greatly speed-up OPAM = invocations * after an upgrade, propose to reinstall a pinned package only if there = were some changes * improvements to the solver heuristics * better error messages on cyclic dependencies 0.9.2 [Jan 2013] * Install all the API files * Fix `opam repo remove repo-name` * speed-up `opam config env` * support for `opam-foo` scripts (which can be called using `opam foo`) * 'opam update pinned-package' works * Fix 'opam-mk-repo -a' * Fix 'opam-mk-repo -i' * clean-up pinned cache dir when a pinned package fails to install 0.9.1 [Jan 2013] * Use ocaml-re 1.2.0 From amc79@cam.ac.uk Fri Mar 15 01:23:06 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGJMQ-0000z7-Tp (Exim 4.70) (return-path ); Fri, 15 Mar 2013 01:23:06 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from cpc17-cmbg14-2-0-cust360.5-4.cable.virginmedia.com ([86.6.155.105]:56442 helo=[192.168.0.10]) by ppsw-51.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.158]:587) with esmtpsa (PLAIN:amc79) (TLSv1:AES128-SHA:128) id 1UGJMQ-0005Lh-Yq (Exim 4.72) (return-path ); Fri, 15 Mar 2013 01:23:06 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: [ocaml-infra] Wireframe demo of OCaml.og From: Amir Chaudhry In-Reply-To: Date: Fri, 15 Mar 2013 01:23:13 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <94DE2397-ED49-438C-A159-A0ADB7C08A79@cam.ac.uk> References: To: Sebastien Mondet X-Mailer: Apple Mail (2.1499) Cc: "platform@lists.ocaml.org" , Mirage List , infrastructure , "cl-ocamllabs@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 01:23:07 -0000 Content-Length: 2540 Lines: 77 Hi Seb, Thanks for the feedback. You're right about the width vs height and in these wireframes you're = actually seeing the whole page at once. All displays (apart from = phones) should see the full width of the page but may have to scroll to = see content further down the page. People on large monitors may not = have to scroll far. I should point out that I fixed the width of these = wireframes at 1024 wide, which is similar to tablet resolutions. On = wider monitors, we'd likely add more whitespace either side so as to = avoid stretching the content too far. There isn't a straightforward way to see how the site would look from a = mobile without explicitly wireframing it that way, but we are conscious = of it. Since we're using the bootstrap [1] grid, we do get = mobile-friendly views for free, provided we arrange the content well. = As an example, visit the bootstrap site (below) from a desktop and = slowly reduce the width of your browser window. You'll see how the = content reflows to fit the window until it eventually becomes a single = column (as I see it on my phone). [1] http://twitter.github.com/bootstrap/ Best wishes, Amir On 14 Mar 2013, at 18:45, Sebastien Mondet = wrote: >=20 > Hi >=20 > Very impressive work!! >=20 > One question:=20 > even without fonts and colors, the wireframes are supposed to be about = page layout, right?=20 > On the videos, the pseudo-browser is higher than wider while most = people seem to have nowadays w > h. > Is there a way to view how the layout is intended to adapt to other = "page geometries" (or to mobile-browsers by the way)? or is it too early = in the design? >=20 >=20 > Thanks & Cheers > Seb >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > On Thu, Mar 14, 2013 at 12:53 PM, Amir Chaudhry = wrote: > Dear all, >=20 > I've been working on putting together wireframes to demonstrate = aspects of the design and functionality we'd like to introduce into the = OCaml.org website. Today, I published a blog post summarising this = work, which includes screencasts and a link to a clickable demo you can = play with. >=20 > http://amirchaudhry.com/wireframe-demos-for-ocamlorg >=20 > I'd be keen to know what you think so please send any feedback via the = infrastructure mailing list (infrastructure@lists.ocaml.org). >=20 > Best wishes, > Amir > _______________________________________________ > Infrastructure mailing list > Infrastructure@lists.ocaml.org > http://lists.ocaml.org/listinfo/infrastructure >=20 From daniel.buenzli@erratique.ch Fri Mar 15 05:54:57 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGNbV-0003s9-UZ (Exim 4.70) (return-path ); Fri, 15 Mar 2013 05:54:57 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamScore: ss X-Cam-SpamDetails: score 2.0 from SpamAssassin-3.3.2-1456366 * 2.0 RCVD_IN_MAPS_RBL RBL: Relay in RBL, * http://www.mail-abuse.com/enduserinfo_rbl.html * [178.197.233.37 listed in rbl-plus.mail-abuse.ja.net] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail6.webfaction.com ([74.55.86.74]:40467 helo=smtp.webfaction.com) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UGNbV-00047t-X7 (Exim 4.72) (return-path ); Fri, 15 Mar 2013 05:54:57 +0000 Received: from [172.20.10.2] (37-233.197-178.cust.bluewin.ch [178.197.233.37]) by smtp.webfaction.com (Postfix) with ESMTP id D00AE20BF768; Fri, 15 Mar 2013 05:54:54 +0000 (UTC) Date: Fri, 15 Mar 2013 06:54:52 +0100 From: =?utf-8?Q?Daniel_B=C3=BCnzli?= To: Amir Chaudhry Message-ID: <6DFACCA6E9514EE99323A21A4954F2A2@erratique.ch> In-Reply-To: References: Subject: Re: [ocaml-platform] Wireframe demo of OCaml.og X-Mailer: sparrow 1.6.4 (build 1178) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: "=?utf-8?Q?platform=40lists.ocaml.org?=" , Mirage List , infrastructure , "=?utf-8?Q?cl-ocamllabs=40lists.cam.ac.uk?=" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 05:54:58 -0000 Content-Length: 894 Lines: 16 Hello, > On wider monitors, we'd likely add more whitespace either side so as to avoid stretching the content too far. W.r.t. page layout stretchability but on the narrower side, a good test to have in mind is to check if on a laptop you can have an 80-columns text editor and the documentation side by side without needing horizontal scrolling. Regarding the Modules tab I'm not sure I understood you well so it may already be the idea. But if the package has a single module it would be nice if it shows directly the documentation of that module to save one useless click. Also, it's more a comment for opam, but it would be nice if the documentation system could do the same locally with the same experience but showing you only the documentation of your installed packages (with a landing page that lists the packages and their respective modules for direct access). Best, Daniel From amc79@cam.ac.uk Fri Mar 15 08:48:07 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGQJ5-0000F6-HL (Exim 4.70) (return-path ); Fri, 15 Mar 2013 08:48:07 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from cpc17-cmbg14-2-0-cust360.5-4.cable.virginmedia.com ([86.6.155.105]:52489 helo=[192.168.0.7]) by ppsw-50.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.157]:587) with esmtpsa (PLAIN:amc79) (TLSv1:AES128-SHA:128) id 1UGQJ5-0008V9-qf (Exim 4.72) (return-path ); Fri, 15 Mar 2013 08:48:07 +0000 References: <6DFACCA6E9514EE99323A21A4954F2A2@erratique.ch> Mime-Version: 1.0 (1.0) In-Reply-To: <6DFACCA6E9514EE99323A21A4954F2A2@erratique.ch> Content-Type: multipart/alternative; boundary=Apple-Mail-C0E72B7C-35E7-4224-8361-347AEB640526 Content-Transfer-Encoding: 7bit Message-Id: <0EF4ED47-02EA-4FC5-B8DC-9FD0DE023949@cam.ac.uk> X-Mailer: iPhone Mail (10A525) From: Amir Chaudhry Subject: Re: [ocaml-platform] Wireframe demo of OCaml.og Date: Fri, 15 Mar 2013 08:48:08 +0000 To: =?utf-8?Q?Daniel_B=C3=BCnzli?= Cc: "platform@lists.ocaml.org" , Mirage List , infrastructure , "cl-ocamllabs@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 08:48:07 -0000 Content-Length: 4667 Lines: 93 --Apple-Mail-C0E72B7C-35E7-4224-8361-347AEB640526 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable -- sent via mobile On 15 Mar 2013, at 05:54, Daniel B=C3=BCnzli w= rote: > Hello, >=20 >> On wider monitors, we'd likely add more whitespace either side so as to a= void stretching the content too far. >=20 > W.r.t. page layout stretchability but on the narrower side, a good test to= have in mind is to check if on a laptop you can have an 80-columns text edi= tor and the documentation side by side without needing horizontal scrolling.= Hadn't thought of that so I'll bear it in mind. However, it may still depend= on the resolution of the laptop in question (eg I have a 'retina' MacBook w= hich has more horizontal space than my old laptop).=20 > Regarding the Modules tab I'm not sure I understood you well so it may alr= eady be the idea. But if the package has a single module it would be nice if= it shows directly the documentation of that module to save one useless clic= k.=20 Yes, that's the plan. I did mention it but probably wasn't clear enough. > Also, it's more a comment for opam, but it would be nice if the documentat= ion system could do the same locally with the same experience but showing yo= u only the documentation of your installed packages (with a landing page tha= t lists the packages and their respective modules for direct access). Yes, this is also something we want :) It's not just an opam thing as it in= volves Leo's upcoming work on documentation, as well as using packages like c= ow, opam2web, weberizer etc. Doing this locally, would enable the creation o= f properly cross-referenced docs for a particular user's environment.=20 Best wishes, Amir --Apple-Mail-C0E72B7C-35E7-4224-8361-347AEB640526 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

--
sent via mobile

On 15 Mar 2013, at 05:= 54, Daniel B=C3=BCnzli <da= niel.buenzli@erratique.ch> wrote:

Hello,

<= span>On wider monitors, we'd likely add more whitespace either side so as to= avoid stretching the content too far.
<= br>W.r.t. page layout stretchability but on the narrower side, a good t= est to have in mind is to check if on a laptop you can have an 80-columns te= xt editor and the documentation side by side without needing horizontal scro= lling.

Hadn't thought of th= at so I'll bear it in mind. However, it may still depend on the resolution o= f the laptop in question (eg I have a 'retina' MacBook which has more horizo= ntal space than my old laptop). 


Regarding the Modules tab I'm not sure I= understood you well so it may already be the idea. But if the package has a= single module it would be nice if it shows directly the documentation of th= at module to save one useless click.

=
Yes, that's the plan. I did mention it but probably wasn't clear e= nough.


<= span>Also, it's more a comment for opam, but it would be nice if the documen= tation system could do the same locally with the same experience but showing= you only the documentation of your installed packages (with a landing page t= hat lists the packages and their respective modules for direct access).

Yes, this is also something we w= ant :)  It's not just an opam thing as it involves Leo's upcoming work o= n documentation, as well as using packages like cow, opam2web, weberizer etc= . Doing this locally, would enable the creation of properly cross-refer= enced docs for a particular user's environment. 

Best wishes,
Amir

= --Apple-Mail-C0E72B7C-35E7-4224-8361-347AEB640526-- From raphlalou@gmail.com Fri Mar 15 09:50:34 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGRHW-0003D0-BQ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 15 Mar 2013 09:50:34 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1456366 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.223.179 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (raphlalou[at]gmail.com) * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ie0-f179.google.com ([209.85.223.179]:41142) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UGRHV-0008T1-YU (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 15 Mar 2013 09:50:34 +0000 Received: by mail-ie0-f179.google.com with SMTP id k11so4127900iea.10 for ; Fri, 15 Mar 2013 02:50:33 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.43.135.68 with SMTP id if4mr4121925icc.44.1363341032964; Fri, 15 Mar 2013 02:50:32 -0700 (PDT) Received: by 10.42.18.9 with HTTP; Fri, 15 Mar 2013 02:50:32 -0700 (PDT) Date: Fri, 15 Mar 2013 09:50:32 +0000 Message-ID: Subject: Self-hosted web-apps From: Raphael Proust To: cl-mirage@lists.cam.ac.uk Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 09:50:34 -0000 Content-Length: 450 Lines: 18 Hey, TL;DR: you can have `web-apps' that live as a self-modifying HTML+CSS+Javascript file. You just need to synchronize this file to distribute your application across devices. http://drwho.virtadpt.net/archive/2013/03/14/a-new-way-to-write-web-applica= tions Wouldn't that be the kind of things a Javascript backend for mirage should produce? I.e. one file that can be opened by your browser. Cheers, --=20 ______________ Rapha=C3=ABl Proust From thomas.gazagnaire@gmail.com Fri Mar 15 10:04:25 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGRUv-0003pe-5L (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 15 Mar 2013 10:04:25 +0000 X-Cam-SpamDetails: score 0.2 from SpamAssassin-3.3.2-1456366 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.212.169 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 0.9 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-wi0-f169.google.com ([209.85.212.169]:36843) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UGRUu-0006Dy-8Y (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 15 Mar 2013 10:04:25 +0000 Received: by mail-wi0-f169.google.com with SMTP id l13so371650wie.0 for ; Fri, 15 Mar 2013 03:04:24 -0700 (PDT) X-Received: by 10.194.58.202 with SMTP id t10mr9776239wjq.4.1363341864491; Fri, 15 Mar 2013 03:04:24 -0700 (PDT) Received: from [192.168.1.110] ([213.30.181.210]) by mx.google.com with ESMTPS id q13sm2265101wie.0.2013.03.15.03.04.21 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 15 Mar 2013 03:04:22 -0700 (PDT) Subject: Re: Self-hosted web-apps Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: Date: Fri, 15 Mar 2013 11:04:20 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <650BFA20-5803-4EB3-94D7-B82BDE53A002@gmail.com> References: To: Raphael Proust X-Mailer: Apple Mail (2.1085) Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 10:04:25 -0000 Content-Length: 527 Lines: 17 > TL;DR: you can have `web-apps' that live as a self-modifying > HTML+CSS+Javascript file. You just need to synchronize this file to > distribute your application across devices. >=20 > = http://drwho.virtadpt.net/archive/2013/03/14/a-new-way-to-write-web-applic= ations >=20 > Wouldn't that be the kind of things a Javascript backend for mirage > should produce? I.e. one file that can be opened by your browser. Very interesting read, thx! Ihttp://remotestorage.io/ seems indeed quite = interesting for mirage. -- Thomas= From thomas.gazagnaire@gmail.com Fri Mar 15 13:34:24 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGUm8-00042a-QH (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 15 Mar 2013 13:34:24 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1456366 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.176 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-we0-f176.google.com ([74.125.82.176]:48302) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UGUm7-000694-SP (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 15 Mar 2013 13:34:24 +0000 Received: by mail-we0-f176.google.com with SMTP id s43so3095218wey.21 for ; Fri, 15 Mar 2013 06:34:23 -0700 (PDT) X-Received: by 10.194.237.129 with SMTP id vc1mr10594609wjc.20.1363354463607; Fri, 15 Mar 2013 06:34:23 -0700 (PDT) Received: from [192.168.1.110] ([213.30.181.210]) by mx.google.com with ESMTPS id fx5sm3183154wib.11.2013.03.15.06.34.20 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 15 Mar 2013 06:34:21 -0700 (PDT) Sender: Thomas Gazagnaire Subject: Re: OPAM 1.0.0 released ! Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Thomas Gazagnaire In-Reply-To: <999F571F-57A2-46F0-9DA8-83DC7026363E@ocamlpro.com> Date: Fri, 15 Mar 2013 14:34:16 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <999F571F-57A2-46F0-9DA8-83DC7026363E@ocamlpro.com> To: Thomas Gazagnaire X-Mailer: Apple Mail (2.1085) Cc: platform , OCaml List , Mirage List , opam-devel X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 13:34:24 -0000 Content-Length: 6466 Lines: 163 As I've received few questions related to upgrading OPAM itself, I've = added Anil's response to [1] to the FAQ: https://github.com/OCamlPro/opam/wiki/FAQ [1] https://github.com/OCamlPro/opam/issues/528 Best, Thomas On Mar 14, 2013, at 9:28 PM, Thomas Gazagnaire wrote: > I nm *very* happy to announce the first official release of OPAM! >=20 > Many of you already know and use OPAM so I won't be long. Please read = http://www.ocamlpro.com/blog/2013/01/17/opam-beta.html for a longer = description. >=20 > 1.0.0 fixes many bugs and add few new features to the previously = announced beta-release. >=20 > The most visible new feature, which should be useful for beginners = with OCaml and OPAM, is an auto-configuration tool. This tool easily = enables all the features of OPAM (auto-completion, fix the loading of = scripts for the toplevel, opam-switch-eval alias, etc). This tool runs = interactively on each `opam init` invocation. If you don't like OPAM to = change your configuration files, use `opam init --no-setup`. If you = trust the tool blindly, use `opam init --auto-setup`. You can later = review the setup by doing `opam config setup --list` and call the tool = again using `opam config setup` (and you can of course manually edit = your ~/.profile (or ~/.zshrc for zsh users), ~/.ocamlinit and = ~/.opam/opam-init/*). >=20 > Please report: > - Bug reports and feature requests for the OPAM tool: = http://github.com/OCamlPro/opam/issues > - Packaging issues or requests for a new package: = http://github.com/OCamlPro/opam-repository/issues > - General queries to: http://lists.ocaml.org/listinfo/platform > - More specific queries about the internals of OPAM to: = http://lists.ocaml.org/listinfo/opam-devel >=20 > On behalf on the OPAM team, > Thomas >=20 > =3D=3D=3D Install =3D=3D=3D >=20 > Packages for Debian and OSX (at least homebrew) should follow shortly = and I'm looking for volunteers to create and maintain rpm packages. The = binary installer is up-to-date for Linux and Darwin 64-bit = architectures, the 32-bit version for Linux should arrive shortly. >=20 > If you want to build from sources, the full archive (including = dependencies) is available here: >=20 > http://www.ocamlpro.com/pub/opam-full-latest.tar.gz >=20 > =3D=3D=3D Upgrade =3D=3D=3D >=20 > If you are upgrading from 0.9.* you won't have anything special to do = apart installing the new binary. You can then update your package = metadata by running `opam update`. If you want to use the auto-setup = feature, remove the "eval `opam config env` line you have previously = added in your ~/.profile and run `opam config setup --all`. >=20 > So everything should be fine. But you never know ... so if something = goes horribly wrong in the upgrade process (of if your are upgrading = from an old version of OPAM) you can still trash your ~/.opam, manually = remove what OPAM added in your ~/.profile (~/.zshrc for zsh users) and = ~/.ocamlinit, and start again from scratch.=20 >=20 > =3D=3D=3D Random stats =3D=3D=3D >=20 > Great success on github. Thanks everybody for the great contributions! >=20 > https://github.com/OCamlPro/opam: +2000 commits, 26 contributors > https://github.com/OCamlPro/opam-repository: +1700 commits, 75 = contributors, 370+ packages >=20 > on http://opam.ocamlpro.com/ > +400 unique visitor per week, 15k 'opam update' per week > +1300 unique visitor per month, 55k 'opam update' per month > 3815 unique visitor since the alpha release >=20 > =3D=3D=3D Changelog =3D=3D=3D >=20 > The full change-log since the beta release in January: >=20 > 1.0.0 [Mar 2013] > * Improve the lexer performance (thx to @oandrieu) > * Fix various typos (thx to @chaudhuri) > * Fix build issue (thx to @avsm) >=20 > 0.9.6 [Mar 2013] > * Fix installation of pinned packages on BSD (thx to @smondet) > * Fix configuration for zsh users (thx to @AltGr) > * Fix loading of `~/.profile` when using dash (eg. in Debian/Ubuntu) > * Fix installation of packages with symbolic links (regression = introduced in 0.9.5) >=20 > 0.9.5 [Mar 2013] > * If necessary, apply patches and substitute files before removing a = package > * Fix `opam remove --keep-build-dir` keeps the folder if a = source archive is extracted > * Add build and install rules using ocamlbuild to help distro = packagers > * Support arbitrary level of nested subdirectories in packages = repositories > * Add `opam config exec "CMD ARG1 ... ARGn" --switch=3DSWITCH` to = execute a command in a subshell > * Improve the behaviour of `opam update` wrt. pinned packages > * Change the default external solver criteria (only useful if you have = aspcud installed on your machine) > * Add support for global and user configuration for OPAM (`opam config = setup`) > * Stop yelling when OPAM is not up-to-date > * Update or generate `~/.ocamlinit` when running `opam init` > * Fix tests on *BSD (thx Arnaud Degroote) > * Fix compilation for the source archive >=20 > 0.9.4 [Feb 2013] > * Disable auto-removal of unused dependencies. This can now be enabled = on-demand using `-a` > * Fix compilation and basic usage on Cygwin > * Fix BSD support (use `type` instead of `which` to detect existing = commands) > * Add a way to tag external dependencies in OPAM files > * Better error messages when trying to upgrade pinned packages > * Display `depends` and `depopts` fields in `opam info` > * `opam info pkg.version` shows the metadata for this given package = version > * Add missing `doc` fields in `.install` files > * `opam list` now only shows installable packages >=20 > 0.9.3 [Feb 2013] > * Add system compiler constraints in OPAM files > * Better error messages in case of conflicts > * Cleaner API to install/uninstall packages > * On upgrade, OPAM now perform all the remove action first > * Use a cache for main storing OPAM metadata: this greatly speed-up = OPAM invocations > * after an upgrade, propose to reinstall a pinned package only if = there were some changes > * improvements to the solver heuristics > * better error messages on cyclic dependencies >=20 > 0.9.2 [Jan 2013] > * Install all the API files > * Fix `opam repo remove repo-name` > * speed-up `opam config env` > * support for `opam-foo` scripts (which can be called using `opam = foo`) > * 'opam update pinned-package' works > * Fix 'opam-mk-repo -a' > * Fix 'opam-mk-repo -i' > * clean-up pinned cache dir when a pinned package fails to install >=20 > 0.9.1 [Jan 2013] > * Use ocaml-re 1.2.0 >=20 From vb@luminar.eu.org Sat Mar 16 17:21:39 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGunb-0005tf-L8 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 16 Mar 2013 17:21:39 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.4 from SpamAssassin-3.3.2-1456805 * -2.4 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:43205) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UGunb-0002gh-XC (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 16 Mar 2013 17:21:39 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id BCAC4BF41A for ; Sat, 16 Mar 2013 18:21:38 +0100 (CET) Message-ID: <5144AA22.7030105@luminar.eu.org> Date: Sat, 16 Mar 2013 17:21:38 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130221 Thunderbird/17.0.3 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: mirari updates References: <20130227215220.GA2255@dark.recoil.org> <512E857D.2060305@vb.fdn.fr> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Mar 2013 17:21:39 -0000 Content-Length: 1447 Lines: 38 On 28/02/2013 18:13, Anil Madhavapeddy wrote: > I had a quick chat with Haris about this. I think what we're missing i= n the UNIX backend is a simple control socket that mirrors the operations= that Xenstore provides a VM: start, stop, suspend, hotplug device, and s= o on. Without this, we're forced to hardcode many control operations in = the library directly. > > If we make the runloop of a UNIX Mirage process mimic the Xen version, = then many of these hacks go away. For instance, the bridging configurati= on can be done from a command line utility that sets up tun (or pcap) dev= ices, and then passes the fd to the listening Mirage UNIX process. > > It's very Erlang and actor-like, but quite easy to build into Mirari wi= th a bit of cmdliner magic. Dave already has an Lwt fd-passing library I= believe. > > -anil Hi. This afternoon I decided to work again on Mirari and I=92ve been thinking= =20 about things. First, this message. I did not understand it much at first=20 reading but it makes more and more sense. If I understand well, the best option is to make mirari emulate xenstore=20 when using the unix backend. This way, we can get rid of all the "hacks"=20 for the UNIX backend, the mirage kernel will just use the same=20 interfaces "as it was running on Xen" and mirari will emulate the=20 environment (basically a xenstore daemon if I understand well). I=92m=20 going to work towards that. Cheers, Vincent From vb@luminar.eu.org Sat Mar 16 18:51:33 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGwCb-0006l4-78 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 16 Mar 2013 18:51:33 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.4 from SpamAssassin-3.3.2-1456805 * -2.4 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:43190) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UGwCa-0000o4-SV (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 16 Mar 2013 18:51:33 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id 73E4DBF41A for ; Sat, 16 Mar 2013 19:51:32 +0100 (CET) Message-ID: <5144BF34.7030609@luminar.eu.org> Date: Sat, 16 Mar 2013 18:51:32 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130221 Thunderbird/17.0.3 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: mirari updates References: <20130227215220.GA2255@dark.recoil.org> <512E857D.2060305@vb.fdn.fr> <5144AA22.7030105@luminar.eu.org> In-Reply-To: <5144AA22.7030105@luminar.eu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Mar 2013 18:51:33 -0000 Content-Length: 1246 Lines: 41 On 16/03/2013 17:21, Vincent Bernardoff wrote: >> > > Hi. > > This afternoon I decided to work again on Mirari and I=92ve been thinki= ng > about things. First, this message. I did not understand it much at firs= t > reading but it makes more and more sense. > > If I understand well, the best option is to make mirari emulate xenstor= e > when using the unix backend. This way, we can get rid of all the "hacks= " > for the UNIX backend, the mirage kernel will just use the same > interfaces "as it was running on Xen" and mirari will emulate the > environment (basically a xenstore daemon if I understand well). I=92m > going to work towards that. > > Cheers, > > Vincent > > Ok, I think it starts to become more clear. What you were suggesting is=20 that a Mirage-UNIX instance would have a control socket and its=20 configuration would be tunable by connecting to this socket and sending=20 messages/fds. It sounds an interesting option indeed. Could you indicate me where to look to see how it is implemented on the=20 xen backend (the interaction with xenstore). I=92d like to understand mor= e=20 basically about the interaction between xen and mirage, and how the=20 configuration is done when using the Xen backend. Cheers, Vincent From anil@recoil.org Sat Mar 16 19:22:05 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGwg9-0007G3-MQ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 16 Mar 2013 19:22:05 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamScore: ss X-Cam-SpamDetails: score 2.9 from SpamAssassin-3.3.2-1456805 * 2.9 FSL_HELO_BARE_IP_2 FSL_HELO_BARE_IP_2 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:44802 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UGwg8-0002MA-rd (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 16 Mar 2013 19:22:05 +0000 Received: (qmail 13316 invoked by uid 634); 16 Mar 2013 19:22:04 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED,TVD_RCVD_IP X-Spam-Check-By: dark.recoil.org Received: from 64.125.195.54.t00963-01.above.net (HELO [10.99.98.1]) (64.125.195.54) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sat, 16 Mar 2013 19:22:03 +0000 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: mirari updates From: Anil Madhavapeddy In-Reply-To: <5144AA22.7030105@luminar.eu.org> Date: Sat, 16 Mar 2013 14:22:04 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <340916C2-0176-4AC7-B72E-419ACE03A9B7@recoil.org> References: <20130227215220.GA2255@dark.recoil.org> <512E857D.2060305@vb.fdn.fr> <5144AA22.7030105@luminar.eu.org> To: Vincent Bernardoff X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Mar 2013 19:22:05 -0000 Content-Length: 2433 Lines: 55 On 16 Mar 2013, at 12:21, Vincent Bernardoff wrote: > On 28/02/2013 18:13, Anil Madhavapeddy wrote: >> I had a quick chat with Haris about this. I think what we're missing = in the UNIX backend is a simple control socket that mirrors the = operations that Xenstore provides a VM: start, stop, suspend, hotplug = device, and so on. Without this, we're forced to hardcode many control = operations in the library directly. >>=20 >> If we make the runloop of a UNIX Mirage process mimic the Xen = version, then many of these hacks go away. For instance, the bridging = configuration can be done from a command line utility that sets up tun = (or pcap) devices, and then passes the fd to the listening Mirage UNIX = process. >>=20 >> It's very Erlang and actor-like, but quite easy to build into Mirari = with a bit of cmdliner magic. Dave already has an Lwt fd-passing = library I believe. >>=20 >> -anil >=20 > Hi. >=20 > This afternoon I decided to work again on Mirari and I=92ve been = thinking about things. First, this message. I did not understand it much = at first reading but it makes more and more sense. >=20 > If I understand well, the best option is to make mirari emulate = xenstore when using the unix backend. This way, we can get rid of all = the "hacks" for the UNIX backend, the mirage kernel will just use the = same interfaces "as it was running on Xen" and mirari will emulate the = environment (basically a xenstore daemon if I understand well). I=92m = going to work towards that. Dave had some patches to remove the Xen dependency from xenstored and = let it run on a normal UNIX box. However, I wonder if a better approach might be to explicitly encode the = various operations, and map those to Xen. The reasoning here is that the = Xenstore interfaces are a bit ad-hoc, and so a fresh approach would = avoid propagating various bugs to future backends. Remember that there is more to this interface than just Xen+UNIX: we = also (eventually) want to integrate a Javascript, kFreeBSD and Raspberry = Pi backend: each of these will have their own unique ways of expressing = device hotplug. Why not just start off the UNIX one with a very simple 'process = lifecycle' -- start, stop, suspend, resume that is similar to Xen, and = then a device registration mechanism for network+block. This should map = to Xen fairly easily, but not introduce a dependency on Xenstore. -anil From anil@recoil.org Sat Mar 16 21:22:01 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGyYD-0002Zf-4o (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 16 Mar 2013 21:22:01 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1456805 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:13503 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UGyYA-0003WF-qj (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 16 Mar 2013 21:22:01 +0000 Received: (qmail 6133 invoked by uid 634); 16 Mar 2013 21:21:57 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from airband-63-133-128-186.airband.net (HELO [10.99.98.1]) (63.133.128.186) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sat, 16 Mar 2013 21:21:56 +0000 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: invertible protocol descriptions in Coq Date: Sat, 16 Mar 2013 16:21:21 -0500 Message-Id: To: "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Vincent Hanquez , Prashanth Mundkur X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Mar 2013 21:22:01 -0000 Content-Length: 410 Lines: 14 Found this very cool paper on specifying invertible protocol = descriptions in Coq: http://staff.aist.go.jp/reynald.affeldt/documents/plpv2012-preprint.pdf ..and they cover TLS. It would be a nice project to extract this to = OCaml/Haskell to guarantee parsing equivalence from the formal spec (this is basically = a much better implementation of MPL or FP, embedded in the Coq proof = assistant). -anil= From anil@recoil.org Sat Mar 16 21:25:46 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGybq-0002dU-8h (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 16 Mar 2013 21:25:46 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamScore: ss X-Cam-SpamDetails: score 2.9 from SpamAssassin-3.3.2-1456805 * 2.9 FSL_HELO_BARE_IP_2 FSL_HELO_BARE_IP_2 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:13791 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UGybp-0004Uj-rX (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 16 Mar 2013 21:25:46 +0000 Received: (qmail 24413 invoked by uid 634); 16 Mar 2013 21:25:45 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED,TVD_RCVD_IP X-Spam-Check-By: dark.recoil.org Received: from 64.125.195.54.t00963-01.above.net (HELO [10.99.98.1]) (64.125.195.54) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sat, 16 Mar 2013 21:25:44 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: Self-hosted web-apps From: Anil Madhavapeddy In-Reply-To: <650BFA20-5803-4EB3-94D7-B82BDE53A002@gmail.com> Date: Sat, 16 Mar 2013 16:25:43 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <8B02A3CD-B570-4D0F-B357-80D179B961A8@recoil.org> References: <650BFA20-5803-4EB3-94D7-B82BDE53A002@gmail.com> To: Thomas Gazagnaire X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk, Raphael Proust X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Mar 2013 21:25:46 -0000 Content-Length: 862 Lines: 26 On 15 Mar 2013, at 05:04, Thomas Gazagnaire = wrote: >> TL;DR: you can have `web-apps' that live as a self-modifying >> HTML+CSS+Javascript file. You just need to synchronize this file to >> distribute your application across devices. >>=20 >> = http://drwho.virtadpt.net/archive/2013/03/14/a-new-way-to-write-web-applic= ations >>=20 >> Wouldn't that be the kind of things a Javascript backend for mirage >> should produce? I.e. one file that can be opened by your browser. >=20 > Very interesting read, thx! Ihttp://remotestorage.io/ seems indeed = quite interesting for mirage. I like it; it would be a nice target for the Irminsule backend too. So if we take the OCaml compiler-libs and link those in too, then we can just have full self-modifying Javascript that replicates via the storage synchronisation, right? :-) -a From anil@recoil.org Sat Mar 16 21:38:01 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UGynh-0002uF-0k (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 16 Mar 2013 21:38:01 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1456805 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:44565 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UGynf-0006vr-qV (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 16 Mar 2013 21:38:00 +0000 Received: (qmail 27250 invoked by uid 634); 16 Mar 2013 21:37:59 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from airband-63-133-128-186.airband.net (HELO [10.99.98.1]) (63.133.128.186) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sat, 16 Mar 2013 21:37:58 +0000 Content-Type: text/plain; charset=koi8-r Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: ocaml-dns From: Anil Madhavapeddy In-Reply-To: <20130314185125.GA91758@ermine.home> Date: Sat, 16 Mar 2013 16:37:43 -0500 Content-Transfer-Encoding: 7bit Message-Id: References: <0EFB8E47-92E2-4D26-A784-BDD85B7D94E9@recoil.org> <20121114124350.GA77471@ermine.home> <20121204125604.GA10989@dark.recoil.org> <20130314185125.GA91758@ermine.home> To: Anastasia Gornostaeva X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Mar 2013 21:38:01 -0000 Content-Length: 880 Lines: 25 On 14 Mar 2013, at 13:51, Anastasia Gornostaeva wrote: > > I've decided to use your ocaml-dns library to get SRV records for my > xmpp library. Could you give me an example of code, please? CCing cl-mirage as I'm on the road, so if someone else could elaborate if necessary that'd be useful. https://github.com/mirage/ocaml-dns/tree/master/lib_test has some test examples, and there is a 'dig' clone here: https://github.com/mirage/ocaml-dns/blob/master/lwt/mldig.ml (which uses Lwt) It should be straightforward to write a SRV resolver from this. The main thing on my TODO list for the DNS library before 1.0 is to remove the dependency on OpenSSL for the DNSSEC code that Haris wrote, and implement that in pure OCaml. Let us know if you run into any problems (the library's still beta, so we can break interfaces if needed before a 1.0 release). -a From raphlalou@gmail.com Sun Mar 17 08:56:55 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UH9Oh-0005jy-4W (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 17 Mar 2013 08:56:55 +0000 X-Cam-SpamDetails: score 0.2 from SpamAssassin-3.3.2-1457225 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.223.174 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (raphlalou[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 0.9 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ie0-f174.google.com ([209.85.223.174]:50136) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UH9Og-0003Us-83 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 17 Mar 2013 08:56:55 +0000 Received: by mail-ie0-f174.google.com with SMTP id k10so5839762iea.5 for ; Sun, 17 Mar 2013 01:56:53 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.50.154.133 with SMTP id vo5mr4233946igb.77.1363510613737; Sun, 17 Mar 2013 01:56:53 -0700 (PDT) Received: by 10.42.18.9 with HTTP; Sun, 17 Mar 2013 01:56:53 -0700 (PDT) In-Reply-To: <8B02A3CD-B570-4D0F-B357-80D179B961A8@recoil.org> References: <650BFA20-5803-4EB3-94D7-B82BDE53A002@gmail.com> <8B02A3CD-B570-4D0F-B357-80D179B961A8@recoil.org> Date: Sun, 17 Mar 2013 08:56:53 +0000 Message-ID: Subject: Re: Self-hosted web-apps From: Raphael Proust To: Anil Madhavapeddy Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: cl-mirage@lists.cam.ac.uk, Thomas Gazagnaire X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Mar 2013 08:56:55 -0000 Content-Length: 1507 Lines: 46 Also relevant to this discussion: someone wants to rewrite git in Javascript: http://www.kickstarter.com/projects/creationix/js-git This would make the synchronisation part git-like. Another (better?) way to write that would be to pipe a completed version of https://github.com/samoht/cagit through js_of_ocaml. So the plan would be: - compile backend code to JS - generate UI code to HTML+CSS (possibly "just a repl" or "terminal emulato= r") - link in synchronisation code (js-git or cagit|js_of_ocaml) On Sat, Mar 16, 2013 at 9:25 PM, Anil Madhavapeddy wrote: > On 15 Mar 2013, at 05:04, Thomas Gazagnaire = wrote: > >>> TL;DR: you can have `web-apps' that live as a self-modifying >>> HTML+CSS+Javascript file. You just need to synchronize this file to >>> distribute your application across devices. >>> >>> http://drwho.virtadpt.net/archive/2013/03/14/a-new-way-to-write-web-app= lications >>> >>> Wouldn't that be the kind of things a Javascript backend for mirage >>> should produce? I.e. one file that can be opened by your browser. >> >> Very interesting read, thx! Ihttp://remotestorage.io/ seems indeed quite= interesting for mirage. > > I like it; it would be a nice target for the Irminsule backend too. > > So if we take the OCaml compiler-libs and link those in too, then we > can just have full self-modifying Javascript that replicates via the > storage synchronisation, right? :-) > > -a > --=20 ______________ Rapha=C3=ABl Proust From anil@recoil.org Sun Mar 17 15:40:01 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHFgn-0005Rg-7Q (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 17 Mar 2013 15:40:01 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamScore: ss X-Cam-SpamDetails: score 2.4 from SpamAssassin-3.3.2-1457225 * 2.4 FSL_HELO_BARE_IP_2 FSL_HELO_BARE_IP_2 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:30304 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UHFgm-0004Ab-RQ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 17 Mar 2013 15:40:01 +0000 Received: (qmail 1976 invoked by uid 634); 17 Mar 2013 15:40:00 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED,TVD_RCVD_IP X-Spam-Check-By: dark.recoil.org Received: from 64.125.195.54.t00963-01.above.net (HELO [10.99.98.1]) (64.125.195.54) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sun, 17 Mar 2013 15:39:59 +0000 Content-Type: text/plain; charset=koi8-r Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: ocaml-dns From: Anil Madhavapeddy In-Reply-To: <20130317145249.GA19172@ermine.home> Date: Sun, 17 Mar 2013 10:39:51 -0500 Content-Transfer-Encoding: 7bit Message-Id: References: <0EFB8E47-92E2-4D26-A784-BDD85B7D94E9@recoil.org> <20121114124350.GA77471@ermine.home> <20121204125604.GA10989@dark.recoil.org> <20130314185125.GA91758@ermine.home> <20130317145249.GA19172@ermine.home> To: Anastasia Gornostaeva X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Mar 2013 15:40:01 -0000 Content-Length: 2714 Lines: 65 On 17 Mar 2013, at 09:52, Anastasia Gornostaeva wrote: > On Sat, Mar 16, 2013 at 04:37:43PM -0500, Anil Madhavapeddy wrote: > >> implement that in pure OCaml. >> >> Let us know if you run into any problems (the library's still beta, so >> we can break interfaces if needed before a 1.0 release). > > > ocamlfind: Package `ocplib-endian' not found - required by `cstruct' > ocamlfind: Package `optcomp' not found - required by `ocplib-endian' > Error: Unbound module Re_str ocplib-endian is a library which implements integer marshalling/unmarshal support into Bigarray. OCaml-4.01.0 and higher support compiler primitives which eliminate a lot of intermediate boxing, and hence perform better. optcomp provides #if preprocessor support to detect the right compiler version, and fall back to the slower manual functions on older compilers. cstruct provides the syntax extension that's analogous to Bitstring, except that it works directly on Bigarray and avoids copies. Performance is higher as a result (note that Cstruct doesn't currently do much of the static analysis that Bitstring does, so it can get even better with some more effort). > $ ... install (outside of opam) > W: Cannot find source header for module Zone_lexer in library dns > W: Cannot find source header for module Zone_parser in library dns > ocamlfind: Package dns is already installed > - (file /home/ermine/.opam/system/lib/dns/META already exists) Presumably this is because you've already installed a DNS package? OCamlfind doesn't let you keep multiple installations with the same name, so remove the old one first. > So many strange dependences! They're only strange until you know what they're for :-) OPAM takes care of all this for you, so it's not really an issue any more. We've been working on improving *BSD compatibility there, btw, so it should be better than it used to be. However, it looks from your example above that you already used OPAM to install it, so it's a bit of a mystery to me why you didn't have the dependencies installed: depends: ["lwt" {>="2.4.1"} "cstruct" {>="0.6.0"} "ocamlfind" "cryptokit" "re" "uri" "cmdliner" ] > Looking at github.com/ygrek/yadns - it requeres only bitstring to parse > dns stuff. Sure... if you don't mind Bitstring copying into OCaml strings, then it's more straightforward (we used to use it in ocaml-dns also). The Mirage DNS (and the rest of the stack) tries to minimise data copies, as it also has to append the UDP/Ethernet headers after it assembles a packet. Using OCaml strings locks you into a world where you have more heap and GC pressure, and then you're stuck at a certain level of performance forever... -anil From ermine@sulci.ru Sun Mar 17 16:06:24 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHG6K-0005xY-C6 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 17 Mar 2013 16:06:24 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1457225 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from av6800.comex.ru ([217.10.34.155]:48088 helo=home.mk.pp.ru) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UHG6J-00038T-rT (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 17 Mar 2013 16:06:24 +0000 Received: from ermine.home (ermine.home [192.168.2.2]) by home.mk.pp.ru (Postfix) with ESMTP id DD63020A25; Sun, 17 Mar 2013 20:06:21 +0400 (MSK) Received: by ermine.home (Postfix, from userid 1002) id 2CB7E2E158; Sun, 17 Mar 2013 20:05:33 +0400 (MSK) Date: Sun, 17 Mar 2013 20:05:33 +0400 From: Anastasia Gornostaeva To: Anil Madhavapeddy Subject: Re: ocaml-dns Message-ID: <20130317160533.GB19172@ermine.home> References: <0EFB8E47-92E2-4D26-A784-BDD85B7D94E9@recoil.org> <20121114124350.GA77471@ermine.home> <20121204125604.GA10989@dark.recoil.org> <20130314185125.GA91758@ermine.home> <20130317145249.GA19172@ermine.home> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Mar 2013 16:06:24 -0000 Content-Length: 3289 Lines: 76 On Sun, Mar 17, 2013 at 10:39:51AM -0500, Anil Madhavapeddy wrote: It is difficult to use opam still. I would to switch to opam when it compile and install pgocaml and eliom at least. So I've compiled ocaml-dns not using opam and as you've seen it failed because: 1) ocaml setupm.ml -configure does not check all dependences 2) it checks if opam is installed on the machine but does not check if was compiled via opam. It is really mystery :-) > On 17 Mar 2013, at 09:52, Anastasia Gornostaeva wrote: > > > On Sat, Mar 16, 2013 at 04:37:43PM -0500, Anil Madhavapeddy wrote: > > > >> implement that in pure OCaml. > >> > >> Let us know if you run into any problems (the library's still beta, so > >> we can break interfaces if needed before a 1.0 release). > > > > > > ocamlfind: Package `ocplib-endian' not found - required by `cstruct' > > ocamlfind: Package `optcomp' not found - required by `ocplib-endian' > > Error: Unbound module Re_str > > ocplib-endian is a library which implements integer marshalling/unmarshal > support into Bigarray. OCaml-4.01.0 and higher support compiler primitives > which eliminate a lot of intermediate boxing, and hence perform better. > > optcomp provides #if preprocessor support to detect the right compiler > version, and fall back to the slower manual functions on older compilers. > > cstruct provides the syntax extension that's analogous to Bitstring, > except that it works directly on Bigarray and avoids copies. Performance > is higher as a result (note that Cstruct doesn't currently do much of > the static analysis that Bitstring does, so it can get even better with > some more effort). > > > $ ... install (outside of opam) > > W: Cannot find source header for module Zone_lexer in library dns > > W: Cannot find source header for module Zone_parser in library dns > > ocamlfind: Package dns is already installed > > - (file /home/ermine/.opam/system/lib/dns/META already exists) > > Presumably this is because you've already installed a DNS package? > OCamlfind doesn't let you keep multiple installations with the same name, > so remove the old one first. > > > So many strange dependences! > > They're only strange until you know what they're for :-) OPAM takes care > of all this for you, so it's not really an issue any more. We've been > working on improving *BSD compatibility there, btw, so it should be better > than it used to be. > > However, it looks from your example above that you already used OPAM to > install it, so it's a bit of a mystery to me why you didn't have the > dependencies installed: > > depends: ["lwt" {>="2.4.1"} "cstruct" {>="0.6.0"} "ocamlfind" > "cryptokit" "re" "uri" "cmdliner" ] > > > > Looking at github.com/ygrek/yadns - it requeres only bitstring to parse > > dns stuff. > > Sure... if you don't mind Bitstring copying into OCaml strings, then it's > more straightforward (we used to use it in ocaml-dns also). The Mirage DNS > (and the rest of the stack) tries to minimise data copies, as it also has to > append the UDP/Ethernet headers after it assembles a packet. > > Using OCaml strings locks you into a world where you have more heap and GC > pressure, and then you're stuck at a certain level of performance forever... > > -anil From scott.dj@gmail.com Sun Mar 17 16:16:44 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHGGK-0006Dx-6u (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 17 Mar 2013 16:16:44 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1457225 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.160.52 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-pb0-f52.google.com ([209.85.160.52]:48776) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UHGGJ-0005Zk-Wk (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 17 Mar 2013 16:16:44 +0000 Received: by mail-pb0-f52.google.com with SMTP id ma3so5792278pbc.39 for ; Sun, 17 Mar 2013 09:16:42 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.66.6.66 with SMTP id y2mr6074025pay.217.1363537002095; Sun, 17 Mar 2013 09:16:42 -0700 (PDT) Received: by 10.70.40.205 with HTTP; Sun, 17 Mar 2013 09:16:42 -0700 (PDT) In-Reply-To: <20130317160533.GB19172@ermine.home> References: <0EFB8E47-92E2-4D26-A784-BDD85B7D94E9@recoil.org> <20121114124350.GA77471@ermine.home> <20121204125604.GA10989@dark.recoil.org> <20130314185125.GA91758@ermine.home> <20130317145249.GA19172@ermine.home> <20130317160533.GB19172@ermine.home> Date: Sun, 17 Mar 2013 16:16:42 +0000 Message-ID: Subject: Re: ocaml-dns From: David Scott To: Anastasia Gornostaeva Content-Type: multipart/alternative; boundary=bcaec520f54fa5eae004d8213278 Cc: "cl-mirage@lists.cam.ac.uk List" , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Mar 2013 16:16:44 -0000 Content-Length: 9111 Lines: 238 --bcaec520f54fa5eae004d8213278 Content-Type: text/plain; charset=ISO-8859-1 Hi, On Sun, Mar 17, 2013 at 4:05 PM, Anastasia Gornostaeva wrote: > On Sun, Mar 17, 2013 at 10:39:51AM -0500, Anil Madhavapeddy wrote: > > It is difficult to use opam still. > I would to switch to opam when it compile and install pgocaml and eliom at > least. > FYI opam is designed to work in parallel with other package managers. Personally I use Debian binary packages for some tasks and use opam for others, all on the same box. Cheers, Dave So I've compiled ocaml-dns not using opam and as you've seen it failed > because: > 1) ocaml setupm.ml -configure does not check all dependences > 2) it checks if opam is installed on the machine but does not check if > was compiled via opam. It is really mystery :-) > > > > On 17 Mar 2013, at 09:52, Anastasia Gornostaeva wrote: > > > > > On Sat, Mar 16, 2013 at 04:37:43PM -0500, Anil Madhavapeddy wrote: > > > > > >> implement that in pure OCaml. > > >> > > >> Let us know if you run into any problems (the library's still beta, so > > >> we can break interfaces if needed before a 1.0 release). > > > > > > > > > ocamlfind: Package `ocplib-endian' not found - required by `cstruct' > > > ocamlfind: Package `optcomp' not found - required by `ocplib-endian' > > > Error: Unbound module Re_str > > > > ocplib-endian is a library which implements integer marshalling/unmarshal > > support into Bigarray. OCaml-4.01.0 and higher support compiler > primitives > > which eliminate a lot of intermediate boxing, and hence perform better. > > > > optcomp provides #if preprocessor support to detect the right compiler > > version, and fall back to the slower manual functions on older compilers. > > > > cstruct provides the syntax extension that's analogous to Bitstring, > > except that it works directly on Bigarray and avoids copies. Performance > > is higher as a result (note that Cstruct doesn't currently do much of > > the static analysis that Bitstring does, so it can get even better with > > some more effort). > > > > > $ ... install (outside of opam) > > > W: Cannot find source header for module Zone_lexer in library dns > > > W: Cannot find source header for module Zone_parser in library dns > > > ocamlfind: Package dns is already installed > > > - (file /home/ermine/.opam/system/lib/dns/META already exists) > > > > Presumably this is because you've already installed a DNS package? > > OCamlfind doesn't let you keep multiple installations with the same name, > > so remove the old one first. > > > > > So many strange dependences! > > > > They're only strange until you know what they're for :-) OPAM takes care > > of all this for you, so it's not really an issue any more. We've been > > working on improving *BSD compatibility there, btw, so it should be > better > > than it used to be. > > > > However, it looks from your example above that you already used OPAM to > > install it, so it's a bit of a mystery to me why you didn't have the > > dependencies installed: > > > > depends: ["lwt" {>="2.4.1"} "cstruct" {>="0.6.0"} "ocamlfind" > > "cryptokit" "re" "uri" "cmdliner" ] > > > > > > > Looking at github.com/ygrek/yadns - it requeres only bitstring to > parse > > > dns stuff. > > > > Sure... if you don't mind Bitstring copying into OCaml strings, then it's > > more straightforward (we used to use it in ocaml-dns also). The Mirage > DNS > > (and the rest of the stack) tries to minimise data copies, as it also > has to > > append the UDP/Ethernet headers after it assembles a packet. > > > > Using OCaml strings locks you into a world where you have more heap and > GC > > pressure, and then you're stuck at a certain level of performance > forever... > > > > -anil > > -- Dave Scott --bcaec520f54fa5eae004d8213278 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi,

On Sun, Mar 17, 2013 at 4:05 PM, Anastasia Gornostaeva <ermine@sulci.ru> wrote:
On Sun, Mar 17, 2013 at 10:39:51AM -0500, An= il Madhavapeddy wrote:

It is difficult to use opam still.
I would to switch to opam when it compile and install pgocaml and eliom at = least.

FYI opam is designed to work in = parallel with other package managers. Personally I use Debian binary packag= es for some tasks and use opam for others, all on the same box.

Cheers,
Dave

So I've compiled ocaml-dns not using opam and as you've seen it fai= led
because:
1) ocaml setupm.ml -conf= igure does not check all dependences
2) it checks if opam is installed on the machine but does not check if
was compiled via opam. It is really mystery :-)


> On 17 Mar 2013, at 09:52, Anastasia Gornostaeva <ermine@sulci.ru> wrote:
>
> > On Sat, Mar 16, 2013 at 04:37:43PM -0500, Anil Madhavapeddy wrote= :
> >
> >> implement that in pure OCaml.
> >>
> >> Let us know if you run into any problems (the library's s= till beta, so
> >> we can break interfaces if needed before a 1.0 release).
> >
> >
> > ocamlfind: Package `ocplib-endian' not found - required by `c= struct'
> > ocamlfind: Package `optcomp' not found - required by `ocplib-= endian'
> > Error: Unbound module Re_str
>
> ocplib-endian is a library which implements integer marshalling/unmars= hal
> support into Bigarray. =A0OCaml-4.01.0 and higher support compiler pri= mitives
> which eliminate a lot of intermediate boxing, and hence perform better= .
>
> optcomp provides #if preprocessor support to detect the right compiler=
> version, and fall back to the slower manual functions on older compile= rs.
>
> cstruct provides the syntax extension that's analogous to Bitstrin= g,
> except that it works directly on Bigarray and avoids copies. =A0Perfor= mance
> is higher as a result (note that Cstruct doesn't currently do much= of
> the static analysis that Bitstring does, so it can get even better wit= h
> some more effort).
>
> > $ ... install (outside of opam)
> > W: Cannot find source header for module Zone_lexer in library dns=
> > W: Cannot find source header for module Zone_parser in library dn= s
> > ocamlfind: Package dns is already installed
> > - (file /home/ermine/.opam/system/lib/dns/META already exists) >
> Presumably this is because you've already installed a DNS package?=
> OCamlfind doesn't let you keep multiple installations with the sam= e name,
> so remove the old one first.
>
> > So many strange dependences!
>
> They're only strange until you know what they're for :-) =A0OP= AM takes care
> of all this for you, so it's not really an issue any more. =A0We&#= 39;ve been
> working on improving *BSD compatibility there, btw, so it should be be= tter
> than it used to be.
>
> However, it looks from your example above that you already used OPAM t= o
> install it, so it's a bit of a mystery to me why you didn't ha= ve the
> dependencies installed:
>
> depends: ["lwt" {>=3D"2.4.1"} "cstruct&quo= t; {>=3D"0.6.0"} "ocamlfind"
> =A0 =A0 =A0 =A0 =A0 =A0"cryptokit" "re" "uri&= quot; "cmdliner" ]
>
>
> > Looking at github.com/ygrek/yadns - it requeres only bitstring to parse
> > dns stuff.
>
> Sure... if you don't mind Bitstring copying into OCaml strings, th= en it's
> more straightforward (we used to use it in ocaml-dns also). The Mirage= DNS
> (and the rest of the stack) tries to minimise data copies, as it also = has to
> append the UDP/Ethernet headers after it assembles a packet.
>
> Using OCaml strings locks you into a world where you have more heap an= d GC
> pressure, and then you're stuck at a certain level of performance = forever...
>
> -anil




--
= Dave Scott
--bcaec520f54fa5eae004d8213278-- From anil@recoil.org Sun Mar 17 16:41:12 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHGe0-0006nr-8l (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 17 Mar 2013 16:41:12 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1457225 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:6482 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UHGdy-0003o3-So (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 17 Mar 2013 16:41:12 +0000 Received: (qmail 15148 invoked by uid 634); 17 Mar 2013 16:41:10 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from airband-63-133-128-186.airband.net (HELO [10.99.98.1]) (63.133.128.186) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sun, 17 Mar 2013 16:41:09 +0000 Content-Type: text/plain; charset=koi8-r Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: ocaml-dns From: Anil Madhavapeddy In-Reply-To: <20130317160533.GB19172@ermine.home> Date: Sun, 17 Mar 2013 11:40:47 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <0EFB8E47-92E2-4D26-A784-BDD85B7D94E9@recoil.org> <20121114124350.GA77471@ermine.home> <20121204125604.GA10989@dark.recoil.org> <20130314185125.GA91758@ermine.home> <20130317145249.GA19172@ermine.home> <20130317160533.GB19172@ermine.home> To: Anastasia Gornostaeva X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Mar 2013 16:41:12 -0000 Content-Length: 1312 Lines: 42 On 17 Mar 2013, at 11:05, Anastasia Gornostaeva wrote: > On Sun, Mar 17, 2013 at 10:39:51AM -0500, Anil Madhavapeddy wrote: >=20 > It is difficult to use opam still. > I would to switch to opam when it compile and install pgocaml and = eliom at least. Do feel free to open up build failure issues on: http://github.com/OCamlPro/opam-repository/issues Both of those compile fine on MacOS X and Linux, but quite probably fail on BSD with some random Makefile problem. Should be easy to fix... > So I've compiled ocaml-dns not using opam and as you've seen it failed > because: > 1) ocaml setupm.ml -configure does not check all dependences Right... it checks for cstruct, but the other libs are transitive = dependencies needed for cstruct to compile. I don't think those are meant to into = the DNS _oasis file. > 2) it checks if opam is installed on the machine but does not check if > was compiled via opam. It is really mystery :-) I don't understand this. Do you mean that ocaml-dns checks for OPAM?=20 It doesn't do anything with OPAM at all; it just calls the OCaml = toolchain on the current PATH. You might want to do delete setup.data if you're seeing odd caching = issues in OASIS, and rerun the build. I find that it doesn't pick up compiler changes sometimes. -anil From ermine@sulci.ru Sun Mar 17 18:16:57 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHI8f-0001Lt-KY (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 17 Mar 2013 18:16:57 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1457225 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from av6800.comex.ru ([217.10.34.155]:10258 helo=home.mk.pp.ru) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UHI8f-0004FS-WI (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 17 Mar 2013 18:16:57 +0000 Received: from ermine.home (ermine.home [192.168.2.2]) by home.mk.pp.ru (Postfix) with ESMTP id EC08620A25 for ; Sun, 17 Mar 2013 22:16:55 +0400 (MSK) Received: by ermine.home (Postfix, from userid 1002) id 23F9C2E158; Sun, 17 Mar 2013 22:16:07 +0400 (MSK) Date: Sun, 17 Mar 2013 22:16:07 +0400 From: Anastasia Gornostaeva To: cl-mirage Subject: Re: ocaml-dns Message-ID: <20130317181607.GA31389@ermine.home> References: <0EFB8E47-92E2-4D26-A784-BDD85B7D94E9@recoil.org> <20121114124350.GA77471@ermine.home> <20121204125604.GA10989@dark.recoil.org> <20130314185125.GA91758@ermine.home> <20130317145249.GA19172@ermine.home> <20130317160533.GB19172@ermine.home> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Mar 2013 18:16:57 -0000 Content-Length: 1072 Lines: 36 > > So I've compiled ocaml-dns not using opam and as you've seen it faile= d > > because: > > 1) ocaml setupm.ml -configure does not check all dependences >=20 > Right... it checks for cstruct, but the other libs are transitive depen= dencies > needed for cstruct to compile. I don't think those are meant to into t= he > DNS _oasis file. Oh, it is my fault then :-) I manually copied cstruct from opam and didnt= check for its dependences. Sorry. >=20 > > 2) it checks if opam is installed on the machine but does not check i= f > > was compiled via opam. It is really mystery :-) >=20 > I don't understand this. Do you mean that ocaml-dns checks for OPAM?=20 > It doesn't do anything with OPAM at all; it just calls the OCaml toolch= ain > on the current PATH. > You might want to do delete setup.data if you're seeing odd caching iss= ues > in OASIS, and rerun the build. I find that it doesn't pick up compiler > changes sometimes. >=20 Oh, it is my fault again :-( After removing setup.data it install fine. Sorry for useless worring. =D5=CB=D8=DB=D4=D5 >=20 From ermine@sulci.ru Mon Mar 18 08:46:44 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHViO-0004dj-46 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 08:46:44 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1457395 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from av6800.comex.ru ([217.10.34.155]:32752 helo=home.mk.pp.ru) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UHViN-0006ri-RN (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 08:46:44 +0000 Received: from ermine.home (ermine.home [192.168.2.2]) by home.mk.pp.ru (Postfix) with ESMTP id 7D89D20A51; Mon, 18 Mar 2013 12:46:42 +0400 (MSK) Received: by ermine.home (Postfix, from userid 1002) id 0EB9D2E0E1; Mon, 18 Mar 2013 12:45:53 +0400 (MSK) Date: Mon, 18 Mar 2013 12:45:52 +0400 From: Anastasia Gornostaeva To: Anil Madhavapeddy Subject: Re: ocaml-dns Message-ID: <20130318084552.GA57194@ermine.home> References: <0EFB8E47-92E2-4D26-A784-BDD85B7D94E9@recoil.org> <20121114124350.GA77471@ermine.home> <20121204125604.GA10989@dark.recoil.org> <20130314185125.GA91758@ermine.home> <20130317145249.GA19172@ermine.home> <20130317160533.GB19172@ermine.home> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: cl-mirage X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Mar 2013 08:46:44 -0000 Content-Length: 271 Lines: 10 Hi, It seems Dns does not provide any way to connect to DNS server and resolve any queries. Am I wrong? I've found Dns_resolver for using with lwt, but it absents by default (even in opam installation). Should I re-implement Dns_resolver for plain networking? ermine From crowcroft@gmail.com Mon Mar 18 11:59:22 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHYio-0006kK-6S (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 11:59:22 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1457395 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.83.48 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (crowcroft[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ee0-f48.google.com ([74.125.83.48]:46331) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UHYin-0006RD-rO (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 11:59:22 +0000 Received: by mail-ee0-f48.google.com with SMTP id t10so2598548eei.35 for ; Mon, 18 Mar 2013 04:59:21 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.14.179.5 with SMTP id g5mr48442131eem.41.1363607961272; Mon, 18 Mar 2013 04:59:21 -0700 (PDT) Sender: crowcroft@gmail.com Received: by 10.223.82.67 with HTTP; Mon, 18 Mar 2013 04:59:21 -0700 (PDT) In-Reply-To: References: Date: Mon, 18 Mar 2013 11:59:21 +0000 X-Google-Sender-Auth: n2SW2ta22iWQA9MnQWnGeTR3zmU Message-ID: Subject: Re: invertible protocol descriptions in Coq From: Jon Crowcroft To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=047d7b603fe6252cd704d831b830 Cc: Vincent Hanquez , "cl-mirage@lists.cam.ac.uk List" , Prashanth Mundkur X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Mar 2013 11:59:22 -0000 Content-Length: 2886 Lines: 64 --047d7b603fe6252cd704d831b830 Content-Type: text/plain; charset=ISO-8859-1 hm - so i think this is a weird paper - the example from eric rescorla's TLS work is easily sorted with an LALR1 parser and tokenisning the input stream not at the PACKET level, or byte, but by turning the protocol into a collection of microprotocols- indeed, this is how most people think about TCP - tcp is a collection of seperate protocols (state management for connection setup/teardown; sequenced data delivery; flow control; congestion control' urgent pointer; etc) living inside one packet format - don't get hung up on the packet format - that is a mere _lexical_ problem... On Sat, Mar 16, 2013 at 9:21 PM, Anil Madhavapeddy wrote: > Found this very cool paper on specifying invertible protocol descriptions > in Coq: > > http://staff.aist.go.jp/reynald.affeldt/documents/plpv2012-preprint.pdf > > ..and they cover TLS. It would be a nice project to extract this to > OCaml/Haskell > to guarantee parsing equivalence from the formal spec (this is basically a > much > better implementation of MPL or FP, embedded in the Coq proof assistant). > > -anil > --047d7b603fe6252cd704d831b830 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable hm - so i think this is a weird paper - the example from eric rescorla'= s TLS work is easily sorted with an LALR1 parser and tokenisning the input = stream not at the PACKET level, or byte, but by turning the protocol into a= collection of microprotocols- indeed, this is how most people think about = TCP - tcp is a collection of seperate protocols (state management for conne= ction setup/teardown; sequenced data delivery; flow control; congestion con= trol' urgent pointer; etc) living inside one packet format - don't = get hung up on the packet format - that is a mere _lexical_ problem...

On Sat, Mar 16, 2013 at 9:21 PM, Anil Madhav= apeddy <anil@recoil.org> wrote:
Found this very cool paper on specifying invertible protocol descriptions i= n Coq:

http://staff.aist.go.jp/reynald.affeldt/document= s/plpv2012-preprint.pdf

..and they cover TLS. =A0It would be a nice project to extract this to OCam= l/Haskell
to guarantee parsing equivalence from the formal spec (this is basically a = much
better implementation of MPL or FP, embedded in the Coq proof assistant).
-anil

--047d7b603fe6252cd704d831b830-- From anil@recoil.org Mon Mar 18 16:01:32 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHcVA-0000WE-Fl (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 16:01:32 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamScore: ss X-Cam-SpamDetails: score 2.4 from SpamAssassin-3.3.2-1457395 * 2.4 FSL_HELO_BARE_IP_2 FSL_HELO_BARE_IP_2 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:15462 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UHcV8-0004FP-Su (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 16:01:32 +0000 Received: (qmail 7679 invoked by uid 634); 18 Mar 2013 16:01:30 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED,TVD_RCVD_IP X-Spam-Check-By: dark.recoil.org Received: from 64.125.195.54.t00963-01.above.net (HELO [10.99.98.1]) (64.125.195.54) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Mon, 18 Mar 2013 16:01:30 +0000 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: invertible protocol descriptions in Coq From: Anil Madhavapeddy In-Reply-To: Date: Mon, 18 Mar 2013 11:01:30 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Jon Crowcroft X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Vincent Hanquez , "cl-mirage@lists.cam.ac.uk List" , Prashanth Mundkur X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Mar 2013 16:01:32 -0000 Content-Length: 1815 Lines: 46 Right, but this is purely about the lexical/parsing problem, which is = necessary (but not sufficient) to implement the whole protocol. I = believe Lem/Ott are more appropriate for specifying the dynamic semantic = specification of how these 'lexed' packets should be put together. *Invertible* parsing is very useful though -- it covers the efficient = construction of packets rather than just classification of incoming = packets, and reduces code duplication. This is what MPL solved, as well = as Prashanth's follow up FP language [1], and the paper below. [1] https://github.com/pmundkur/fp -anil On 18 Mar 2013, at 06:59, Jon Crowcroft = wrote: > hm - so i think this is a weird paper - the example from eric = rescorla's TLS work is easily sorted with an LALR1 parser and = tokenisning the input stream not at the PACKET level, or byte, but by = turning the protocol into a collection of microprotocols- indeed, this = is how most people think about TCP - tcp is a collection of seperate = protocols (state management for connection setup/teardown; sequenced = data delivery; flow control; congestion control' urgent pointer; etc) = living inside one packet format - don't get hung up on the packet format = - that is a mere _lexical_ problem... >=20 > On Sat, Mar 16, 2013 at 9:21 PM, Anil Madhavapeddy = wrote: > Found this very cool paper on specifying invertible protocol = descriptions in Coq: >=20 > = http://staff.aist.go.jp/reynald.affeldt/documents/plpv2012-preprint.pdf >=20 > ..and they cover TLS. It would be a nice project to extract this to = OCaml/Haskell > to guarantee parsing equivalence from the formal spec (this is = basically a much > better implementation of MPL or FP, embedded in the Coq proof = assistant). >=20 > -anil >=20 From Richard.Mortier@nottingham.ac.uk Mon Mar 18 16:07:04 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHcaW-0000fx-Rl (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 16:07:04 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1457395 * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine03-20433-10.icritical.com ([93.159.202.53]:47251) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UHcaV-0000T0-ZP (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 16:07:04 +0000 Received: (qmail 1420 invoked from network); 18 Mar 2013 16:06:59 -0000 Received: from localhost (127.0.0.1) by engine03-20433-10.icritical.com with SMTP; 18 Mar 2013 16:06:59 -0000 Received: from engine03-20433-10.icritical.com ([127.0.0.1]) by localhost (engine03-20433-10.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 01331-03 for ; Mon, 18 Mar 2013 16:06:59 +0000 (GMT) Received: (qmail 30801 invoked by uid 599); 18 Mar 2013 16:03:36 -0000 Received: from unknown (HELO smtp4.nottingham.ac.uk) (128.243.220.65) by engine03-20433-10.icritical.com (qpsmtpd/0.28) with ESMTP; Mon, 18 Mar 2013 16:03:36 +0000 Received: from uiwexhub01.ad.nottingham.ac.uk ([128.243.15.133]) by smtp4.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1UHcXE-0002dh-1z for cl-mirage@lists.cam.ac.uk; Mon, 18 Mar 2013 16:03:40 +0000 From: Richard Mortier To: Anastasia Gornostaeva Date: Mon, 18 Mar 2013 16:03:38 +0000 Subject: Re: ocaml-dns Thread-Topic: ocaml-dns Thread-Index: Ac4j8ieUxdL2Gg2cQmC9nPGu4v5RRQ== Message-ID: References: <0EFB8E47-92E2-4D26-A784-BDD85B7D94E9@recoil.org> <20121114124350.GA77471@ermine.home> <20121204125604.GA10989@dark.recoil.org> <20130314185125.GA91758@ermine.home> <20130317145249.GA19172@ermine.home> <20130317160533.GB19172@ermine.home> <20130318084552.GA57194@ermine.home> In-Reply-To: <20130318084552.GA57194@ermine.home> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine03-20433-10.icritical.com Cc: cl-mirage , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Mar 2013 16:07:04 -0000 Content-Length: 2156 Lines: 46 hi anastasia; On 18 Mar 2013, at 03:45, Anastasia Gornostaeva wrote: > It seems Dns does not provide any way to connect to DNS server and resolv= e > any queries. Am I wrong? >=20 > I've found Dns_resolver for using with lwt, but it absents by default (ev= en in > opam installation). Should I re-implement Dns_resolver for plain networki= ng? we use Lwt for the networking here, so we never did one for "plain networki= ng". when you say "absents by default", what do you mean? if you have the n= ecessary libraries installed, do a "make distclean" and then "make" it shou= ld cause the oasis setup to run with --enable-tests and --enable-lwt which = causes mldig to be built, giving a simple command line client. let me know = if it doesn't work. (fwiw using 4.00.1 system compiler on osx, i had to "opam install cstruct r= e lwt cmdliner cryptokit uri" before i got a build working; but it did work= once i installed those libs.) really, the thing to do would be to follow the pattern that's being taken w= ith cohttp -- see eg., https://github.com/yallop/ocaml-cohttp. that is, cre= ate a client monad that abstracts over specific backends (Lwt, Core, etc) a= nd can encapsulate the (somewhat complex) logic that a DNS client has to im= plement. this would go into the lib/ directory, and there would then be spe= cific client instantiations client_lwt/ client_core/ and so on. --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From anil@recoil.org Mon Mar 18 16:16:39 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHcjn-0000uu-Nb (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 16:16:39 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1457395 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:18510 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UHcjj-0001uU-Sv (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 16:16:39 +0000 Received: (qmail 7714 invoked by uid 634); 18 Mar 2013 16:16:35 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from airband-63-133-128-186.airband.net (HELO [10.99.98.1]) (63.133.128.186) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Mon, 18 Mar 2013 16:16:35 +0000 Content-Type: text/plain; charset=koi8-r Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: ocaml-dns From: Anil Madhavapeddy In-Reply-To: Date: Mon, 18 Mar 2013 11:16:35 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <5EAC217A-5E19-40A3-BE73-4194122AC6F6@recoil.org> References: <0EFB8E47-92E2-4D26-A784-BDD85B7D94E9@recoil.org> <20121114124350.GA77471@ermine.home> <20121204125604.GA10989@dark.recoil.org> <20130314185125.GA91758@ermine.home> <20130317145249.GA19172@ermine.home> <20130317160533.GB19172@ermine.home> <20130318084552.GA57194@ermine.home> To: Richard Mortier X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage , Anastasia Gornostaeva X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Mar 2013 16:16:39 -0000 Content-Length: 2053 Lines: 49 On 18 Mar 2013, at 11:03, Richard Mortier = wrote: > hi anastasia; >=20 > On 18 Mar 2013, at 03:45, Anastasia Gornostaeva wrote: >=20 >> It seems Dns does not provide any way to connect to DNS server and = resolve >> any queries. Am I wrong? >>=20 >> I've found Dns_resolver for using with lwt, but it absents by default = (even in >> opam installation). Should I re-implement Dns_resolver for plain = networking? >=20 > we use Lwt for the networking here, so we never did one for "plain = networking". when you say "absents by default", what do you mean? if you = have the necessary libraries installed, do a "make distclean" and then = "make" it should cause the oasis setup to run with --enable-tests and = --enable-lwt which causes mldig to be built, giving a simple command = line client. let me know if it doesn't work. >=20 > (fwiw using 4.00.1 system compiler on osx, i had to "opam install = cstruct re lwt cmdliner cryptokit uri" before i got a build working; but = it did work once i installed those libs.) Lwt is an optional dependency, so it should work with 'opam install lwt = dns'. Mort's example above is required to satisfy the build = dependencies, whereas my line will just install the DNS library. > really, the thing to do would be to follow the pattern that's being = taken with cohttp -- see eg., https://github.com/yallop/ocaml-cohttp. = that is, create a client monad that abstracts over specific backends = (Lwt, Core, etc) and can encapsulate the (somewhat complex) logic that a = DNS client has to implement. this would go into the lib/ directory, and = there would then be specific client instantiations client_lwt/ = client_core/ and so on. Yeah. There's a non-trivial amount of logic in a client resolver = (iterative, recursive, timeouts, etc), and it would be great to factor = that out across Lwt and Async. Anastasia, are you using Lwt in your code, or native Unix IO? If the = latter, we'll need to add a new backend for that (presumably using = threads to block?) -anil= From ermine@sulci.ru Mon Mar 18 16:51:54 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHdHu-0003Pk-Be (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 16:51:54 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1457395 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from av6800.comex.ru ([217.10.34.155]:33885 helo=home.mk.pp.ru) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UHdHt-0003HZ-XM (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 16:51:54 +0000 Received: from ermine.home (ermine.home [192.168.2.2]) by home.mk.pp.ru (Postfix) with ESMTP id A187C20A51; Mon, 18 Mar 2013 20:51:50 +0400 (MSK) Received: by ermine.home (Postfix, from userid 1002) id 15D032E0E1; Mon, 18 Mar 2013 20:51:01 +0400 (MSK) Date: Mon, 18 Mar 2013 20:51:01 +0400 From: Anastasia Gornostaeva To: Anil Madhavapeddy Subject: Re: ocaml-dns Message-ID: <20130318165101.GA61221@ermine.home> References: <20121204125604.GA10989@dark.recoil.org> <20130314185125.GA91758@ermine.home> <20130317145249.GA19172@ermine.home> <20130317160533.GB19172@ermine.home> <20130318084552.GA57194@ermine.home> <5EAC217A-5E19-40A3-BE73-4194122AC6F6@recoil.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <5EAC217A-5E19-40A3-BE73-4194122AC6F6@recoil.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: cl-mirage X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Mar 2013 16:51:54 -0000 Content-Length: 671 Lines: 18 Hello, 1) I dont know who installed dns by opam. I didnd issue command 'opam install dns' It was installed by some depedency. I dont know -- there is no analogue of REQUIRED_BY feature from FreeBSD Packages. 2) > Anastasia, are you using Lwt in your code, or native Unix IO? If the latter, we'll need to add a new backend for that (presumably using threads to block?) I want to resolve SRV and TXT records in library that can use lwt (not link against lwt directly). But it isn't necessary to use threads when you need only select one host from SRV record, connect to and live in that connection up to death. Mu library is http://github.com/ermine/xmpp :-) ermine From anil@recoil.org Mon Mar 18 17:05:01 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHdUb-0003h7-Pe (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 17:05:01 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamScore: ss X-Cam-SpamDetails: score 2.4 from SpamAssassin-3.3.2-1457395 * 2.4 FSL_HELO_BARE_IP_2 FSL_HELO_BARE_IP_2 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:21261 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UHdUb-0005lZ-QZ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 17:05:01 +0000 Received: (qmail 8649 invoked by uid 634); 18 Mar 2013 17:05:01 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED,TVD_RCVD_IP X-Spam-Check-By: dark.recoil.org Received: from 64.125.195.54.t00963-01.above.net (HELO [10.99.98.1]) (64.125.195.54) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Mon, 18 Mar 2013 17:05:00 +0000 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: ANN: cohttp-0.9.6 Date: Mon, 18 Mar 2013 12:05:03 -0500 Message-Id: <0755428F-4456-4755-B226-508B863980E1@recoil.org> To: cl-mirage List Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Jeremy Yallop X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Mar 2013 17:05:01 -0000 Content-Length: 452 Lines: 14 Cohttp-0.9.6 is available on OPAM now: 0.9.6 (2012-03-18): * Depend on Async (>=3D 109.12.00), which has an incompatible API with = earlier versions. * Rearrange core library files for `obuild` support. Note that the next version of Cohttp will have an incompatible = interface, as Jeremy Yallop is refactoring it to share code across Lwt = and Async. This release is just a point release to clear the master = branch for Jeremy's changes. -anil= From prashanth.mundkur@gmail.com Mon Mar 18 22:03:49 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHi9l-0001k5-Sh (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 22:03:49 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1457395 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.210.45 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (prashanth.mundkur[at]gmail.com) * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-da0-f45.google.com ([209.85.210.45]:47461) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UHi9k-0008IE-Sz (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 18 Mar 2013 22:03:49 +0000 Received: by mail-da0-f45.google.com with SMTP id v40so1604776dad.4 for ; Mon, 18 Mar 2013 15:03:48 -0700 (PDT) X-Received: by 10.68.75.109 with SMTP id b13mr35942622pbw.25.1363644227883; Mon, 18 Mar 2013 15:03:47 -0700 (PDT) Received: from damage.epl.nokia ([131.228.210.43]) by mx.google.com with ESMTPS id tm1sm21537747pbc.11.2013.03.18.15.03.46 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 18 Mar 2013 15:03:47 -0700 (PDT) Date: Mon, 18 Mar 2013 14:02:21 -0700 From: Prashanth Mundkur To: Anil Madhavapeddy Subject: Re: invertible protocol descriptions in Coq Message-ID: <20130318210221.GA6379@damage.epl.nokia> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Vincent Hanquez , Jon Crowcroft , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Mar 2013 22:03:50 -0000 Content-Length: 2049 Lines: 37 Interesting, thanks! Some folks are getting back to even the lexical/parsing aspects of packet parsing for security: http://www.cs.dartmouth.edu/~sergey/langsec/ --prashanth On 11:01 Mon 18 Mar, Anil Madhavapeddy wrote: > Right, but this is purely about the lexical/parsing problem, which is necessary (but not sufficient) to implement the whole protocol. I believe Lem/Ott are more appropriate for specifying the dynamic semantic specification of how these 'lexed' packets should be put together. > > *Invertible* parsing is very useful though -- it covers the efficient construction of packets rather than just classification of incoming packets, and reduces code duplication. This is what MPL solved, as well as Prashanth's follow up FP language [1], and the paper below. > > [1] https://github.com/pmundkur/fp > > -anil > > On 18 Mar 2013, at 06:59, Jon Crowcroft wrote: > > > hm - so i think this is a weird paper - the example from eric rescorla's TLS work is easily sorted with an LALR1 parser and tokenisning the input stream not at the PACKET level, or byte, but by turning the protocol into a collection of microprotocols- indeed, this is how most people think about TCP - tcp is a collection of seperate protocols (state management for connection setup/teardown; sequenced data delivery; flow control; congestion control' urgent pointer; etc) living inside one packet format - don't get hung up on the packet format - that is a mere _lexical_ problem... > > > > On Sat, Mar 16, 2013 at 9:21 PM, Anil Madhavapeddy wrote: > > Found this very cool paper on specifying invertible protocol descriptions in Coq: > > > > http://staff.aist.go.jp/reynald.affeldt/documents/plpv2012-preprint.pdf > > > > ..and they cover TLS. It would be a nice project to extract this to OCaml/Haskell > > to guarantee parsing equivalence from the formal spec (this is basically a much > > better implementation of MPL or FP, embedded in the Coq proof assistant). > > > > -anil > > > -- prashanth From anil@recoil.org Tue Mar 19 14:36:39 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UHxeZ-0004hS-6l (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 19 Mar 2013 14:36:39 +0000 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:15006 helo=dark.recoil.org) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1UHxcd-0006mc-7O (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 19 Mar 2013 14:36:39 +0000 Received: (qmail 8222 invoked by uid 634); 19 Mar 2013 14:34:38 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from Unknown (HELO [172.20.0.32]) (12.232.133.66) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 19 Mar 2013 14:34:36 +0000 Content-Type: text/plain; charset=koi8-r Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: ocaml-dns From: Anil Madhavapeddy In-Reply-To: <20130318165101.GA61221@ermine.home> Date: Tue, 19 Mar 2013 09:34:37 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20121204125604.GA10989@dark.recoil.org> <20130314185125.GA91758@ermine.home> <20130317145249.GA19172@ermine.home> <20130317160533.GB19172@ermine.home> <20130318084552.GA57194@ermine.home> <5EAC217A-5E19-40A3-BE73-4194122AC6F6@recoil.org> <20130318165101.GA61221@ermine.home> To: Anastasia Gornostaeva X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Mar 2013 14:36:39 -0000 Content-Length: 1087 Lines: 34 On 18 Mar 2013, at 11:51, Anastasia Gornostaeva wrote: >=20 > 1) I dont know who installed dns by opam. I didnd issue command 'opam = install dns' > It was installed by some depedency. I dont know -- there is no = analogue of > REQUIRED_BY feature from FreeBSD Packages. Good point; there might be a way to do this already by inspecting the = roots files within ~/.opam, but I've created an issue to track the question: https://github.com/OCamlPro/opam/issues/545 >=20 > 2) >> Anastasia, are you using Lwt in your code, or native Unix IO? If the = latter, we'll need to add a new backend for that (presumably using = threads to block?) >=20 > I want to resolve SRV and TXT records in library that can use lwt (not = link > against lwt directly). > But it isn't necessary to use threads when you need only select one = host from SRV > record, connect to and live in that connection up to death. > Mu library is http://github.com/ermine/xmpp :-) Ok, so you want to do blocking UNIX I/O for DNS. That should be easy = enough to make a module variant for.=20 -anil From Richard.Mortier@nottingham.ac.uk Fri Mar 22 13:13:43 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ1mx-000710-Ed (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 13:13:43 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1459191 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine03-20433-11.icritical.com ([93.159.202.117]:37261) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UJ1mw-0008J4-ZP (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 13:13:43 +0000 Received: (qmail 23792 invoked from network); 19 Mar 2013 17:17:14 -0000 Received: from localhost (127.0.0.1) by engine03-20433-11.icritical.com with SMTP; 19 Mar 2013 17:17:14 -0000 Received: from engine03-20433-11.icritical.com ([127.0.0.1]) by localhost (engine03-20433-11.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 22600-05 for ; Tue, 19 Mar 2013 17:16:31 +0000 (GMT) Received: (qmail 21839 invoked by uid 599); 19 Mar 2013 17:12:11 -0000 Received: from unknown (HELO smtp3.nottingham.ac.uk) (128.243.44.55) by engine03-20433-11.icritical.com (qpsmtpd/0.28) with ESMTP; Tue, 19 Mar 2013 17:12:11 +0000 Received: from uiwexhub01.ad.nottingham.ac.uk ([128.243.15.133]) by smtp3.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1UHz3Q-00037b-7t for cl-mirage@lists.cam.ac.uk; Tue, 19 Mar 2013 16:06:24 +0000 From: Richard Mortier To: cl-mirage List Date: Tue, 19 Mar 2013 16:06:22 +0000 Subject: very odd behaviour using 4.00.1+mirage-unix Thread-Topic: very odd behaviour using 4.00.1+mirage-unix Thread-Index: Ac4ku7OIR6cljfW/Q5ORPbx7WZiM1g== Message-ID: <6D68D9CA-E4F9-4336-B6F9-9E2B00821CAA@nottingham.ac.uk> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine03-20433-11.icritical.com X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 13:13:43 -0000 Content-Length: 2995 Lines: 82 hi all; i am seeing very odd behaviour using the unix direct networking stack on li= nux via eg., mirage-www.=20 on osx, using "opam switch 4.00.1+mirage-unix", when i "sudo make run" in m= irage-www, all is fine -- the webserver runs, i can browse to 10.0.0.2:80, = retrieve pages, etc. no problems. similarly using "opam switch 4.00.1" and = browsing to localhost:80.=20 when i do the same thing on linux, everything is fine under "opam switch 4.= 00.1" -- i can browse to localhost:80 without problems.=20 but if i "opam switch 4.00.1+mirage-unix", browsing fails -- nothing listen= ing on 10.0.0.2:80 (i get a TCP RST). this occurs with ubuntu 12.04 running= linux 3.2.0-23 on top of virtualbox on a mac or on windows; and on ubuntu = 10.04 running linux 2.6.32 as a dom0 on xen 4.1 pvops. investigating so far, it seems that the tap0 interface is created and given= the ip address 10.0.0.2; and an appropriate route table entry is created: Destination Gateway Genmask Flags Metric Ref Use Ifa= ce 10.0.0.0 * 255.255.255.0 U 0 0 0 tap= 0 ...but when i examine the route cache, it seems to have mapped 10.0.0.2 to = lo rather than tap0: vagrant@precise64:~$ ip route show cache | grep -A1 10.0.0.2 local 10.0.0.2 from 10.0.0.2 tos lowdelay dev lo cache -- 10.0.0.2 from 10.0.0.2 tos lowdelay dev lo cache local 10.0.0.2 tos lowdelay dev lo src 10.0.0.2 cache tcpdump bears this out -- the SYN is visible on lo not on tap0. i have also tried this with linux 3.7 on virtualbox/osx as that kernel rev = removed the route cache, but see the same brokenness. looking at all routin= g tables, everything seems in order: vagrant@precise64:~$ ip route list table all | grep 10.0.0.2 10.0.0.0/24 dev tap0 proto kernel scope link src 10.0.0.2 broadcast 10.0.0.0 dev tap0 table local proto kernel scope link src 10.= 0.0.2 local 10.0.0.2 dev tap0 table local proto kernel scope host src 10.0.0.= 2 broadcast 10.0.0.255 dev tap0 table local proto kernel scope link src 1= 0.0.0.2 =20 however tcpdump shows the same results as before -- the TCP SYN is visible = on lo not on tap0, and gets a RST in response as a result. pinging 10.0.0.2 in all cases works just fine, presumably because that's be= ing dealt with in-kernel and isn't hitting whichever part of the network st= ack that is confused. all-- has anyone else seen any problems like this using mirage-unix on linu= x, or does it all just work for you? i don't have a box that's just running= vanilla linux on the metal without some form of virtualisation to test tha= t -- any chance someone else does? is it possible it's specifically a tunt= ap/virtualisation problem? vincent-- i understand you're working on ocaml tuntap bindings? any chance= they might have some unit tests and are available yet -- i think that woul= d really help me try to debug this... --=20 Cheers, R. -- Scanned by iCritical. From vb@luminar.eu.org Wed Mar 20 18:34:49 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UINqb-0001E2-AA (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 20 Mar 2013 18:34:49 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.5 from SpamAssassin-3.3.2-1458179 * -2.5 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:60258) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UINqa-0001ex-ZN (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 20 Mar 2013 18:34:49 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id A9A61BFC80 for ; Wed, 20 Mar 2013 19:34:48 +0100 (CET) Message-ID: <514A0148.1090501@luminar.eu.org> Date: Wed, 20 Mar 2013 18:34:48 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130221 Thunderbird/17.0.3 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Net.Manager Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Mar 2013 18:34:49 -0000 Content-Length: 255 Lines: 12 Hi, In the UNIX backend, Net.Manager.configure is used but apparently does not anything (there is a hack in OS.Netif that hardcode the IP). How is it supposed to work in the end ? Can Net.Manager.configure can work at all on UNIX ? Cheers, Vincent From vb@luminar.eu.org Wed Mar 20 20:24:58 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UIPZC-0002pc-8R (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 20 Mar 2013 20:24:58 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.5 from SpamAssassin-3.3.2-1458179 * -2.5 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:40334) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UIPZB-0004iw-YR (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 20 Mar 2013 20:24:58 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id 74B2FBF576 for ; Wed, 20 Mar 2013 21:24:57 +0100 (CET) Message-ID: <514A1B19.3000702@luminar.eu.org> Date: Wed, 20 Mar 2013 20:24:57 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130221 Thunderbird/17.0.3 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Updates Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Mar 2013 20:24:58 -0000 Content-Length: 1317 Lines: 40 Hi, I=92m making progresses on the ocaml-tuntap library=20 (github/vbmithr/ocaml-tuntap). It works fairly well on Linux. On MacOSX,=20 no work has been done. I set up a Mac today and installed opam on it. I=20 could not make obuild compile the library though, see: https://github.com/vincenthz/obuild/issues/29 I have been able to delete completely the bindings from mirage-platform=20 and have ocaml-tuntap do all what was done previously, it works well=20 (under Linux for now). Given that we have an OCaml fd-passing library from Dave, I suggest that=20 we use systematically fd-passing and have an uniform interface between=20 Linux/OSX. So Mirage on UNIX: 1. the unikernel listens to a unix domain socket somewhere 2. An external tool (mirari run) creates a tap interface via=20 ocaml-tuntap and gives it to the unikernel (via the unix domain socket).=20 It also sets an IP (the one used by the unikernel) on the tap interface. There is no differentiation between PCAP and ETH anymore, all PCAP stuff=20 is dropped. The code will be exactly the same between Linux and MacOSX. That would complete mirari-run on UNIX pretty much. Next step for me to implement: ocaml-platform does not use tuntap=20 directly, but instead listens to a unix domain socket and wait for the fd= . That all folks Vincent From vb@luminar.eu.org Fri Mar 22 11:10:31 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UIzrj-0004gn-In (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 11:10:31 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.5 from SpamAssassin-3.3.2-1459191 * -2.5 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:58196) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UIzrj-0001aw-QF (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 11:10:31 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id AC836BF5A4; Fri, 22 Mar 2013 12:10:30 +0100 (CET) Message-ID: <514C3C26.8080409@luminar.eu.org> Date: Fri, 22 Mar 2013 11:10:30 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130221 Thunderbird/17.0.3 MIME-Version: 1.0 To: Richard Mortier , cl-mirage@lists.cam.ac.uk Subject: Re: Net.Manager References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> In-Reply-To: <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 11:10:31 -0000 Content-Length: 1014 Lines: 36 On 22/03/2013 10:35, Richard Mortier wrote: > did you get any answers to this? still haven't had a response to my mai= l about problems on linux... :/ > > having said that, it sounds like your ocaml-tuntap will fix all this an= yway:) > > On 20 Mar 2013, at 18:34, Vincent Bernardoff wrote: Hi Richard, Yeah, I have a quite clear vision about what to do. If I=92m correct: Tuntap support in macs is limited, they can only create transcient=20 tun/tap interfaces through tuntaposx. PCAP was used have macs able to attach to a tun/tap using a different=20 library (PCAP). I don=92t know exactly why PCAP was used for. My idea is to use ocaml-tuntap in macosx and linux the same way,=20 creating transcient interfaces and passing the fd over a unix domain=20 socket. This would work in all cases. Currently I=92m blocked because obuild does not build ocaml-tuntap on mac= s=20 (And since I don=92t want to change build systems I=92m currently trying = to=20 fix obuild). That=92s about it. Cheers, Vincent From Richard.Mortier@nottingham.ac.uk Fri Mar 22 13:22:11 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ1v9-0007Kt-71 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 13:22:11 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1459191 * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine01-20433-9.icritical.com ([151.236.209.34]:43478) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UJ1v8-0004qz-SH (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 13:22:11 +0000 Received: (qmail 22790 invoked from network); 22 Mar 2013 13:22:07 -0000 Received: from localhost (127.0.0.1) by engine01-20433-9.icritical.com with SMTP; 22 Mar 2013 13:22:07 -0000 Received: from engine01-20433-9.icritical.com ([127.0.0.1]) by localhost (engine01-20433-9.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 22781-01 for ; Fri, 22 Mar 2013 13:22:06 +0000 (GMT) Received: (qmail 20935 invoked by uid 599); 22 Mar 2013 13:18:42 -0000 Received: from unknown (HELO smtp4.nottingham.ac.uk) (128.243.220.65) by engine01-20433-9.icritical.com (qpsmtpd/0.28) with ESMTP; Fri, 22 Mar 2013 13:18:42 +0000 Received: from uiwexhub01.ad.nottingham.ac.uk ([128.243.15.133]) by smtp4.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1UJ1rs-0004ux-Od for cl-mirage@lists.cam.ac.uk; Fri, 22 Mar 2013 13:18:48 +0000 From: Richard Mortier To: cl-mirage List Date: Fri, 22 Mar 2013 13:18:48 +0000 Subject: Re: Net.Manager Thread-Topic: Net.Manager Thread-Index: Ac4m/8mYQns28Ej1RH+mHFb00xyH6Q== Message-ID: <991850A8-3539-473A-B266-E788A6C0BA6B@nottingham.ac.uk> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> In-Reply-To: <514C3C26.8080409@luminar.eu.org> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine01-20433-9.icritical.com Cc: Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 13:22:11 -0000 Content-Length: 1223 Lines: 27 On 22 Mar 2013, at 11:10, Vincent Bernardoff wrote: > On 22/03/2013 10:35, Richard Mortier wrote: >> did you get any answers to this? still haven't had a response to my mail= about problems on linux... :/ (and, in case anyone was wondering wtf i'm on about, i've just seen said ma= il actually hit the list. it's quite possible it was held up by our embarra= ssingly bad IS and mail system here at nottingham... would still love an an= swer though :) --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From h.rotsos@gmail.com Fri Mar 22 13:45:34 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ2Hm-0000K1-ML (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 13:45:34 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1459191 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.214.178 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (h.rotsos[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ob0-f178.google.com ([209.85.214.178]:40517) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UJ2Hl-0002Ma-sW (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 13:45:34 +0000 Received: by mail-ob0-f178.google.com with SMTP id wd20so4037922obb.9 for ; Fri, 22 Mar 2013 06:45:33 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.60.31.225 with SMTP id d1mr1725023oei.120.1363959932967; Fri, 22 Mar 2013 06:45:32 -0700 (PDT) Sender: h.rotsos@gmail.com Received: by 10.60.24.231 with HTTP; Fri, 22 Mar 2013 06:45:32 -0700 (PDT) In-Reply-To: <514C3C26.8080409@luminar.eu.org> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> Date: Fri, 22 Mar 2013 13:45:32 +0000 X-Google-Sender-Auth: OXsjkFsQlDYQb0YuHHxxPyMi3UE Message-ID: Subject: Re: Net.Manager From: Haris Rotsos To: Vincent Bernardoff Content-Type: multipart/alternative; boundary=e89a8fb1fbe24ae55504d883ab5d Cc: Richard Mortier , "cl-mirage@lists.cam.ac.uk Mailing List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 13:45:34 -0000 Content-Length: 6586 Lines: 173 --e89a8fb1fbe24ae55504d883ab5d Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Configure is effective in the Unix backend. The trick is to ifconfig appropriately the tun interface once it is created. The default configuration will simply assign a 10.0.X.1 ip address, where X is the id of the device in the Net stack of Mirage. On 22 March 2013 11:10, Vincent Bernardoff wrote: > On 22/03/2013 10:35, Richard Mortier wrote: > >> did you get any answers to this? still haven't had a response to my mail >> about problems on linux... :/ >> >> having said that, it sounds like your ocaml-tuntap will fix all this >> anyway:) >> >> On 20 Mar 2013, at 18:34, Vincent Bernardoff wrote: >> > > Hi Richard, > > > Yeah, I have a quite clear vision about what to do. If I=E2=80=99m correc= t: > > Tuntap support in macs is limited, they can only create transcient tun/ta= p > interfaces through tuntaposx. > in order to create a tuntap interface you need to open an fd to a /dev/tapX or /dev/taunX. The device is open only while the problem is running and keeps the fd open. This is the case for both linux and mac. The difference is that on linux there is a tun daemon and thep tunctl command which can persist a tun interface without a requirement for an fd. > > PCAP was used have macs able to attach to a tun/tap using a different > library (PCAP). I don=E2=80=99t know exactly why PCAP was used for. > Pcap is used in the unix backend in order to create a user space openflow switch in MacOSX. In such a setting you need to be able to have both tun/tap interfaces as well as ethernet network devices. In MacOSX there is no ethernet raw socket thing unfortunately and the best way to intercept traffic in such a case is through a bpf device. > > My idea is to use ocaml-tuntap in macosx and linux the same way, creating > transcient interfaces and passing the fd over a unix domain socket. This > would work in all cases. > > In this case you won't be able to integrate an existing network interface to the Mirage Network stack. You need at least to consider raw ethernet sockets for linux, bpf for MacOSX and find some nice abstraction to expose this through the Manager(?) API. > Currently I=E2=80=99m blocked because obuild does not build ocaml-tuntap = on macs > (And since I don=E2=80=99t want to change build systems I=E2=80=99m curre= ntly trying to fix > obuild). > > That=E2=80=99s about it. > > Cheers, > > Vincent > > --=20 Charalampos Rotsos PhD student The University of Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD Phone: +44-(0) 1223 767032 Email: cr409@cl.cam.ac.uk --e89a8fb1fbe24ae55504d883ab5d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Configure is effective in the Unix backend. The trick is t= o ifconfig appropriately the tun interface once it is created. The default = configuration will simply assign a 10.0.X.1 ip address, where X is the id o= f the device in the Net stack of Mirage.=C2=A0


On 22 March 2= 013 11:10, Vincent Bernardoff <vb@luminar.eu.org> wrote:
=
On 22/03/2013 10:35, Richard Mortier wrote:
did you get any answers to this? still haven't had a response to my mai= l about problems on linux... :/

having said that, it sounds like your ocaml-tuntap will fix all this anyway= :)

On 20 Mar 2013, at 18:34, Vincent Bernardoff wrote:

Hi Richard,


Yeah, I have a quite clear vision about what to do. If I=E2=80=99m correct:=

Tuntap support in macs is limited, they can only create transcient tun/tap = interfaces through tuntaposx.

in = order to create a tuntap interface you need to open an fd to a /dev/tapX or= /dev/taunX. The device is open only while the problem is running and keeps= the fd open. This is the case for both linux and mac. The difference is th= at on linux there is a tun daemon and thep tunctl command which can persist= a tun interface without a requirement for an fd.=C2=A0
=C2=A0

PCAP was used have macs able to attach to a tun/tap using a different libra= ry (PCAP). I don=E2=80=99t know exactly why PCAP was used for.

Pcap is used in the unix backend in order to = create a user space openflow switch in MacOSX. In such a setting you need t= o be able to have both tun/tap interfaces as well as ethernet network devic= es. In MacOSX there is no ethernet raw socket thing unfortunately and the b= est way to intercept traffic in such a case is through a bpf device.
=C2=A0

My idea is to use ocaml-tuntap in macosx and linux the same way, creating t= ranscient interfaces and passing the fd over a unix domain socket. This wou= ld work in all cases.


In this case you won't be ab= le to integrate an existing network interface to the Mirage Network stack. = You need at least to consider raw ethernet sockets for linux, bpf for MacOS= X and find some nice abstraction to expose this through the Manager(?) API.= =C2=A0
=C2=A0
Currently I=E2=80=99m blocked because obuild does not build ocaml-tuntap on= macs (And since I don=E2=80=99t want to change build systems I=E2=80=99m c= urrently trying to fix obuild).

That=E2=80=99s about it.

Cheers,

Vincent




--
Charalampos = Rotsos
PhD student
The University of Cambridge
Computer Laboratory=
William Gates Building
JJ Thomson Avenue
Cambridge
CB3 0FD

Phone: +44-(0) 1223 767032
Email: cr409@cl.cam.ac.uk
--e89a8fb1fbe24ae55504d883ab5d-- From vb@luminar.eu.org Fri Mar 22 15:21:27 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ3mZ-0003Yh-Qt (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 15:21:27 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.5 from SpamAssassin-3.3.2-1459191 * -2.5 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:59690) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UJ3mY-0005LH-XZ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 15:21:27 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id 2AF66C0ACC for ; Fri, 22 Mar 2013 16:21:26 +0100 (CET) Message-ID: <514C76F5.7050705@luminar.eu.org> Date: Fri, 22 Mar 2013 15:21:25 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130221 Thunderbird/17.0.3 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: very odd behaviour using 4.00.1+mirage-unix References: <6D68D9CA-E4F9-4336-B6F9-9E2B00821CAA@nottingham.ac.uk> In-Reply-To: <6D68D9CA-E4F9-4336-B6F9-9E2B00821CAA@nottingham.ac.uk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 15:21:27 -0000 Content-Length: 944 Lines: 18 On 19/03/2013 16:06, Richard Mortier wrote: > all-- has anyone else seen any problems like this using mirage-unix on linux, or does it all just work for you? i don't have a box that's just running vanilla linux on the metal without some form of virtualisation to test that -- any chance someone else does? is it possible it's specifically a tuntap/virtualisation problem? > > vincent-- i understand you're working on ocaml tuntap bindings? any chance they might have some unit tests and are available yet -- i think that would really help me try to debug this... Last time I tried, it was working well on Linux (I managed to get the website working with the UNIX backend). Have a look at github.com/vbmithr/ocaml-tuntap for the bindings, there is a "tunctl" utility that you can try and the code is documented. I was not able to compile it under OSX for now (obuild problem), but the Linux version is working for me. Cheers, Vincent From vb@luminar.eu.org Fri Mar 22 15:43:48 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ48C-0004X5-Hp (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 15:43:48 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.5 from SpamAssassin-3.3.2-1459191 * -2.5 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:52851) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UJ48C-0001DU-QE (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 15:43:48 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id E0D9EC0ACC for ; Fri, 22 Mar 2013 16:43:47 +0100 (CET) Message-ID: <514C7C33.1050902@luminar.eu.org> Date: Fri, 22 Mar 2013 15:43:47 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Net.Manager References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 15:43:48 -0000 Content-Length: 2465 Lines: 66 On 22/03/2013 13:45, Haris Rotsos wrote: > in order to create a tuntap interface you need to open an fd to a > /dev/tapX or /dev/taunX. The device is open only while the problem is > running and keeps the fd open. This is the case for both linux and mac. No, in Linux you have open /dev/net/tun and use ioctl to configure the=20 mode of operation, if you want it persistent or not, user/group=20 ownership, etc. > The difference is that on linux there is a tun daemon and thep tunctl > command which can persist a tun interface without a requirement for an = fd. I did not see any "tun daemon" on my Linux whatsoever. I don=E2=80=99t us= e the=20 tunctl command either. I=E2=80=99m using the "ip" command (ip tuntap add = mode=20 tap) which creates a persistent tap for me. Indeed there is no=20 requirement for a fd. I made an OCaml "tunctl" command to test the=20 ocaml-tuntap bindings, it is correctly replicating the behaviour of the=20 "ip tuntap" command. > > PCAP was used have macs able to attach to a tun/tap using a > different library (PCAP). I don=E2=80=99t know exactly why PCAP was= used for. > > > Pcap is used in the unix backend in order to create a user space > openflow switch in MacOSX. In such a setting you need to be able to hav= e > both tun/tap interfaces as well as ethernet network devices. In MacOSX > there is no ethernet raw socket thing unfortunately and the best way to > intercept traffic in such a case is through a bpf device. > > > My idea is to use ocaml-tuntap in macosx and linux the same way, > creating transcient interfaces and passing the fd over a unix domai= n > socket. This would work in all cases. > > > In this case you won't be able to integrate an existing network > interface to the Mirage Network stack. You need at least to consider ra= w > ethernet sockets for linux, bpf for MacOSX and find some nice > abstraction to expose this through the Manager(?) API. Ok, just read about raw ethernet sockets. I did not know that it was=20 possible, it is interesting. The possibility I was thinking about was to=20 "integrate an existing network interface" in creating a tap interface=20 and bridge it with the real network interface. This way you probably can=20 read inbound/outbound ethernet frames. Would it be possible to do that=20 on MacOSX ? Could you please point me to some documentation on how to do that on Macs= ? Thank you very much for the explanation. Cheers, Vincent From anil@recoil.org Fri Mar 22 16:18:06 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ4fO-0005xz-9X (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:18:06 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1459191 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:11179 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UJ4fN-0004zN-YY (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:18:06 +0000 Received: (qmail 6182 invoked by uid 634); 22 Mar 2013 16:18:05 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.97]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 22 Mar 2013 16:18:05 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: Net.Manager From: Anil Madhavapeddy In-Reply-To: <514C7C33.1050902@luminar.eu.org> Date: Fri, 22 Mar 2013 16:18:02 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <7FB7524C-3253-4C2D-BB98-CAB308355F83@recoil.org> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> To: Vincent Bernardoff X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 16:18:06 -0000 Content-Length: 1435 Lines: 32 On 22 Mar 2013, at 15:43, Vincent Bernardoff wrote: >> In this case you won't be able to integrate an existing network >> interface to the Mirage Network stack. You need at least to consider = raw >> ethernet sockets for linux, bpf for MacOSX and find some nice >> abstraction to expose this through the Manager(?) API. >=20 > Ok, just read about raw ethernet sockets. I did not know that it was = possible, it is interesting. The possibility I was thinking about was to = "integrate an existing network interface" in creating a tap interface = and bridge it with the real network interface. This way you probably can = read inbound/outbound ethernet frames. Would it be possible to do that = on MacOSX ? I think we should separate out the concern of attaching an existing = network interface from just standard bridging. On the Mac, there's no = built-in support for bridging, which makes it important to attach an = existing interface. However, VirtualBox *does* have a kernel bridging module on MacOS X, so = could we perhaps use that somehow? However, the interface that involves opening an fd (through whatever = means, be it tuntap or bpf) and passing it through to the Mirage = application via sendfd seems to be the most robust solution in terms of = interface design. We could even write a manual MacOS bridge to send = traffic back-and-forth from an interface if we're really stuck later. -anil= From h.rotsos@gmail.com Fri Mar 22 16:21:40 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ4iq-00065D-PM (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:21:40 +0000 X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1459191 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.219.41 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (h.rotsos[at]gmail.com) * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-oa0-f41.google.com ([209.85.219.41]:39384) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UJ4ip-0005Us-7x (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:21:40 +0000 Received: by mail-oa0-f41.google.com with SMTP id f4so2002528oah.28 for ; Fri, 22 Mar 2013 09:21:38 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.60.171.102 with SMTP id at6mr2512145oec.60.1363969298589; Fri, 22 Mar 2013 09:21:38 -0700 (PDT) Sender: h.rotsos@gmail.com Received: by 10.60.24.231 with HTTP; Fri, 22 Mar 2013 09:21:38 -0700 (PDT) In-Reply-To: <514C7C33.1050902@luminar.eu.org> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> Date: Fri, 22 Mar 2013 16:21:38 +0000 X-Google-Sender-Auth: IGQVi4ozQn2suxWGTgYZ_BlZNVY Message-ID: Subject: Re: Net.Manager From: Haris Rotsos To: Vincent Bernardoff Content-Type: multipart/alternative; boundary=bcaec54ee09486f00904d885d9a7 Cc: "cl-mirage@lists.cam.ac.uk Mailing List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 16:21:40 -0000 Content-Length: 8282 Lines: 223 --bcaec54ee09486f00904d885d9a7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 22 March 2013 15:43, Vincent Bernardoff wrote: > On 22/03/2013 13:45, Haris Rotsos wrote: > > in order to create a tuntap interface you need to open an fd to a >> /dev/tapX or /dev/taunX. The device is open only while the problem is >> running and keeps the fd open. This is the case for both linux and mac. >> > > No, in Linux you have open /dev/net/tun and use ioctl to configure the > mode of operation, if you want it persistent or not, user/group ownership= , > etc. > > > The difference is that on linux there is a tun daemon and thep tunctl >> command which can persist a tun interface without a requirement for an f= d. >> > > I did not see any "tun daemon" on my Linux whatsoever. I don=E2=80=99t us= e the > tunctl command either. I=E2=80=99m using the "ip" command (ip tuntap add = mode tap) > which creates a persistent tap for me. Indeed there is no requirement for= a > fd. I made an OCaml "tunctl" command to test the ocaml-tuntap bindings, i= t > is correctly replicating the behaviour of the "ip tuntap" command. yes you are right. This was my errorus understanding of the tuntap module in linux. > > > >> PCAP was used have macs able to attach to a tun/tap using a >> different library (PCAP). I don=E2=80=99t know exactly why PCAP was = used for. >> >> >> Pcap is used in the unix backend in order to create a user space >> openflow switch in MacOSX. In such a setting you need to be able to have >> both tun/tap interfaces as well as ethernet network devices. In MacOSX >> there is no ethernet raw socket thing unfortunately and the best way to >> intercept traffic in such a case is through a bpf device. >> >> >> My idea is to use ocaml-tuntap in macosx and linux the same way, >> creating transcient interfaces and passing the fd over a unix domain >> socket. This would work in all cases. >> >> >> In this case you won't be able to integrate an existing network >> interface to the Mirage Network stack. You need at least to consider raw >> ethernet sockets for linux, bpf for MacOSX and find some nice >> abstraction to expose this through the Manager(?) API. >> > > Ok, just read about raw ethernet sockets. I did not know that it was > possible, it is interesting. The possibility I was thinking about was to > "integrate an existing network interface" in creating a tap interface and > bridge it with the real network interface. This way you probably can read > inbound/outbound ethernet frames. Would it be possible to do that on MacO= SX > ? > > Could you please point me to some documentation on how to do that on Macs= ? > Short is that there is no bridging functionality on MacOSX. Secondly, I am not sure but in linux I think that the bridge module is a learning bridge module, so if the kernel module learns your mac address then you might not receive any traffic there on. There is I think though a small trick to set the bridge interface into promiscuous mode, but I haven't tested it yet. > > Thank you very much for the explanation. > > Cheers, > > Vincent > > > --=20 Charalampos Rotsos PhD student The University of Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD Phone: +44-(0) 1223 767032 Email: cr409@cl.cam.ac.uk --bcaec54ee09486f00904d885d9a7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable



On 22 March 2013 15:43, Vincent Bernardoff <vb@luminar.eu.org&= gt; wrote:
On 22/03/2013 13:45, Haris Rotsos wrote= :

in order to create a tuntap interface you need to open an fd to a
/dev/tapX or /dev/taunX. The device is open only while the problem is
running and keeps the fd open. This is the case for both linux and mac.

No, in Linux you have open /dev/net/tun and use ioctl to configure the mode= of operation, if you want it persistent or not, user/group ownership, etc.=


The difference is that on linux there is a tun daemon and thep tunctl
command which can persist a tun interface without a requirement for an fd.<= br>

I did not see any "tun daemon" on my Linux whatsoever. I don=E2= =80=99t use the tunctl command either. I=E2=80=99m using the "ip"= command (ip tuntap add mode tap) which creates a persistent tap for me. In= deed there is no requirement for a fd. I made an OCaml "tunctl" c= ommand to test the ocaml-tuntap bindings, it is correctly replicating the b= ehaviour of the "ip tuntap" command.


yes you are right. This was my errorus u= nderstanding of the tuntap module in linux.=C2=A0
=C2=A0



=C2=A0 =C2=A0 PCAP was used have macs able to attach to a tun/tap using a =C2=A0 =C2=A0 different library (PCAP). I don=E2=80=99t know exactly why PC= AP was used for.


Pcap is used in the unix backend in order to create a user space
openflow switch in MacOSX. In such a setting you need to be able to have both tun/tap interfaces as well as ethernet network devices. In MacOSX
there is no ethernet raw socket thing unfortunately and the best way to
intercept traffic in such a case is through a bpf device.


=C2=A0 =C2=A0 My idea is to use ocaml-tuntap in macosx and linux the same w= ay,
=C2=A0 =C2=A0 creating transcient interfaces and passing the fd over a unix= domain
=C2=A0 =C2=A0 socket. This would work in all cases.


In this case you won't be able to integrate an existing network
interface to the Mirage Network stack. You need at least to consider raw ethernet sockets for linux, bpf for MacOSX and find some nice
abstraction to expose this through the Manager(?) API.

Ok, just read about raw ethernet sockets. I did not know that it was possib= le, it is interesting. The possibility I was thinking about was to "in= tegrate an existing network interface" in creating a tap interface and= bridge it with the real network interface. This way you probably can read = inbound/outbound ethernet frames. Would it be possible to do that on MacOSX= ?

Could you please point me to some documentation on how to do that on Macs ?=

Short is that there is no bridging fun= ctionality on MacOSX. Secondly, I am not sure but in linux I think that the= bridge module is a learning bridge module, so if the kernel module learns = your mac address then you might not receive any traffic there on. There is = I think though a small trick to set the bridge interface into promiscuous m= ode, but I haven't tested it yet.=C2=A0
=C2=A0

Thank you very much for the explanation.

Cheers,

Vincent





--
Charalampos = Rotsos
PhD student
The University of Cambridge
Computer Laboratory=
William Gates Building
JJ Thomson Avenue
Cambridge
CB3 0FD

Phone: +44-(0) 1223 767032
Email: cr409@cl.cam.ac.uk
--bcaec54ee09486f00904d885d9a7-- From h.rotsos@gmail.com Fri Mar 22 16:24:58 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ4m2-0006E4-PL (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:24:58 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1459191 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.219.45 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (h.rotsos[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-oa0-f45.google.com ([209.85.219.45]:40839) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UJ4m1-00087p-ZJ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:24:58 +0000 Received: by mail-oa0-f45.google.com with SMTP id o6so4458406oag.4 for ; Fri, 22 Mar 2013 09:24:57 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.60.10.226 with SMTP id l2mr2497039oeb.67.1363969497101; Fri, 22 Mar 2013 09:24:57 -0700 (PDT) Sender: h.rotsos@gmail.com Received: by 10.60.24.231 with HTTP; Fri, 22 Mar 2013 09:24:57 -0700 (PDT) In-Reply-To: References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> Date: Fri, 22 Mar 2013 16:24:57 +0000 X-Google-Sender-Auth: mJPTOjLMEq1YaTh6Z-nIav3Ppfw Message-ID: Subject: Re: Net.Manager From: Haris Rotsos To: Vincent Bernardoff Content-Type: multipart/alternative; boundary=e89a8fb1f8d05c00fa04d885e56f Cc: "cl-mirage@lists.cam.ac.uk Mailing List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 16:24:58 -0000 Content-Length: 10303 Lines: 285 --e89a8fb1f8d05c00fa04d885e56f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 22 March 2013 16:21, Haris Rotsos wrote: > > > > On 22 March 2013 15:43, Vincent Bernardoff wrote: > >> On 22/03/2013 13:45, Haris Rotsos wrote: >> >> in order to create a tuntap interface you need to open an fd to a >>> /dev/tapX or /dev/taunX. The device is open only while the problem is >>> running and keeps the fd open. This is the case for both linux and mac. >>> >> >> No, in Linux you have open /dev/net/tun and use ioctl to configure the >> mode of operation, if you want it persistent or not, user/group ownershi= p, >> etc. >> >> >> The difference is that on linux there is a tun daemon and thep tunctl >>> command which can persist a tun interface without a requirement for an >>> fd. >>> >> >> I did not see any "tun daemon" on my Linux whatsoever. I don=E2=80=99t u= se the >> tunctl command either. I=E2=80=99m using the "ip" command (ip tuntap add= mode tap) >> which creates a persistent tap for me. Indeed there is no requirement fo= r a >> fd. I made an OCaml "tunctl" command to test the ocaml-tuntap bindings, = it >> is correctly replicating the behaviour of the "ip tuntap" command. > > > > yes you are right. This was my errorus understanding of the tuntap module > in linux. > > >> >> >> >>> PCAP was used have macs able to attach to a tun/tap using a >>> different library (PCAP). I don=E2=80=99t know exactly why PCAP was= used for. >>> >>> >>> Pcap is used in the unix backend in order to create a user space >>> openflow switch in MacOSX. In such a setting you need to be able to hav= e >>> both tun/tap interfaces as well as ethernet network devices. In MacOSX >>> there is no ethernet raw socket thing unfortunately and the best way to >>> intercept traffic in such a case is through a bpf device. >>> >>> >>> My idea is to use ocaml-tuntap in macosx and linux the same way, >>> creating transcient interfaces and passing the fd over a unix domai= n >>> socket. This would work in all cases. >>> >>> >>> In this case you won't be able to integrate an existing network >>> interface to the Mirage Network stack. You need at least to consider ra= w >>> ethernet sockets for linux, bpf for MacOSX and find some nice >>> abstraction to expose this through the Manager(?) API. >>> >> >> Ok, just read about raw ethernet sockets. I did not know that it was >> possible, it is interesting. The possibility I was thinking about was to >> "integrate an existing network interface" in creating a tap interface an= d >> bridge it with the real network interface. This way you probably can rea= d >> inbound/outbound ethernet frames. Would it be possible to do that on Mac= OSX >> ? >> >> Could you please point me to some documentation on how to do that on Mac= s >> ? >> > > Short is that there is no bridging functionality on MacOSX. Secondly, I a= m > not sure but in linux I think that the bridge module is a learning bridge > module, so if the kernel module learns your mac address then you might no= t > receive any traffic there on. There is I think though a small trick to se= t > the bridge interface into promiscuous mode, but I haven't tested it yet. > Sorry, this makes little sense now that I am reading it again. The example I am thinking of is a user level network bridge that I am working on and this approach I think will make my life a bit difficult. For normal mirage apps the bridging approach should be ok. > > >> >> Thank you very much for the explanation. >> >> Cheers, >> >> Vincent >> >> >> > > > -- > Charalampos Rotsos > PhD student > The University of Cambridge > Computer Laboratory > William Gates Building > JJ Thomson Avenue > Cambridge > CB3 0FD > > Phone: +44-(0) 1223 767032 > Email: cr409@cl.cam.ac.uk > --=20 Charalampos Rotsos PhD student The University of Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD Phone: +44-(0) 1223 767032 Email: cr409@cl.cam.ac.uk --e89a8fb1f8d05c00fa04d885e56f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable



On 22 March 2013 16:21, Haris Rotsos <cr409@cl.cam.ac.uk>= wrote:



On 22 March 2013 1= 5:43, Vincent Bernardoff <vb@luminar.eu.org> wrote:
On 22/03/2013 13:45, Haris Rotsos wrote= :

in order to create a tuntap interface you need to open an fd to a
/dev/tapX or /dev/taunX. The device is open only while the problem is
running and keeps the fd open. This is the case for both linux and mac.

No, in Linux you have open /dev/net/tun and use ioctl to configure the mode= of operation, if you want it persistent or not, user/group ownership, etc.=


The difference is that on linux there is a tun daemon and thep tunctl
command which can persist a tun interface without a requirement for an fd.<= br>

I did not see any "tun daemon" on my Linux whatsoever. I don=E2= =80=99t use the tunctl command either. I=E2=80=99m using the "ip"= command (ip tuntap add mode tap) which creates a persistent tap for me. In= deed there is no requirement for a fd. I made an OCaml "tunctl" c= ommand to test the ocaml-tuntap bindings, it is correctly replicating the b= ehaviour of the "ip tuntap" command.


yes you are right. This was my err= orus understanding of the tuntap module in linux.=C2=A0
=C2=A0



=C2=A0 =C2=A0 PCAP was used have macs able to attach to a tun/tap using a =C2=A0 =C2=A0 different library (PCAP). I don=E2=80=99t know exactly why PC= AP was used for.


Pcap is used in the unix backend in order to create a user space
openflow switch in MacOSX. In such a setting you need to be able to have both tun/tap interfaces as well as ethernet network devices. In MacOSX
there is no ethernet raw socket thing unfortunately and the best way to
intercept traffic in such a case is through a bpf device.


=C2=A0 =C2=A0 My idea is to use ocaml-tuntap in macosx and linux the same w= ay,
=C2=A0 =C2=A0 creating transcient interfaces and passing the fd over a unix= domain
=C2=A0 =C2=A0 socket. This would work in all cases.


In this case you won't be able to integrate an existing network
interface to the Mirage Network stack. You need at least to consider raw ethernet sockets for linux, bpf for MacOSX and find some nice
abstraction to expose this through the Manager(?) API.

Ok, just read about raw ethernet sockets. I did not know that it was possib= le, it is interesting. The possibility I was thinking about was to "in= tegrate an existing network interface" in creating a tap interface and= bridge it with the real network interface. This way you probably can read = inbound/outbound ethernet frames. Would it be possible to do that on MacOSX= ?

Could you please point me to some documentation on how to do that on Macs ?=

Short is that there is no bridgi= ng functionality on MacOSX. Secondly, I am not sure but in linux I think th= at the bridge module is a learning bridge module, so if the kernel module l= earns your mac address then you might not receive any traffic there on. The= re is I think though a small trick to set the bridge interface into promisc= uous mode, but I haven't tested it yet.=C2=A0

Sorry, this makes = little sense now that I am reading it again. The example I am thinking of i= s a user level network bridge that I am working on and this approach I thin= k will make my life a bit difficult. For normal mirage apps the bridging ap= proach should be ok.=C2=A0
=C2=A0
=C2=A0

Thank you very much for the explanation.

Cheers,

Vincent





<= /div>--
Charalampos Rotsos
PhD student
The University of Cambridg= e
Computer Laboratory
William Gates Building
JJ Thomson Avenue
Cambridge
CB3 0FD

Phone: +44-(0) 1223 767032
Email: cr409@cl.cam.ac.uk



--
Charalampos = Rotsos
PhD student
The University of Cambridge
Computer Laboratory=
William Gates Building
JJ Thomson Avenue
Cambridge
CB3 0FD

Phone: +44-(0) 1223 767032
Email: cr409@cl.cam.ac.uk
--e89a8fb1f8d05c00fa04d885e56f-- From vb@luminar.eu.org Fri Mar 22 16:27:59 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ4ox-0006Fq-1N (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:27:59 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.5 from SpamAssassin-3.3.2-1459191 * -2.5 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:57503) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UJ4ow-0004IQ-S6 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:27:58 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id 642A4C0ACC for ; Fri, 22 Mar 2013 17:27:58 +0100 (CET) Message-ID: <514C868E.4050302@luminar.eu.org> Date: Fri, 22 Mar 2013 16:27:58 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Net.Manager References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <7FB7524C-3253-4C2D-BB98-CAB308355F83@recoil.org> In-Reply-To: <7FB7524C-3253-4C2D-BB98-CAB308355F83@recoil.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 16:27:59 -0000 Content-Length: 914 Lines: 22 On 22/03/2013 16:18, Anil Madhavapeddy wrote: > However, the interface that involves opening an fd (through whatever means, be it tuntap or bpf) and passing it through to the Mirage application via sendfd seems to be the most robust solution in terms of interface design. We could even write a manual MacOS bridge to send traffic back-and-forth from an interface if we're really stuck later. > > -anil Yeah. I just fixed the ocaml-fd-send-recv library on MacOSX (minor changes), pull request in djs55's branch. Next step: make ocaml-tuntap compile on MacOSX. i.e. fixing obuild for MacOSX. I'll be working on that, I think improving obuild is crucial for many of Citrix/Mirari/Mirage work anyway. Once that it is done, I'll push my changes to mirage-net such that there is no _creation_ of tuntap interface there, just listening to a socket and getting a tap passed as a fd by mirari. Cheers, Vincent From scott.dj@gmail.com Fri Mar 22 16:44:14 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ54g-0006mL-GZ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:44:14 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1459191 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.210.46 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-da0-f46.google.com ([209.85.210.46]:41084) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UJ54f-0002cM-Su (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:44:14 +0000 Received: by mail-da0-f46.google.com with SMTP id y19so2373882dan.19 for ; Fri, 22 Mar 2013 09:44:13 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.68.201.194 with SMTP id kc2mr3677290pbc.27.1363970652979; Fri, 22 Mar 2013 09:44:12 -0700 (PDT) Received: by 10.70.40.205 with HTTP; Fri, 22 Mar 2013 09:44:12 -0700 (PDT) In-Reply-To: <514C868E.4050302@luminar.eu.org> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <7FB7524C-3253-4C2D-BB98-CAB308355F83@recoil.org> <514C868E.4050302@luminar.eu.org> Date: Fri, 22 Mar 2013 16:44:12 +0000 Message-ID: Subject: Re: Net.Manager From: David Scott To: Vincent Bernardoff Content-Type: multipart/alternative; boundary=e89a8fb208a241493604d8862a82 Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 16:44:14 -0000 Content-Length: 3152 Lines: 93 --e89a8fb208a241493604d8862a82 Content-Type: text/plain; charset=ISO-8859-1 On Fri, Mar 22, 2013 at 4:27 PM, Vincent Bernardoff wrote: > On 22/03/2013 16:18, Anil Madhavapeddy wrote: > >> However, the interface that involves opening an fd (through whatever >> means, be it tuntap or bpf) and passing it through to the Mirage >> application via sendfd seems to be the most robust solution in terms of >> interface design. We could even write a manual MacOS bridge to send >> traffic back-and-forth from an interface if we're really stuck later. >> >> -anil >> > > Yeah. > > I just fixed the ocaml-fd-send-recv library on MacOSX (minor changes), > pull request in djs55's branch. > Thanks -- I've merged this. > > Next step: make ocaml-tuntap compile on MacOSX. i.e. fixing obuild for > MacOSX. I'll be working on that, I think improving obuild is crucial for > many of Citrix/Mirari/Mirage work anyway. > > Once that it is done, I'll push my changes to mirage-net such that there > is no _creation_ of tuntap interface there, just listening to a socket and > getting a tap passed as a fd by mirari. > > Cheers, > > Vincent > > -- Dave Scott --e89a8fb208a241493604d8862a82 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable



On Fri, Mar 22, 2013 at 4:27 PM, Vincent Bernardoff <vb@luminar.eu= .org> wrote:
On 22/03/2013 16:18, Anil = Madhavapeddy wrote:
However, the interface that involves opening an fd (through whatever means,= be it tuntap or bpf) and passing it through to the Mirage application via = sendfd seems to be the most robust solution in terms of interface design. = =A0We could even write a manual MacOS bridge to send traffic back-and-forth= from an interface if we're really stuck later.

-anil

Yeah.

I just fixed the ocaml-fd-send-recv library on MacOSX (minor changes), pull= request in djs55's branch.

T= hanks -- I've merged this.
=A0

Next step: make ocaml-tuntap compile on MacOSX. i.e. fixing obuild for MacO= SX. I'll be working on that, I think improving obuild is crucial for ma= ny of Citrix/Mirari/Mirage work anyway.

Once that it is done, I'll push my changes to mirage-net such that ther= e is no _creation_ of tuntap interface there, just listening to a socket an= d getting a tap passed as a fd by mirari.

Cheers,

Vincent




--
Dave Scott
--e89a8fb208a241493604d8862a82-- From anil@recoil.org Fri Mar 22 16:47:44 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ583-0006ri-WC (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:47:44 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1459191 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:18663 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UJ583-0007xM-YI (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:47:43 +0000 Received: (qmail 13565 invoked by uid 634); 22 Mar 2013 16:47:43 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.97]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 22 Mar 2013 16:47:43 +0000 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: Net.Manager From: Anil Madhavapeddy In-Reply-To: Date: Fri, 22 Mar 2013 16:47:42 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <2816B5B7-92E3-4936-B6AC-EEBAF92532EA@recoil.org> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <7FB7524C-3253-4C2D-BB98-CAB308355F83@recoil.org> <514C868E.4050302@luminar.eu.org> To: David Scott X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 16:47:44 -0000 Content-Length: 382 Lines: 14 On 22 Mar 2013, at 16:44, David Scott wrote: >=20 > I just fixed the ocaml-fd-send-recv library on MacOSX (minor changes), = pull request in djs55's branch. >=20 > Thanks -- I've merged this. > =20 Is this library suitable for inclusion in OPAM stable now? Might as well = get it into mainline testing if the rest of Mirage is going to depend on = it... -a= From scott.dj@gmail.com Fri Mar 22 16:59:49 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ5Jl-000792-4d (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:59:49 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.8 from SpamAssassin-3.3.2-1459191 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.160.52 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-pb0-f52.google.com ([209.85.160.52]:47990) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UJ5Jk-0008VT-Ri (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 16:59:49 +0000 Received: by mail-pb0-f52.google.com with SMTP id ma3so3218471pbc.11 for ; Fri, 22 Mar 2013 09:59:47 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.68.211.37 with SMTP id mz5mr3840487pbc.83.1363971587645; Fri, 22 Mar 2013 09:59:47 -0700 (PDT) Received: by 10.70.40.205 with HTTP; Fri, 22 Mar 2013 09:59:47 -0700 (PDT) In-Reply-To: <2816B5B7-92E3-4936-B6AC-EEBAF92532EA@recoil.org> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <7FB7524C-3253-4C2D-BB98-CAB308355F83@recoil.org> <514C868E.4050302@luminar.eu.org> <2816B5B7-92E3-4936-B6AC-EEBAF92532EA@recoil.org> Date: Fri, 22 Mar 2013 16:59:47 +0000 Message-ID: Subject: Re: Net.Manager From: David Scott To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=e89a8ff1c136f7279f04d8866184 Cc: Mirage List , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 16:59:49 -0000 Content-Length: 1964 Lines: 60 --e89a8ff1c136f7279f04d8866184 Content-Type: text/plain; charset=ISO-8859-1 That sounds like a good idea. I'll make a pull request to opam-repository later (unless someone beats me to it) On Fri, Mar 22, 2013 at 4:47 PM, Anil Madhavapeddy wrote: > On 22 Mar 2013, at 16:44, David Scott wrote: > > > > I just fixed the ocaml-fd-send-recv library on MacOSX (minor changes), > pull request in djs55's branch. > > > > Thanks -- I've merged this. > > > > Is this library suitable for inclusion in OPAM stable now? Might as well > get it into mainline testing if the rest of Mirage is going to depend on > it... > > -a -- Dave Scott --e89a8ff1c136f7279f04d8866184 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
That sounds like a good idea. I'll make a pull request= to opam-repository later (unless someone beats me to it)


On Fri, Mar 22, 2013 at= 4:47 PM, Anil Madhavapeddy <anil@recoil.org> wrote:
On 22 Mar 2013, at 16:44, = David Scott <scott.dj@gmail.com> wrote:
>
> I just fixed the ocaml-fd-send-recv library on MacOSX (minor changes),= pull request in djs55's branch.
>
> Thanks -- I've merged this.
>

Is this library suitable for inclusion in OPAM stable now? Might as w= ell get it into mainline testing if the rest of Mirage is going to depend o= n it...

-a



-- =
Dave Scott
--e89a8ff1c136f7279f04d8866184-- From Richard.Mortier@nottingham.ac.uk Fri Mar 22 17:13:49 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UJ5XJ-0007Rz-A0 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 17:13:49 +0000 X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1459191 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine02-20433-2.icritical.com ([195.62.217.151]:58150) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1UJ5XI-0000Nm-7k (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 22 Mar 2013 17:13:49 +0000 Received: (qmail 7337 invoked from network); 22 Mar 2013 17:12:47 -0000 Received: from localhost (127.0.0.1) by engine02-20433-2.icritical.com with SMTP; 22 Mar 2013 17:12:47 -0000 Received: from engine02-20433-2.icritical.com ([127.0.0.1]) by localhost (engine02-20433-2.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 07075-03 for ; Fri, 22 Mar 2013 17:12:45 +0000 (GMT) Received: (qmail 6297 invoked by uid 599); 22 Mar 2013 17:12:05 -0000 Received: from unknown (HELO smtp3.nottingham.ac.uk) (128.243.44.55) by engine02-20433-2.icritical.com (qpsmtpd/0.28) with ESMTP; Fri, 22 Mar 2013 17:12:05 +0000 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp3.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1UJ5VL-0007Pg-Uc for cl-mirage@lists.cam.ac.uk; Fri, 22 Mar 2013 17:11:47 +0000 From: Richard Mortier To: Anil Madhavapeddy Date: Fri, 22 Mar 2013 17:11:47 +0000 Subject: Re: Net.Manager Thread-Topic: Net.Manager Thread-Index: Ac4nIFW2iM/IgjflRsC7FMjyrKwzKA== Message-ID: <1011EFC0-657C-41CA-8BD8-B9EB4D01EEAB@nottingham.ac.uk> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <7FB7524C-3253-4C2D-BB98-CAB308355F83@recoil.org> In-Reply-To: <7FB7524C-3253-4C2D-BB98-CAB308355F83@recoil.org> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine02-20433-2.icritical.com Cc: "cl-mirage@lists.cam.ac.uk" , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 17:13:49 -0000 Content-Length: 1648 Lines: 35 On 22 Mar 2013, at 16:18, Anil Madhavapeddy wrote: > However, VirtualBox *does* have a kernel bridging module on MacOS X, so c= ould we perhaps use that somehow? quite possibly - i recall at one point with the openflow controller stuff t= hat i tested it running the mirage openflow controller on osx, controlling = openvswitch switches running on linux inside virtualbox on the same host. i= irc there were some magic osx runes to get this to work, but it did, eventu= ally. i have notes somewhere if useful?=20 > However, the interface that involves opening an fd (through whatever mean= s, be it tuntap or bpf) and passing it through to the Mirage application vi= a sendfd seems to be the most robust solution in terms of interface design.= We could even write a manual MacOS bridge to send traffic back-and-forth = from an interface if we're really stuck later. agree. --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From anil@recoil.org Mon Mar 25 16:44:43 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKAVn-0005AJ-BJ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 25 Mar 2013 16:44:43 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1460285 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:10963 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UKAVm-0002F6-qw (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 25 Mar 2013 16:44:43 +0000 Received: (qmail 15024 invoked by uid 634); 25 Mar 2013 16:44:42 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Mon, 25 Mar 2013 16:44:41 +0000 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: verified TLS implementation in F#/F7 Message-Id: Date: Mon, 25 Mar 2013 16:44:50 +0000 To: Mirage List Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Mar 2013 16:44:43 -0000 Content-Length: 419 Lines: 12 Verified TLS in F#, using the refinement types work from MSR Cambridge; http://mitls.rocq.inria.fr tech report: http://mitls.rocq.inria.fr/downloads/miTLS-report.pdf There are a number of caveats regarding performance (e.g. byte arrays = are represented as functional strings, so performance involves huge = amounts of copying), but the core work is extremely nice and you can see = how to make it faster... -anil= From vb@luminar.eu.org Wed Mar 27 11:15:28 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKoKG-0005Xr-1y (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:15:28 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.3 from SpamAssassin-3.3.2-1461022 * -1.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:47035) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UKoKF-0003Ak-YY (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:15:27 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id 4F957C0471 for ; Wed, 27 Mar 2013 12:15:27 +0100 (CET) Message-ID: <5152D4CF.4030002@luminar.eu.org> Date: Wed, 27 Mar 2013 11:15:27 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Net.Manager References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 11:15:28 -0000 Content-Length: 1130 Lines: 33 On 22/03/2013 16:24, Haris Rotsos wrote: > Sorry, this makes little sense now that I am reading it again. The > example I am thinking of is a user level network bridge that I am > working on and this approach I think will make my life a bit difficult. > For normal mirage apps the bridging approach should be ok. Hi, Now it is me that understand what you need. I was really confused about=20 that surprisingly, but now it makes sense. To be really clear, we were talking about two different problems: 1. To handle the creation of virtual ethernet interfaces (TAP) for=20 Mirage. In Linux, it is possible to create such devices or attach to=20 them if they are persistent. In Mac, the only option is to create them. 2. To sniff frames from existing real network interfaces. Indeed,=20 standard linux bridging is probably not providing this feature. For=20 that, there is Harris code in Mirage for Macs, and nothing for Linux=20 yet. For Linux, possibilities include the raw ethernet socket, or=20 bridging+promiscuous mode. I=E2=80=99m currently only working on problem 1. I=E2=80=99ll keep you updated Cheers, Vincent From vb@luminar.eu.org Wed Mar 27 11:22:49 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKoRN-00065g-JC (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:22:49 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.3 from SpamAssassin-3.3.2-1461022 * -1.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:38966) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UKoRN-0008KX-QY (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:22:49 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id E2C9DC0471 for ; Wed, 27 Mar 2013 12:22:48 +0100 (CET) Message-ID: <5152D688.3080706@luminar.eu.org> Date: Wed, 27 Mar 2013 11:22:48 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Net.Manager References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> In-Reply-To: <5152D4CF.4030002@luminar.eu.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 11:22:49 -0000 Content-Length: 554 Lines: 14 On 27/03/2013 11:15, Vincent Bernardoff wrote: > Sorry, this makes little sense now that I am reading it again. The > example I am thinking of is a user level network bridge that I am > working on and this approach I think will make my life a bit difficult. > For normal mirage apps the bridging approach should be ok. For your work, the best would probably to create an ocaml library (like ocaml-tuntap) but devoted to attach real network interfaces (using your code for Mac and raw ethernet socket for Linux -- which I could do). Cheers, Vincent From vb@luminar.eu.org Wed Mar 27 11:25:28 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKoTw-0006Bv-1t (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:25:28 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.3 from SpamAssassin-3.3.2-1461022 * -1.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:39161) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UKoTv-0000r1-Rl (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:25:27 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id 73997C0471 for ; Wed, 27 Mar 2013 12:25:27 +0100 (CET) Message-ID: <5152D727.9000500@luminar.eu.org> Date: Wed, 27 Mar 2013 11:25:27 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Net.Manager References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> <5152D688.3080706@luminar.eu.org> In-Reply-To: <5152D688.3080706@luminar.eu.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 11:25:28 -0000 Content-Length: 552 Lines: 14 On 27/03/2013 11:22, Vincent Bernardoff wrote: > For your work, the best would probably to create an ocaml library (like > ocaml-tuntap) but devoted to attach real network interfaces (using your > code for Mac and raw ethernet socket for Linux -- which I could do). > > Cheers, > > Vincent I mean, I could create this library if you need the functionality for Linux (it would not be much work normally). It was just very confusing to mix the TAP things with the frame sniffing things in the same file with nearly identical names (plug vs attach). From anil@recoil.org Wed Mar 27 11:32:56 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKobA-0006X1-Gw (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:32:56 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461022 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:29921 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UKobA-00021j-WZ (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:32:56 +0000 Received: (qmail 20548 invoked by uid 634); 27 Mar 2013 11:32:55 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 27 Mar 2013 11:32:55 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Net.Manager From: Anil Madhavapeddy In-Reply-To: <5152D727.9000500@luminar.eu.org> Date: Wed, 27 Mar 2013 11:32:53 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> <5152D688.3080706@luminar.eu.org> <5152D727.9000500@luminar.eu.org> To: Vincent Bernardoff X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 11:32:56 -0000 Content-Length: 817 Lines: 25 On 27 Mar 2013, at 11:25, Vincent Bernardoff wrote: > On 27/03/2013 11:22, Vincent Bernardoff wrote: >> For your work, the best would probably to create an ocaml library = (like >> ocaml-tuntap) but devoted to attach real network interfaces (using = your >> code for Mac and raw ethernet socket for Linux -- which I could do). >>=20 >> Cheers, >>=20 >> Vincent >=20 > I mean, I could create this library if you need the functionality for = Linux (it would not be much work normally). It was just very confusing = to mix the TAP things with the frame sniffing things in the same file = with nearly identical names (plug vs attach). Yeah, separating these concerns definitely makes sense. It's also a = portability maze, so small libraries make it easier to put the = platform-hacks in. -anil From h.rotsos@gmail.com Wed Mar 27 11:34:23 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKocZ-0006cG-IF (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:34:23 +0000 X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1461022 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.219.50 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (h.rotsos[at]gmail.com) * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-oa0-f50.google.com ([209.85.219.50]:48500) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UKocX-000730-8K (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:34:23 +0000 Received: by mail-oa0-f50.google.com with SMTP id n1so5901957oag.9 for ; Wed, 27 Mar 2013 04:34:20 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.60.171.102 with SMTP id at6mr15622431oec.60.1364384060077; Wed, 27 Mar 2013 04:34:20 -0700 (PDT) Sender: h.rotsos@gmail.com Received: by 10.60.24.231 with HTTP; Wed, 27 Mar 2013 04:34:19 -0700 (PDT) In-Reply-To: <5152D4CF.4030002@luminar.eu.org> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> Date: Wed, 27 Mar 2013 11:34:19 +0000 X-Google-Sender-Auth: ED0FJWugm8DVeaEtokpGPu6CNJg Message-ID: Subject: Re: Net.Manager From: Haris Rotsos To: Vincent Bernardoff Content-Type: multipart/alternative; boundary=bcaec54ee0943cfc4c04d8e66b15 Cc: "cl-mirage@lists.cam.ac.uk Mailing List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 11:34:23 -0000 Content-Length: 5837 Lines: 150 --bcaec54ee0943cfc4c04d8e66b15 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 27 March 2013 11:15, Vincent Bernardoff wrote: > On 22/03/2013 16:24, Haris Rotsos wrote: > >> Sorry, this makes little sense now that I am reading it again. The >> example I am thinking of is a user level network bridge that I am >> working on and this approach I think will make my life a bit difficult. >> For normal mirage apps the bridging approach should be ok. >> > > Hi, > > Now it is me that understand what you need. I was really confused about > that surprisingly, but now it makes sense. > Sorry if I made things more complicated than they really are. > > To be really clear, we were talking about two different problems: > > 1. To handle the creation of virtual ethernet interfaces (TAP) for Mirage= . > In Linux, it is possible to create such devices or attach to them if they > are persistent. In Mac, the only option is to create them. > > 2. To sniff frames from existing real network interfaces. Indeed, standar= d > linux bridging is probably not providing this feature. For that, there is > Harris code in Mirage for Macs, and nothing for Linux yet. For Linux, > possibilities include the raw ethernet socket, or bridging+promiscuous mo= de. > A quick and simple solution for this one I think to have a mechanic that will tell us in the ocaml level whether the underline architecture is Linux or Darwin. For Darwin you need to read a frame, remove the first bytes which contain the pcap header and then push it up the stack. For Linux using raw ethernet socket you need to have the same code without the pcap header triming functionality (and add a bit of c code that will open the raw socket fd). Would it be ok to have a prepocessor command together with an option on the compiler? I know that the caml4p already has the Pa_macro extension that exposes some form of prepocessor commands. You only need later to expose a ppopt parameter to the compiler. This technique is already used in the code of the tap_stubs. > > > > I=E2=80=99m currently only working on problem 1. > > I=E2=80=99ll keep you updated > > Cheers, > > Vincent > > --=20 Charalampos Rotsos PhD student The University of Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD Phone: +44-(0) 1223 767032 Email: cr409@cl.cam.ac.uk --bcaec54ee0943cfc4c04d8e66b15 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable



On 27 March 2013 11:15, Vincent Bernardoff <vb@luminar.eu.org&= gt; wrote:
On 22/03/2013 16:24, Haris= Rotsos wrote:
Sorry, this makes little sense now that I am reading it again. The
example I am thinking of is a user level network bridge that I am
working on and this approach I think will make my life a bit difficult.
For normal mirage apps the bridging approach should be ok.

Hi,

Now it is me that understand what you need. I was really confused about tha= t surprisingly, but now it makes sense.

Sorry if I made things more complicated than they really are.=C2=A0<= /div>
=C2=A0

To be really clear, we were talking about two different problems:

1. To handle the creation of virtual ethernet interfaces (TAP) for Mirage. = In Linux, it is possible to create such devices or attach to them if they a= re persistent. In Mac, the only option is to create them.

2. To sniff frames from existing real network interfaces. Indeed, standard = linux bridging is probably not providing this feature. For that, there is H= arris code in Mirage for Macs, and nothing for Linux yet. For Linux, possib= ilities include the raw ethernet socket, or bridging+promiscuous mode.

A quick and simple solution for this= one I think to have a mechanic that will tell us in the ocaml level whethe= r the underline architecture is Linux or Darwin. For Darwin you need to rea= d a frame, remove the first bytes which contain the pcap header and then pu= sh it up the stack. For Linux using raw ethernet socket you need to have th= e same code without the pcap header triming functionality (and add a bit of= c code that will open the raw socket fd). Would it be ok to have a prepoce= ssor command together with an option on the compiler? I know that the caml4= p already has the Pa_macro extension that exposes some form of prepocessor = commands. You only need later to expose a ppopt parameter to the compiler. = This technique is already used in the code of the tap_stubs.=C2=A0
=C2=A0



I=E2=80=99m currently only working on problem 1.

I=E2=80=99ll keep you updated

Cheers,

Vincent




--
Charalampos = Rotsos
PhD student
The University of Cambridge
Computer Laboratory=
William Gates Building
JJ Thomson Avenue
Cambridge
CB3 0FD

Phone: +44-(0) 1223 767032
Email: cr409@cl.cam.ac.uk
--bcaec54ee0943cfc4c04d8e66b15-- From h.rotsos@gmail.com Wed Mar 27 11:36:33 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKoef-0006jT-BI (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:36:33 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1461022 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.214.175 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (h.rotsos[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ob0-f175.google.com ([209.85.214.175]:63849) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UKoed-0003XM-Yv (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:36:33 +0000 Received: by mail-ob0-f175.google.com with SMTP id va7so5005166obc.34 for ; Wed, 27 Mar 2013 04:36:31 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.182.164.8 with SMTP id ym8mr3654925obb.68.1364384190954; Wed, 27 Mar 2013 04:36:30 -0700 (PDT) Sender: h.rotsos@gmail.com Received: by 10.60.24.231 with HTTP; Wed, 27 Mar 2013 04:36:30 -0700 (PDT) In-Reply-To: References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> <5152D688.3080706@luminar.eu.org> <5152D727.9000500@luminar.eu.org> Date: Wed, 27 Mar 2013 11:36:30 +0000 X-Google-Sender-Auth: DmJ3LPJnZ706cByFOOuXgjly9WM Message-ID: Subject: Re: Net.Manager From: Haris Rotsos To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=e89a8f646a070a8e8904d8e6739e Cc: "cl-mirage@lists.cam.ac.uk Mailing List" , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 11:36:33 -0000 Content-Length: 3518 Lines: 100 --e89a8f646a070a8e8904d8e6739e Content-Type: text/plain; charset=UTF-8 sorry replied too late to the first mail. Ok tell me when the code is ready for the tap case to see the resulting API and we can see then how we can fit something in a more modular way. On 27 March 2013 11:32, Anil Madhavapeddy wrote: > On 27 Mar 2013, at 11:25, Vincent Bernardoff wrote: > > > On 27/03/2013 11:22, Vincent Bernardoff wrote: > >> For your work, the best would probably to create an ocaml library (like > >> ocaml-tuntap) but devoted to attach real network interfaces (using your > >> code for Mac and raw ethernet socket for Linux -- which I could do). > >> > >> Cheers, > >> > >> Vincent > > > > I mean, I could create this library if you need the functionality for > Linux (it would not be much work normally). It was just very confusing to > mix the TAP things with the frame sniffing things in the same file with > nearly identical names (plug vs attach). > > Yeah, separating these concerns definitely makes sense. It's also a > portability maze, so small libraries make it easier to put the > platform-hacks in. > > -anil > > > -- Charalampos Rotsos PhD student The University of Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD Phone: +44-(0) 1223 767032 Email: cr409@cl.cam.ac.uk --e89a8f646a070a8e8904d8e6739e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
sorry replied too late to the first mail. Ok tell me when = the code is ready for the tap case to see the resulting API and we can see = then how we can fit something in a more modular way.=C2=A0


On 27 March 2013 11:32, Anil Madhavapedd= y <anil@recoil.org> wrote:
On 27 Mar 2013, at 11:25, Vincent B= ernardoff <vb@luminar.eu.org>= ; wrote:

> On 27/03/2013 11:22, Vincent Bernardoff wrote:
>> For your work, the best would probably to create an ocaml library = (like
>> ocaml-tuntap) but devoted to attach real network interfaces (using= your
>> code for Mac and raw ethernet socket for Linux -- which I could do= ).
>>
>> Cheers,
>>
>> Vincent
>
> I mean, I could create this library if you need the functionality for = Linux (it would not be much work normally). It was just very confusing to m= ix the TAP things with the frame sniffing things in the same file with near= ly identical names (plug vs attach).

Yeah, separating these concerns definitely makes sense. =C2=A0I= t's also a portability maze, so small libraries make it easier to put t= he platform-hacks in.

-anil





-- Charalampos Rotsos
PhD student
The University of Cambridge
Compu= ter Laboratory
William Gates Building
JJ Thomson Avenue
Cambridge<= br> CB3 0FD

Phone: +44-(0) 1223 767032
Email: cr409@cl.cam.ac.uk
--e89a8f646a070a8e8904d8e6739e-- From vb@luminar.eu.org Wed Mar 27 11:44:39 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKomV-0007HS-Db (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:44:39 +0000 X-Cam-SpamDetails: score -1.4 from SpamAssassin-3.3.2-1461022 * -1.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:33461) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UKomT-0002v6-7w (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:44:39 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id 3539FBF5A4 for ; Wed, 27 Mar 2013 12:44:37 +0100 (CET) Message-ID: <5152DBA5.9010104@luminar.eu.org> Date: Wed, 27 Mar 2013 11:44:37 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Net.Manager References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> <5152D688.3080706@luminar.eu.org> <5152D727.9000500@luminar.eu.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 11:44:39 -0000 Content-Length: 576 Lines: 19 On 27/03/2013 11:36, Haris Rotsos wrote: > sorry replied too late to the first mail. Ok tell me when the code is > ready for the tap case to see the resulting API and we can see then how > we can fit something in a more modular way. > For now I made that: https://github.com/vbmithr/ocaml-tuntap This is a Linux/Mac library that covers only creating/attaching tun/tap interfaces, extracted out of the mirage tab stubs and completed for Linux. Regarding the code in mirage/platform/.../netif, I’m still thinking about it and will give a proposal soon. Cheers Vincent From anil@recoil.org Wed Mar 27 11:46:07 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKonv-0007Lb-0B (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:46:07 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461022 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:32303 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UKonu-0008Ax-Xg (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 11:46:06 +0000 Received: (qmail 21006 invoked by uid 634); 27 Mar 2013 11:46:06 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 27 Mar 2013 11:46:05 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Net.Manager From: Anil Madhavapeddy In-Reply-To: <5152DBA5.9010104@luminar.eu.org> Date: Wed, 27 Mar 2013 11:46:03 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <9C4CC087-F27E-4281-8A1F-48036FE6FA49@recoil.org> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> <5152D688.3080706@luminar.eu.org> <5152D727.9000500@luminar.eu.org> <5152DBA5.9010104@luminar.eu.org> To: Vincent Bernardoff , Richard Mortier X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 11:46:07 -0000 Content-Length: 635 Lines: 22 On 27 Mar 2013, at 11:44, Vincent Bernardoff wrote: > On 27/03/2013 11:36, Haris Rotsos wrote: >> sorry replied too late to the first mail. Ok tell me when the code is >> ready for the tap case to see the resulting API and we can see then = how >> we can fit something in a more modular way. >>=20 >=20 > For now I made that: > https://github.com/vbmithr/ocaml-tuntap >=20 > This is a Linux/Mac library that covers only creating/attaching = tun/tap interfaces, extracted out of the mirage tab stubs and completed = for Linux. Mort: does this fix your weird Linux routing problem with the old tap = code? -anil From crowcroft@gmail.com Wed Mar 27 17:18:28 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKtzY-0003sm-1b (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 17:18:28 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.6 from SpamAssassin-3.3.2-1461022 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.83.50 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (crowcroft[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ee0-f50.google.com ([74.125.83.50]:58974) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UKtzX-0004OR-R9 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 17:18:27 +0000 Received: by mail-ee0-f50.google.com with SMTP id e53so1077484eek.37 for ; Wed, 27 Mar 2013 10:18:27 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.14.111.72 with SMTP id v48mr57943159eeg.11.1364404707166; Wed, 27 Mar 2013 10:18:27 -0700 (PDT) Sender: crowcroft@gmail.com Received: by 10.223.82.67 with HTTP; Wed, 27 Mar 2013 10:18:26 -0700 (PDT) Received: by 10.223.82.67 with HTTP; Wed, 27 Mar 2013 10:18:26 -0700 (PDT) In-Reply-To: <9C4CC087-F27E-4281-8A1F-48036FE6FA49@recoil.org> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> <5152D688.3080706@luminar.eu.org> <5152D727.9000500@luminar.eu.org> <5152DBA5.9010104@luminar.eu.org> <9C4CC087-F27E-4281-8A1F-48036FE6FA49@recoil.org> Date: Wed, 27 Mar 2013 18:18:26 +0100 X-Google-Sender-Auth: m3tTi1zrwdNDN6UQXneqG2KCUwc Message-ID: Subject: Re: Net.Manager From: Jon Crowcroft To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=089e016815fce6b16004d8eb394a Cc: Richard Mortier , "cl-mirage@lists.cam.ac.uk List" , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 17:18:28 -0000 Content-Length: 2342 Lines: 67 --089e016815fce6b16004d8eb394a Content-Type: text/plain; charset=ISO-8859-1 Might be worth asking Malcolm Scott about this.his moose/xen stuff should have taken some view on it.. On 27 Mar 2013 12:46, "Anil Madhavapeddy" wrote: > On 27 Mar 2013, at 11:44, Vincent Bernardoff wrote: > > > On 27/03/2013 11:36, Haris Rotsos wrote: > >> sorry replied too late to the first mail. Ok tell me when the code is > >> ready for the tap case to see the resulting API and we can see then how > >> we can fit something in a more modular way. > >> > > > > For now I made that: > > https://github.com/vbmithr/ocaml-tuntap > > > > This is a Linux/Mac library that covers only creating/attaching tun/tap > interfaces, extracted out of the mirage tab stubs and completed for Linux. > > Mort: does this fix your weird Linux routing problem with the old tap code? > > -anil > > > --089e016815fce6b16004d8eb394a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

Might be worth asking Malcolm Scott about this.his moose/xen stuff shoul= d have taken some view on it..

On 27 Mar 2013 12:46, "Anil Madhavapeddy&qu= ot; <anil@recoil.org> wrote:
On 27 Mar 2013, at 11:44, Vincent Bernardoff <vb@luminar.eu.org> wrote:

> On 27/03/2013 11:36, Haris Rotsos wrote:
>> sorry replied too late to the first mail. Ok tell me when the code= is
>> ready for the tap case to see the resulting API and we can see the= n how
>> we can fit something in a more modular way.
>>
>
> For now I made that:
> = https://github.com/vbmithr/ocaml-tuntap
>
> This is a Linux/Mac library that covers only creating/attaching tun/ta= p interfaces, extracted out of the mirage tab stubs and completed for Linux= .

Mort: does this fix your weird Linux routing problem with the old tap code?=

-anil


--089e016815fce6b16004d8eb394a-- From Richard.Mortier@nottingham.ac.uk Wed Mar 27 17:32:22 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKuD0-0004B4-Jf (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 17:32:22 +0000 X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1461022 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine04-20433-4.icritical.com ([195.62.217.152]:35709) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1UKuCz-0004bI-8j (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 17:32:22 +0000 Received: (qmail 10962 invoked from network); 27 Mar 2013 17:32:07 -0000 Received: from localhost (127.0.0.1) by engine04-20433-4.icritical.com with SMTP; 27 Mar 2013 17:32:07 -0000 Received: from engine04-20433-4.icritical.com ([127.0.0.1]) by localhost (engine04-20433-4.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 10770-07 for ; Wed, 27 Mar 2013 17:32:05 +0000 (GMT) Received: (qmail 10006 invoked by uid 599); 27 Mar 2013 17:31:15 -0000 Received: from unknown (HELO smtp4.nottingham.ac.uk) (128.243.220.65) by engine04-20433-4.icritical.com (qpsmtpd/0.28) with ESMTP; Wed, 27 Mar 2013 17:31:15 +0000 Received: from uiwexhub01.ad.nottingham.ac.uk ([128.243.15.133]) by smtp4.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1UKuC5-0003c1-BI for cl-mirage@lists.cam.ac.uk; Wed, 27 Mar 2013 17:31:25 +0000 From: Richard Mortier To: Anil Madhavapeddy Date: Wed, 27 Mar 2013 17:31:24 +0000 Subject: Re: Net.Manager Thread-Topic: Net.Manager Thread-Index: Ac4rEOeZ40k+jCQFTDyWBSSLh0/BlQ== Message-ID: References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> <5152D688.3080706@luminar.eu.org> <5152D727.9000500@luminar.eu.org> <5152DBA5.9010104@luminar.eu.org> <9C4CC087-F27E-4281-8A1F-48036FE6FA49@recoil.org> In-Reply-To: <9C4CC087-F27E-4281-8A1F-48036FE6FA49@recoil.org> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine04-20433-4.icritical.com Cc: "cl-mirage@lists.cam.ac.uk List" , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 17:32:22 -0000 Content-Length: 1509 Lines: 38 On 27 Mar 2013, at 11:46, Anil Madhavapeddy wrote: > On 27 Mar 2013, at 11:44, Vincent Bernardoff wrote: >=20 >> On 27/03/2013 11:36, Haris Rotsos wrote: >>> sorry replied too late to the first mail. Ok tell me when the code is >>> ready for the tap case to see the resulting API and we can see then how >>> we can fit something in a more modular way. >>>=20 >>=20 >> For now I made that: >> https://github.com/vbmithr/ocaml-tuntap >>=20 >> This is a Linux/Mac library that covers only creating/attaching tun/tap = interfaces, extracted out of the mirage tab stubs and completed for Linux. >=20 > Mort: does this fix your weird Linux routing problem with the old tap cod= e? hopefully :) i'll give it a try later... --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From anil@recoil.org Wed Mar 27 19:12:35 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKvlz-0005o7-Pk (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 19:12:35 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461022 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:30095 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UKvlz-0005Vi-RD (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 19:12:35 +0000 Received: (qmail 30058 invoked by uid 634); 27 Mar 2013 19:12:34 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 27 Mar 2013 19:12:34 +0000 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: ANN: mirage-platform-0.8.1 Message-Id: <2EC43943-E2D0-4C04-96DC-B6A1F766BF3A@recoil.org> Date: Wed, 27 Mar 2013 19:12:33 +0000 To: "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 19:12:35 -0000 Content-Length: 248 Lines: 10 I've just released mirage-platform-0.8.1 with one rather important fix = that unbreaks Xen storage block devices (they were racy before). Thanks = Jon! 0.8.1 (27-Mar-2013): * [xen] Fix the Xen block interface to wait for init correctly. -anil= From anil@recoil.org Wed Mar 27 19:36:47 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKw9P-00065q-9v (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 19:36:47 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461022 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:16968 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UKw9O-0004MG-Z8 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 19:36:47 +0000 Received: (qmail 23923 invoked by uid 634); 27 Mar 2013 19:36:46 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 27 Mar 2013 19:36:45 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: [Xen-devel] Request for input: Extended event channel support From: Anil Madhavapeddy In-Reply-To: Date: Wed, 27 Mar 2013 19:36:43 +0000 Content-Transfer-Encoding: 7bit Message-Id: <8E3B77A0-C082-4F98-995E-CC1514631D49@recoil.org> References: To: George Dunlap X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: xen-users@lists.xen.org, "cl-mirage@lists.cam.ac.uk List" , xen-devel@lists.xen.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 19:36:47 -0000 Content-Length: 1357 Lines: 28 On 27 Mar 2013, at 11:23, George Dunlap wrote: > > The FIFO solution makes event delivery a matter of adding items to a > highly structured linked list. The number of event channels for the > interface design has a theoretical maximum of 2^28; the current > implementation is limimited at 2^17, which is over 100,000. The > number is the same for both 32-bit and 64-bit kernels. Is there any reason for such a low default? If I'm not mistaken, every guest needs at least 2 event channels (console, xenstore) and probably has two more for a net and disk device. With stub-domains in the mix, we could easily imagine running 25,000 VMs with a couple of megabytes of RAM each using Mirage (which can boot very low memory guests without too much trouble). This does run into other problems with CPU scheduling and device scalability, but it would be nice if any proposed event channel upgrade went well above this level rather than scrape it. I personally prefer the 4.3 solution (despite the priority hack for the timers) just because the existing limitation is so very trivial to hit. However, I have no view on the level of technical debt that would incur if it subsequently required switching to the FIFO solution in 4.4 and causing yet another round of upgrades. That's your problem; I just want the extra domains :-) -anil From anil@recoil.org Wed Mar 27 19:45:58 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKwII-0006DB-BX (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 19:45:58 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461022 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:39059 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UKwIH-0006hO-Yb (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 19:45:58 +0000 Received: (qmail 5250 invoked by uid 634); 27 Mar 2013 19:45:57 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 27 Mar 2013 19:45:57 +0000 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: the unix/xen interface for suspend/reusme Message-Id: Date: Wed, 27 Mar 2013 19:45:55 +0000 To: "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 19:45:58 -0000 Content-Length: 1044 Lines: 22 In Vincent's new UNIX model, we end up with a UNIX daemon that can = accept 'OS requests' via a control socket. The first set of inputs is = primarily to do with device hotplug (discussed here in the tuntap = discussion), but there is also another interesting one in Xen: = suspend/resume and live migration. Vincent: see this pull request from Jon to mirage-skeleton: https://github.com/mirage/mirage-skeleton/pull/2/files Suspend/resume is a little complicated in Xen since the application has = to somehow also keep an eye on the xenstore notifications, and invoke = callbacks to do something (e.g. send a gratuitous ARP on wakeup). Jon's example code is quite intrusive and probably ought to be factored = into the OS library somewhere so that applications can register interest = in such system-level events more easily. It might be easiest to test = this interface in the UNIX version by transmitting an explicit = suspend/resume across the control socket so that UNIX containers could = be put to sleep just like in Xen... -anil= From dvrabel@cantab.net Wed Mar 27 21:53:13 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKyHR-00017Y-QG (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 21:53:13 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461022 * -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no * trust * [212.23.1.1 listed in list.dnswl.dnsbl.ja.net] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from smarthost01.mail.zen.net.uk ([212.23.1.1]:42278) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UKyHQ-0002xc-R3 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 21:53:13 +0000 Received: from [82.70.146.41] (helo=pear) by smarthost01.mail.zen.net.uk with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1UKyHN-0000O3-3X; Wed, 27 Mar 2013 21:53:09 +0000 Received: from apple.davidvrabel.org.uk ([82.70.146.43]) by pear with esmtp (Exim 4.72) (envelope-from ) id 1UKyHL-0001hu-4u; Wed, 27 Mar 2013 21:53:08 +0000 Message-ID: <51536A3E.2020302@cantab.net> Date: Wed, 27 Mar 2013 21:53:02 +0000 From: David Vrabel User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130307 Thunderbird/17.0.4 MIME-Version: 1.0 To: Anil Madhavapeddy References: <8E3B77A0-C082-4F98-995E-CC1514631D49@recoil.org> In-Reply-To: <8E3B77A0-C082-4F98-995E-CC1514631D49@recoil.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-SA-Exim-Connect-IP: 82.70.146.43 X-SA-Exim-Mail-From: dvrabel@cantab.net X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pear X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,SPF_NEUTRAL autolearn=no version=3.3.1 Subject: Re: [Xen-devel] Request for input: Extended event channel support X-SA-Exim-Version: 4.2.1 (built Mon, 22 Mar 2010 06:26:47 +0000) X-SA-Exim-Scanned: Yes (on pear) X-Originating-Smarthost01-IP: [82.70.146.41] Cc: xen-users@lists.xen.org, George Dunlap , "cl-mirage@lists.cam.ac.uk List" , xen-devel@lists.xen.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 21:53:13 -0000 Content-Length: 1342 Lines: 31 On 27/03/2013 19:36, Anil Madhavapeddy wrote: > On 27 Mar 2013, at 11:23, George Dunlap wrote: >> >> The FIFO solution makes event delivery a matter of adding items to a >> highly structured linked list. The number of event channels for the >> interface design has a theoretical maximum of 2^28; the current >> implementation is limimited at 2^17, which is over 100,000. The >> number is the same for both 32-bit and 64-bit kernels. > > Is there any reason for such a low default? If I'm not mistaken, > every guest needs at least 2 event channels (console, xenstore) and > probably has two more for a net and disk device. 131,072 seemed high enough to me but I'd forgotten about the Mirage use case. This can be trivially raised to 2^19 (524,288). Beyond that, the implementation becomes slightly more complex as the pointers to the event array pages no longer fit in a single page. > With stub-domains in the mix, we could easily imagine running 25,000 > VMs with a couple of megabytes of RAM each using Mirage (which can > boot very low memory guests without too much trouble). Having said that, with 25,000 VMs it would seem sensible to disaggregate things like the console and xenstore (in addition to the network and block backends). Thus reducing the need for event channels for any single domain. David From anil@recoil.org Wed Mar 27 22:28:22 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKypS-0001VN-8v (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 22:28:22 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461022 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:25356 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UKypR-0003Kh-Yu (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 22:28:22 +0000 Received: (qmail 23008 invoked by uid 634); 27 Mar 2013 22:28:21 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 27 Mar 2013 22:28:21 +0000 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: [Xen-devel] Request for input: Extended event channel support From: Anil Madhavapeddy In-Reply-To: <51536A3E.2020302@cantab.net> Date: Wed, 27 Mar 2013 22:28:18 +0000 Content-Transfer-Encoding: 7bit Message-Id: <31D0119A-0154-4454-94E0-0F9D15DC103F@recoil.org> References: <8E3B77A0-C082-4F98-995E-CC1514631D49@recoil.org> <51536A3E.2020302@cantab.net> To: David Vrabel X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: xen-users@lists.xen.org, George Dunlap , "cl-mirage@lists.cam.ac.uk List" , xen-devel@lists.xen.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 22:28:22 -0000 Content-Length: 1975 Lines: 46 On 27 Mar 2013, at 21:53, David Vrabel wrote: > On 27/03/2013 19:36, Anil Madhavapeddy wrote: >> On 27 Mar 2013, at 11:23, George Dunlap wrote: >>> >>> The FIFO solution makes event delivery a matter of adding items to a >>> highly structured linked list. The number of event channels for the >>> interface design has a theoretical maximum of 2^28; the current >>> implementation is limimited at 2^17, which is over 100,000. The >>> number is the same for both 32-bit and 64-bit kernels. >> >> Is there any reason for such a low default? If I'm not mistaken, >> every guest needs at least 2 event channels (console, xenstore) and >> probably has two more for a net and disk device. > > 131,072 seemed high enough to me but I'd forgotten about the Mirage use > case. > > This can be trivially raised to 2^19 (524,288). Beyond that, the > implementation becomes slightly more complex as the pointers to the > event array pages no longer fit in a single page. Makes sense. >> With stub-domains in the mix, we could easily imagine running 25,000 >> VMs with a couple of megabytes of RAM each using Mirage (which can >> boot very low memory guests without too much trouble). > > Having said that, with 25,000 VMs it would seem sensible to disaggregate > things like the console and xenstore (in addition to the network and > block backends). Thus reducing the need for event channels for any > single domain. Yeah indeed; this should be pretty easy to do and let the existing 2^17 be enough for a long time too. We'd need to think a bit about a distributed xenstored to avoid having one hotspot servicing so many VMs. One nice thing about the OCaml xenstored is that it should be possible to make an explicitly distributed implementation of the protocol. The data-structure is already based around immutable trees, so it's a matter of figuring out where to put the consensus logic (probably around /local/domain/*). -anil From liuw@liuw.name Wed Mar 27 22:31:58 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UKysw-0001YZ-En (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 22:31:58 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1461022 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.44 listed in list.dnswl.dnsbl.ja.net] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-wg0-f44.google.com ([74.125.82.44]:49603) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UKysv-0004Q1-Yq (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 27 Mar 2013 22:31:58 +0000 Received: by mail-wg0-f44.google.com with SMTP id z12so1013857wgg.35 for ; Wed, 27 Mar 2013 15:31:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:x-gm-message-state; bh=+QvVenXq6+d/eHCK6tI3pjrb9AzfpQjL+53MtE8ZtuU=; b=GEJnafo47iT59iYQ0R3tzUj4ZW7/JzS9V1/DhxqkV+Zw1a32PFwhjS7XqRXyT73xa1 q7yxRNLXWRpCOlqccEzji5SQfVQZhFn6iRklAVF9EiFD/OmbKAWOUJVz6/J3+9IZlD8H RWpLA111dRDD+lUej0xg2d85PXxyEBCZ3XCXhbvNBJiSULCjJkQP0SWevO7g4Z01nex9 qrMdklrJ97r6bBBQotDFZqU4TI20SIaz9jWMfbk21h1RzzNGmYeHk0+o0F1y5vprkEZj 6HZiQmo5ooWk2etAauoYaQzUHj84/K2xGZhaakMzo6TUk8pTm7VZyVSkDk7rw6tcmBbk uZJw== X-Received: by 10.194.77.110 with SMTP id r14mr34681771wjw.2.1364423517598; Wed, 27 Mar 2013 15:31:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.180.77.161 with HTTP; Wed, 27 Mar 2013 15:31:23 -0700 (PDT) In-Reply-To: <51536A3E.2020302@cantab.net> References: <8E3B77A0-C082-4F98-995E-CC1514631D49@recoil.org> <51536A3E.2020302@cantab.net> From: Wei Liu Date: Wed, 27 Mar 2013 22:31:23 +0000 Message-ID: Subject: Re: [Xen-devel] Request for input: Extended event channel support To: David Vrabel Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQnGzhBQD/Xd4dur89hrGNVm+yhQVht6t7gKNvbV1/PLyM3hjgWO7yZDPd0vgWOuHKbuOzUK Cc: xen-users@lists.xen.org, George Dunlap , "xen-devel@lists.xen.org" , "cl-mirage@lists.cam.ac.uk List" , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2013 22:31:58 -0000 Content-Length: 1781 Lines: 47 On Wed, Mar 27, 2013 at 9:53 PM, David Vrabel wrote: > On 27/03/2013 19:36, Anil Madhavapeddy wrote: >> On 27 Mar 2013, at 11:23, George Dunlap wrote: >>> >>> The FIFO solution makes event delivery a matter of adding items to a >>> highly structured linked list. The number of event channels for the >>> interface design has a theoretical maximum of 2^28; the current >>> implementation is limimited at 2^17, which is over 100,000. The >>> number is the same for both 32-bit and 64-bit kernels. >> >> Is there any reason for such a low default? If I'm not mistaken, >> every guest needs at least 2 event channels (console, xenstore) and >> probably has two more for a net and disk device. > > 131,072 seemed high enough to me but I'd forgotten about the Mirage use > case. > > This can be trivially raised to 2^19 (524,288). Beyond that, the > implementation becomes slightly more complex as the pointers to the > event array pages no longer fit in a single page. > Then that would require 512 pages mapped in Xen in the worst case, plus >> With stub-domains in the mix, we could easily imagine running 25,000 >> VMs with a couple of megabytes of RAM each using Mirage (which can >> boot very low memory guests without too much trouble). > 25,000 pages for domUs if domU uses this ABI as well. This might require bumping global mapping space in Xen, or we can restrict domU to only use default 2-level ABI to solve this problem. But let's not worry about future things for now. Wei. > Having said that, with 25,000 VMs it would seem sensible to disaggregate > things like the console and xenstore (in addition to the network and > block backends). Thus reducing the need for event channels for any > single domain. > > David > From anil@recoil.org Thu Mar 28 11:02:50 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULAba-0005u5-Dr (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 11:02:50 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461449 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:13245 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1ULAbZ-0004gW-ql (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 11:02:50 +0000 Received: (qmail 25893 invoked by uid 634); 28 Mar 2013 11:02:49 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 28 Mar 2013 11:02:48 +0000 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: [Caml-list] Master-slave architecture behind an ocsigen server. From: Anil Madhavapeddy In-Reply-To: <51540395.50202@frisch.fr> Date: Thu, 28 Mar 2013 11:02:46 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <51520CAE.6020009@ens-lyon.org> <51540395.50202@frisch.fr> To: Alain Frisch , "cl-mirage@lists.cam.ac.uk List" X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Martin Jambon , caml users , Philippe Veber X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 11:02:50 -0000 Content-Length: 4547 Lines: 93 On 28 Mar 2013, at 08:47, Alain Frisch wrote: > On 03/28/2013 08:37 AM, Philippe Veber wrote: >> Hi Martin, >> nproc meets exactly my needs: a simple lwt-friendly interface to >> dispatch function calls on a pool of processes that run on the same >> machine. I have only one concern, that should probably be discussed = on >> the ocsigen list, that is I wonder if it is okay to fork the process >> running the ocsigen server. I think I remember warnings on having = parent >> and children processes sharing connections/channels but it's really = not >> clear to me. >=20 > FWIW, LexiFi uses an architecture quite close to this for our = application. The main process manages the GUI and dispatches = computations tasks to external processes. Some points to be noted: >=20 > - Since this is a Windows application, we cannot rely on fork. = Instead, we restart the application (Sys.argv.(0)), with specific = command-line flag, captured by the library in charge of managing = computations. This is done by calling a special function in this = library; the function does nothing in the main process and in the = sub-processes, it starts the special mode and never returns. This gives = a chance to the main application to do some global initialization common = to the main and sub processes (for instance, we dynlink external plugins = in this initialization phase). >=20 > - Computation functions are registered as global values. Registration = returns an opaque handle which can be used to call such a function. We = don't rely on marshaling closures. >=20 > - The GUI process actually spawns a single sub-process (the = Scheduler), which itself manages more worker sub-sub-processes (with a = maximal number of workers). Currently, we don't do very clever = scheduling based on task priorities, but this could easily be added. >=20 > - An external computation can spawn sub-computations (by applying a = parallel "map" to a list) either synchronously (direct style) or = asynchronously (by providing a continuation function, which will be = applied to the list of results, maybe in a different process). In both = cases, this is done by sending those tasks to the Scheduler. The = Scheduler dispatches computation tasks to available workers. In the = synchronous parallel map, the caller runs an inner event loop to = communicate with the Scheduler (and it only accepts sub-tasks created by = itself or one of its descendants). >=20 > - Top-level external computations can be stopped by the main process = (e.g. on user request). Concretely, this kills all workers currently = working on that task or one of its sub-tasks. >=20 > - In addition to sending back the final results, computations can = report progress to their caller and more intermediate results. This is = useful to show a progress bar/status and partial results in the GUI = before the end of the entire computation. >=20 > - Communication between processes is done by exchanging marshaled = "variants" (a tagged representation of OCaml values, generated = automatically using our runtime types). Since we can attach special = variantizers/devariantizers to specific types, this gives a chance to = customize how some values have to be exchanged between processes (e.g. = values relying on internal hash-consing are treated specially to = recreate the maximal sharing in the sub-process). >=20 > - Concretely, the communication between processes is done through = queues of messages implemented with shared memory. (This component was = developed by Fabrice Le Fessant and OCamlPro.) Large computation = arguments or results (above a certain size) are stored on the file = system, to avoid having to keep them in RAM for too long (if all workers = are busy, the computation might wait for some time being started). Are all of the messages through these queues persistent, or just the = larger ones that are too big to fit in the shared memory segment, and = are they always point-to-point streams? We've got a similar need in Xen/Mirage for shared memory communication = and queues, and have been breaking them out into standalone libs such = as: https://github.com/djs55/shared-memory-ring ...which is ABI-compatible with the existing Xen shared memory = interfaces, and also an OCaml version of the transport-agnostic API = sketched out in: http://anil.recoil.org/papers/2012-resolve-fable.pdf The missing link currently is the persistent queuing service, but we're = investigating the options here (ocamlmq looks rather nice). -anil From Richard.Mortier@nottingham.ac.uk Thu Mar 28 11:09:48 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULAiK-00069I-1q (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 11:09:48 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461449 * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine03-20433-8.icritical.com ([93.159.202.52]:44176) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1ULAiJ-0001DH-RD (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 11:09:47 +0000 Received: (qmail 20590 invoked from network); 28 Mar 2013 11:09:45 -0000 Received: from localhost (127.0.0.1) by engine03-20433-8.icritical.com with SMTP; 28 Mar 2013 11:09:45 -0000 Received: from engine03-20433-8.icritical.com ([127.0.0.1]) by localhost (engine03-20433-8.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 20576-01 for ; Thu, 28 Mar 2013 11:09:44 +0000 (GMT) Received: (qmail 18441 invoked by uid 599); 28 Mar 2013 11:05:56 -0000 Received: from unknown (HELO smtp3.nottingham.ac.uk) (128.243.44.55) by engine03-20433-8.icritical.com (qpsmtpd/0.28) with ESMTP; Thu, 28 Mar 2013 11:05:56 +0000 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp3.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1ULAeU-00007l-3B for cl-mirage@lists.cam.ac.uk; Thu, 28 Mar 2013 11:05:50 +0000 From: Richard Mortier To: "cl-mirage@lists.cam.ac.uk List" Date: Thu, 28 Mar 2013 11:05:49 +0000 Subject: mirage + froc = self-scaling? Thread-Topic: mirage + froc = self-scaling? Thread-Index: Ac4rpDRVIIRZ6aDATT6+q2hc/1DFhg== Message-ID: <120C6BA3-69B2-4FC7-87C3-3BF7EA4AFE7F@nottingham.ac.uk> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine03-20433-8.icritical.com X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 11:09:48 -0000 Content-Length: 1395 Lines: 29 hi all; after a chat with anil at asplos, i've begun playing around with using Froc= with Mirage (actually, in the first instance, using Froc with Lwt). FRP is= probably a good approach to try and encode notions of self-scaling that we= 've talked about before -- so a Froc self-adjusting computation becomes the= overall framework within which Lwt threads execute to handle IO. if anyone wants to take a look at what i've done so far, code is in (neat name, huh? :) =20 this is very much work in progress, so suggestions for nice ways to integra= te Froc and Lwt gratefully received! --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From anil@recoil.org Thu Mar 28 11:18:52 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULAr6-0006SN-DJ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 11:18:52 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461449 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:18214 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1ULAr5-0002V7-rs (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 11:18:52 +0000 Received: (qmail 29428 invoked by uid 634); 28 Mar 2013 11:18:51 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 28 Mar 2013 11:18:50 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: mirage + froc = self-scaling? From: Anil Madhavapeddy In-Reply-To: <120C6BA3-69B2-4FC7-87C3-3BF7EA4AFE7F@nottingham.ac.uk> Date: Thu, 28 Mar 2013 11:18:49 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <3255044E-30C6-428F-AB53-3AEC2EE7DD03@recoil.org> References: <120C6BA3-69B2-4FC7-87C3-3BF7EA4AFE7F@nottingham.ac.uk> To: Richard Mortier X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 11:18:52 -0000 Content-Length: 2682 Lines: 71 Some background on FRP, and specifically the FROC self-adjusting = library: = http://ambassadortothecomputers.blogspot.co.uk/2010/05/how-froc-works.html= The goal of this work is to encode the external dependencies of a Mirage = unikernel explicitly, so that changes can ripple through the various = computation units in a well-defined manner. The tricky bit is that I/O = threads ('the outside world') need to be strongly separated from FROC = ('the configuration space') Consider the case of an ARP input packet, which needs to update a = configuration variable (the ARP table), but is in the middle of doing = some I/O. However, we want to safely weave together static input (a = static ARP configuration) with the dynamic inputs, which is where = Froctol comes from. There are many, many cases of such undefined = behaviour in conventional operating systems: what happens to existing = network sockets when a DHCP server rebinds an IP address on your = machine? Raphael, are your experiments with this from last summer available = anywhere? I couldn't spot them in your Github repo list. -anil On 28 Mar 2013, at 11:05, Richard Mortier = wrote: > hi all; >=20 > after a chat with anil at asplos, i've begun playing around with using = Froc with Mirage (actually, in the first instance, using Froc with Lwt). = FRP is probably a good approach to try and encode notions of = self-scaling that we've talked about before -- so a Froc self-adjusting = computation becomes the overall framework within which Lwt threads = execute to handle IO. >=20 > if anyone wants to take a look at what i've done so far, code is in = (neat name, huh? :) =20 >=20 > this is very much work in progress, so suggestions for nice ways to = integrate Froc and Lwt gratefully received! >=20 > --=20 > Cheers, >=20 > R. >=20 >=20 >=20 >=20 > This message and any attachment are intended solely for the addressee = and may contain confidential information. If you have received this = message in error, please send it back to me, and immediately delete it. = Please do not use, copy or disclose the information contained in this = message or in any attachment. Any views or opinions expressed by the = author of this email do not necessarily reflect the views of the = University of Nottingham. >=20 > This message has been checked for viruses but the contents of an = attachment > may still contain software viruses which could damage your computer = system: > you are advised to perform your own checks. Email communications with = the > University of Nottingham may be monitored as permitted by UK = legislation. >=20 From alain@frisch.fr Thu Mar 28 11:23:23 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULAvT-0006dT-J6 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 11:23:23 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461449 * -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no * trust * [193.252.23.212 listed in list.dnswl.dnsbl.ja.net] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from msa03.smtpout.orange.fr ([193.252.23.212]:18667 helo=msa.smtpout.orange.fr) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1ULAvT-0000yq-WY (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 11:23:23 +0000 Received: from [192.168.1.120] ([92.151.115.78]) by mwinf5d52 with ME id GzPM1l00J1hZWdG03zPM84; Thu, 28 Mar 2013 12:23:22 +0100 Message-ID: <5154282B.9090803@frisch.fr> Date: Thu, 28 Mar 2013 12:23:23 +0100 From: Alain Frisch User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:20.0) Gecko/20100101 Thunderbird/20.0 MIME-Version: 1.0 To: Anil Madhavapeddy , "cl-mirage@lists.cam.ac.uk List" Subject: Re: [Caml-list] Master-slave architecture behind an ocsigen server. References: <51520CAE.6020009@ens-lyon.org> <51540395.50202@frisch.fr> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Martin Jambon , caml users , Philippe Veber X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 11:23:23 -0000 Content-Length: 2721 Lines: 65 On 03/28/2013 12:02 PM, Anil Madhavapeddy wrote: > Are all of the messages through these queues persistent, or just the larger ones that are too big to fit in the shared memory segment, and are they always point-to-point streams? The messages always go through shared memory queues (non persistent), but their payload is offloaded to the file system (in temporary files) when it is too large. There is no real persistence, though, because the temporary file is not self-describing (it is not sufficient to restart a computation after a process failure, for instance). (The distribution of computations through a database is closer to real persistence.) Queues are unidirectional point-to-point streams between two processes (we use a pair of such queues for between the main process and the scheduler, and between the scheduler and each worker process). The relevant part of the API is: ======================================================== type t (** The type of point-to-point shared memory queues. *) val create: ?max_size:int -> unit -> t (** A queue is created in one process with [create], passed by name (using [id]) to a single another process which can call [from_id]. *) val from_id: ?max_size:int -> string -> t (** Access a queue created by another process given its unique name. *) val id: t -> string (** Globally (system-wide) unique name attached to the queue. *) val close: t -> unit exception CannotGrow exception BrokenPipe val send: t -> string -> unit (** Non-blocking operation. Raises [BrokenPipe] if the reader has disconnected. Raises [CannotGrow] if the maximum size of the buffer has been reached. *) val read: t -> string option (** Non-blocking operation. Returns [None] if no message is available. Raises [BrokenPipe] if the writer has disconnected. *) ======================================================== If there is some interest for it, maybe Fabrice could release the code with an open source license? One thing which is not supported by this library is a notification mechanism to inform the other side of the queue that messages are available. For now, we simulate that by pinging the processes stdin/stdout descriptors. In the scheduler and the worker, we use select to monitor them (there is probably a big cost of doing so, especially considering how select is emulated under Windows). In the GUI process, we use standard .Net process monitoring facilities to inject callbacks in the main GUI thread. (The OCaml side of our application in mono-threaded, and we use a few external native or .Net threads to monitor system conditions.) Alain From raphlalou@gmail.com Thu Mar 28 11:33:05 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULB4r-00072v-7c (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 11:33:05 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.6 from SpamAssassin-3.3.2-1461449 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.223.169 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (raphlalou[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 1.2 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ie0-f169.google.com ([209.85.223.169]:52294) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1ULB4q-0002s5-QC (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 11:33:05 +0000 Received: by mail-ie0-f169.google.com with SMTP id qd14so9993725ieb.14 for ; Thu, 28 Mar 2013 04:33:03 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.50.25.2 with SMTP id y2mr6896575igf.9.1364470383274; Thu, 28 Mar 2013 04:33:03 -0700 (PDT) Received: by 10.42.18.9 with HTTP; Thu, 28 Mar 2013 04:33:03 -0700 (PDT) In-Reply-To: <3255044E-30C6-428F-AB53-3AEC2EE7DD03@recoil.org> References: <120C6BA3-69B2-4FC7-87C3-3BF7EA4AFE7F@nottingham.ac.uk> <3255044E-30C6-428F-AB53-3AEC2EE7DD03@recoil.org> Date: Thu, 28 Mar 2013 11:33:03 +0000 Message-ID: Subject: Re: mirage + froc = self-scaling? From: Raphael Proust To: Anil Madhavapeddy Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Richard Mortier , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 11:33:05 -0000 Content-Length: 4124 Lines: 109 On Thu, Mar 28, 2013 at 11:18 AM, Anil Madhavapeddy wrote= : > Some background on FRP, and specifically the FROC self-adjusting library: > http://ambassadortothecomputers.blogspot.co.uk/2010/05/how-froc-works.htm= l > > The goal of this work is to encode the external dependencies of a Mirage = unikernel explicitly, so that changes can ripple through the various comput= ation units in a well-defined manner. The tricky bit is that I/O threads (= 'the outside world') need to be strongly separated from FROC ('the configur= ation space') > > Consider the case of an ARP input packet, which needs to update a configu= ration variable (the ARP table), but is in the middle of doing some I/O. H= owever, we want to safely weave together static input (a static ARP configu= ration) with the dynamic inputs, which is where Froctol comes from. There = are many, many cases of such undefined behaviour in conventional operating = systems: what happens to existing network sockets when a DHCP server rebind= s an IP address on your machine? > > Raphael, are your experiments with this from last summer available anywhe= re? I couldn't spot them in your Github repo list. They are in a repo of yours: https://github.com/avsm/ocaml-ld The following comment (in lib/froc_lwt) is interesting. It list things that should be avoided when mixing the two paradigms (cooperative-threading and reactive). (*DOs and DONTs * * DONT expose writables (because it exposes the next DONT to the user) * DONT change and yield (or otherwise switch thread) * This can lead to partial changes being propagated. If you need to change * several values before propagation, treat all the changes as a critical * section, and make sure you propagate before yielding. * DONT bypass Froc_sa.bind by creating a new changeable, manually write and * propagate. * DONT implement events over changeables. This restriction ensures that the * whole system (in particullar, the data-plan) will be reactive (although * theoretically appealing, it is better, for performance reasons, to * restrict the reactive part to the configuration-plan). * DONT use read under a bind (to avoid inconsistencies) but * DO use read when you are in the data-plane (and thus not threading the * changeables around). *) One thing that is quite important with Froc: you can set several changes and push them in one go. Thus, when reconfiguring it is important to not yield in between different changes (otherwise another thread might push). > > -anil > > On 28 Mar 2013, at 11:05, Richard Mortier wrote: > >> hi all; >> >> after a chat with anil at asplos, i've begun playing around with using F= roc with Mirage (actually, in the first instance, using Froc with Lwt). FRP= is probably a good approach to try and encode notions of self-scaling that= we've talked about before -- so a Froc self-adjusting computation becomes = the overall framework within which Lwt threads execute to handle IO. >> >> if anyone wants to take a look at what i've done so far, code is in (neat name, huh? :) >> >> this is very much work in progress, so suggestions for nice ways to inte= grate Froc and Lwt gratefully received! >> >> -- >> Cheers, >> >> R. >> >> >> >> >> This message and any attachment are intended solely for the addressee an= d may contain confidential information. If you have received this message i= n error, please send it back to me, and immediately delete it. Please do = not use, copy or disclose the information contained in this message or in a= ny attachment. Any views or opinions expressed by the author of this email= do not necessarily reflect the views of the University of Nottingham. >> >> This message has been checked for viruses but the contents of an attachm= ent >> may still contain software viruses which could damage your computer syst= em: >> you are advised to perform your own checks. Email communications with th= e >> University of Nottingham may be monitored as permitted by UK legislation= . >> > > --=20 ______________ Rapha=C3=ABl Proust From anil@recoil.org Thu Mar 28 11:36:53 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULB8X-0007Am-Gt (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 11:36:53 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461449 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:16728 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1ULB8U-00022f-rk (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 11:36:51 +0000 Received: (qmail 12651 invoked by uid 634); 28 Mar 2013 11:36:48 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 28 Mar 2013 11:36:47 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: mirage + froc = self-scaling? From: Anil Madhavapeddy In-Reply-To: Date: Thu, 28 Mar 2013 11:36:45 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <80FCD823-A8D6-472E-8F8B-3ADBFBEF8CB6@recoil.org> References: <120C6BA3-69B2-4FC7-87C3-3BF7EA4AFE7F@nottingham.ac.uk> <3255044E-30C6-428F-AB53-3AEC2EE7DD03@recoil.org> To: Raphael Proust X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Richard Mortier , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 11:36:53 -0000 Content-Length: 3210 Lines: 84 On 28 Mar 2013, at 11:33, Raphael Proust wrote: > On Thu, Mar 28, 2013 at 11:18 AM, Anil Madhavapeddy = wrote: >> Some background on FRP, and specifically the FROC self-adjusting = library: >> = http://ambassadortothecomputers.blogspot.co.uk/2010/05/how-froc-works.html= >>=20 >> The goal of this work is to encode the external dependencies of a = Mirage unikernel explicitly, so that changes can ripple through the = various computation units in a well-defined manner. The tricky bit is = that I/O threads ('the outside world') need to be strongly separated = from FROC ('the configuration space') >>=20 >> Consider the case of an ARP input packet, which needs to update a = configuration variable (the ARP table), but is in the middle of doing = some I/O. However, we want to safely weave together static input (a = static ARP configuration) with the dynamic inputs, which is where = Froctol comes from. There are many, many cases of such undefined = behaviour in conventional operating systems: what happens to existing = network sockets when a DHCP server rebinds an IP address on your = machine? >>=20 >> Raphael, are your experiments with this from last summer available = anywhere? I couldn't spot them in your Github repo list. >=20 > They are in a repo of yours: https://github.com/avsm/ocaml-ld Ha! Right under my nose. I need a local search of the hundreds of = repositories now... >=20 > The following comment (in lib/froc_lwt) is interesting. It list things > that should be avoided when mixing the two paradigms > (cooperative-threading and reactive). >=20 >=20 > (*DOs and DONTs > * > * DONT expose writables (because it exposes the next DONT to the user) > * DONT change and yield (or otherwise switch thread) > * This can lead to partial changes being propagated. If you need to = change > * several values before propagation, treat all the changes as a = critical > * section, and make sure you propagate before yielding. > * DONT bypass Froc_sa.bind by creating a new changeable, manually = write and > * propagate. > * DONT implement events over changeables. This restriction ensures = that the > * whole system (in particullar, the data-plan) will be reactive = (although > * theoretically appealing, it is better, for performance reasons, to > * restrict the reactive part to the configuration-plan). > * DONT use read under a bind (to avoid inconsistencies) but > * DO use read when you are in the data-plane (and thus not threading = the > * changeables around). > *) >=20 > One thing that is quite important with Froc: you can set several > changes and push them in one go. Thus, when reconfiguring it is > important to not yield in between different changes (otherwise another > thread might push). Right... I think that the prototype should actually use more privilege = separation to help nail the semantics down. Each Froc cycle should spin up a new = process that runs its own Lwt thread, with explicit handoff between them for = resources. This will obviously be pretty heavyweight, but it's far easier to = optimise it within the same process later than it is to untangle a single I/O = threadpool into multiple ones. -anil From info@gerd-stolpmann.de Thu Mar 28 12:19:13 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULBnV-0001Ef-Lb (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 12:19:13 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461449 * -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no * trust * [212.227.17.10 listed in list.dnswl.dnsbl.ja.net] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from moutng.kundenserver.de ([212.227.17.10]:58560) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1ULBnU-0005Rf-RB (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 12:19:13 +0000 Received: from office1.lan.sumadev.de (dslb-094-219-220-119.pools.arcor-ip.net [94.219.220.119]) by mrelayeu.kundenserver.de (node=mreu4) with ESMTP (Nemesis) id 0LbCQQ-1V63CA2DdK-00koDW; Thu, 28 Mar 2013 13:19:00 +0100 Received: from samsung (ip-5-146-55-186.unitymediagroup.de [5.146.55.186]) by office1.lan.sumadev.de (Postfix) with ESMTPSA id 25BBBC00D0; Thu, 28 Mar 2013 13:19:00 +0100 (CET) Date: Thu, 28 Mar 2013 13:18:59 +0100 From: Gerd Stolpmann Subject: AW: [Caml-list] Master-slave architecture behind an ocsigen server. To: Anil Madhavapeddy In-Reply-To: (from anil@recoil.org on Thu Mar 28 12:02:46 2013) X-Mailer: Balsa 2.4.11 Message-Id: <1364473139.14693.2@samsung> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; DelSp=Yes; Format=Flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable X-Provags-ID: V02:K0:PaO70jxg63qLsi2nqv8Gcz5My+Z5IJSZfcmvOgB1na8 KCMozN6R3/pB9nVqiCjbbbEMfm3iMTSRTDl4FVdR2OA+GQxIoZ C7syaXoHvc5cM8TO4ARaURaLkJCYz8AbgIHUju8SUcg6I0Gh7R 94njrTESDlSHJVV/72RXnzVLNGhjU4gZEoWRmlUzeqni5zaNRF s3CBKm6mw9sZSo9Csl2jzHaHY9CZP+cPjGLrvStKyQNQBS0D2F nr8aVg2EzmQnL/CS5+8sMXOlzhDlWK/9QpH8YnDkHC3xJad0k3 0akLz0IJsvC4dKA7QOXd63/l0cEVSa3sxGjzCfVwgY/5E0bQwM O7iLTO5MIjq9FJuLJGBc= Cc: Martin Jambon , caml users , Philippe Veber , "cl-mirage@lists.cam.ac.uk List" , Alain Frisch X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 12:19:13 -0000 Content-Length: 7422 Lines: 153 Am 28.03.2013 12:02:46 schrieb(en) Anil Madhavapeddy: > On 28 Mar 2013, at 08:47, Alain Frisch wrote: >=20 > > On 03/28/2013 08:37 AM, Philippe Veber wrote: > >> Hi Martin, > >> nproc meets exactly my needs: a simple lwt-friendly interface to > >> dispatch function calls on a pool of processes that run on the same > >> machine. I have only one concern, that should probably be =20 > discussed on > >> the ocsigen list, that is I wonder if it is okay to fork the =20 > process > >> running the ocsigen server. I think I remember warnings on having =20 > parent > >> and children processes sharing connections/channels but it's =20 > really not > >> clear to me. > > > > FWIW, LexiFi uses an architecture quite close to this for our =20 > application. The main process manages the GUI and dispatches =20 > computations tasks to external processes. Some points to be noted: > > > > - Since this is a Windows application, we cannot rely on fork. =20 > Instead, we restart the application (Sys.argv.(0)), with specific =20 > command-line flag, captured by the library in charge of managing =20 > computations. This is done by calling a special function in this =20 > library; the function does nothing in the main process and in the =20 > sub-processes, it starts the special mode and never returns. This =20 > gives a chance to the main application to do some global =20 > initialization common to the main and sub processes (for instance, we =20 > dynlink external plugins in this initialization phase). > > > > - Computation functions are registered as global values. =20 > Registration returns an opaque handle which can be used to call such =20 > a function. We don't rely on marshaling closures. > > > > - The GUI process actually spawns a single sub-process (the =20 > Scheduler), which itself manages more worker sub-sub-processes (with =20 > a maximal number of workers). Currently, we don't do very clever =20 > scheduling based on task priorities, but this could easily be added. > > > > - An external computation can spawn sub-computations (by applying a =20 > parallel "map" to a list) either synchronously (direct style) or =20 > asynchronously (by providing a continuation function, which will be =20 > applied to the list of results, maybe in a different process). In =20 > both cases, this is done by sending those tasks to the Scheduler. =20 > The Scheduler dispatches computation tasks to available workers. In =20 > the synchronous parallel map, the caller runs an inner event loop to =20 > communicate with the Scheduler (and it only accepts sub-tasks created =20 > by itself or one of its descendants). > > > > - Top-level external computations can be stopped by the main =20 > process (e.g. on user request). Concretely, this kills all workers =20 > currently working on that task or one of its sub-tasks. > > > > - In addition to sending back the final results, computations can =20 > report progress to their caller and more intermediate results. This =20 > is useful to show a progress bar/status and partial results in the =20 > GUI before the end of the entire computation. > > > > - Communication between processes is done by exchanging marshaled =20 > "variants" (a tagged representation of OCaml values, generated =20 > automatically using our runtime types). Since we can attach special =20 > variantizers/devariantizers to specific types, this gives a chance to =20 > customize how some values have to be exchanged between processes =20 > (e.g. values relying on internal hash-consing are treated specially =20 > to recreate the maximal sharing in the sub-process). > > > > - Concretely, the communication between processes is done through =20 > queues of messages implemented with shared memory. (This component =20 > was developed by Fabrice Le Fessant and OCamlPro.) Large =20 > computation arguments or results (above a certain size) are stored on =20 > the file system, to avoid having to keep them in RAM for too long (if =20 > all workers are busy, the computation might wait for some time being =20 > started). >=20 > Are all of the messages through these queues persistent, or just the =20 > larger ones that are too big to fit in the shared memory segment, and =20 > are they always point-to-point streams? >=20 > We've got a similar need in Xen/Mirage for shared memory =20 > communication and queues, and have been breaking them out into =20 > standalone libs such as: >=20 > https://github.com/djs55/shared-memory-ring >=20 > ...which is ABI-compatible with the existing Xen shared memory =20 > interfaces, and also an OCaml version of the transport-agnostic API =20 > sketched out in: > http://anil.recoil.org/papers/2012-resolve-fable.pdf Interesting that there are now other shared memory implementations for =20 OCaml. Note that there are a number of them in Ocamlnet, with some =20 specialities not yet mentioned. There is the Netcamlbox library =20 providing message boxes of limited size for exchanging OCaml values =20 directly. That means the value is copied to the shared memory block by =20 the sender, and the receiver can pick it up there without copying it =20 again. Sender and receiver can map the memory at different addresses =20 (the copy procedure invoked by the sender takes care of possible =20 offsets, so that that Netcamlbox also allows the communication between =20 processes that don't have a fork relation). There is no need for =20 marshalling the value. http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/doc/html-main/Netca= mlbox.html Going even beyond that, Netmulticore implements an "ancient" heap in =20 shared memory (like Richard's Ancient lib, but with more options). This =20 heap is organized like OCaml's major heap, and there is even a GC =20 implementation for it. There are a number of data structures (arrays, =20 hash tables, queues, buffers) which are aware of residing in shared =20 memory. For synchronization there are mutexes, semaphores and condition =20 variables. So far the values to manipulate are already in shared =20 memory, programming with Netmulticore feels a lot like programming with =20 multi-threading. In practice, however, you need to frequently copy =20 values in and out, so it is not exactly as convenient. For =20 Netmulticore, all processes must map the shared memory to the same =20 address (easy with "fork"). http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/doc/html-main/Intro= .html#netmulticore http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/doc/html-main/Netmc= ore_tut.html > The missing link currently is the persistent queuing service, but =20 > we're investigating the options here (ocamlmq looks rather nice). There is also Netamqp, which can be used together with RabbitMQ. http://projects.camlcity.org/projects/netamqp.html Gerd > -anil >=20 >=20 > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs >=20 --=20 ------------------------------------------------------------ Gerd Stolpmann, Darmstadt, Germany gerd@gerd-stolpmann.de Creator of GODI and camlcity.org. Contact details: http://www.camlcity.org/contact.html Company homepage: http://www.gerd-stolpmann.de ------------------------------------------------------------= From Richard.Mortier@nottingham.ac.uk Thu Mar 28 12:49:29 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULCGn-0002Bq-Tc (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 12:49:29 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461449 * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine03-20433-3.icritical.com ([93.95.15.170]:39957) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1ULCGn-0002sA-Y3 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 12:49:29 +0000 Received: (qmail 24512 invoked from network); 28 Mar 2013 12:49:30 -0000 Received: from localhost (127.0.0.1) by engine03-20433-3.icritical.com with SMTP; 28 Mar 2013 12:49:30 -0000 Received: from engine03-20433-3.icritical.com ([127.0.0.1]) by localhost (engine03-20433-3.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 24389-03 for ; Thu, 28 Mar 2013 12:49:28 +0000 (GMT) Received: (qmail 23135 invoked by uid 599); 28 Mar 2013 12:47:50 -0000 Received: from unknown (HELO smtp3.nottingham.ac.uk) (128.243.44.55) by engine03-20433-3.icritical.com (qpsmtpd/0.28) with ESMTP; Thu, 28 Mar 2013 12:47:50 +0000 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp3.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1ULCFB-00007y-Qw for cl-mirage@lists.cam.ac.uk; Thu, 28 Mar 2013 12:47:49 +0000 From: Richard Mortier To: Anil Madhavapeddy Date: Thu, 28 Mar 2013 12:47:48 +0000 Subject: Re: mirage + froc = self-scaling? Thread-Topic: mirage + froc = self-scaling? Thread-Index: Ac4rsnQBRT8ImB3dSZSDFcHhQAIBvw== Message-ID: <74F9747D-E408-491E-8182-A5C108B94B6F@nottingham.ac.uk> References: <120C6BA3-69B2-4FC7-87C3-3BF7EA4AFE7F@nottingham.ac.uk> <3255044E-30C6-428F-AB53-3AEC2EE7DD03@recoil.org> <80FCD823-A8D6-472E-8F8B-3ADBFBEF8CB6@recoil.org> In-Reply-To: <80FCD823-A8D6-472E-8F8B-3ADBFBEF8CB6@recoil.org> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine03-20433-3.icritical.com Cc: "cl-mirage@lists.cam.ac.uk List" , Raphael Proust X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 12:49:30 -0000 Content-Length: 1858 Lines: 40 On 28 Mar 2013, at 11:36, Anil Madhavapeddy wrote: > to help nail the semantics down. Each Froc cycle should spin up a new pr= ocess > that runs its own Lwt thread, with explicit handoff between them for reso= urces. just trying to understand what you mean by this. i have to this point been = looking at having a Froc graph constructed from multiple cells (a tuple of = Changeable and setter function), and then having each Lwt thread capture on= e of the setters so that it can push a new value into the Froc graph in res= ponse to external IO. the bit i'm finding mildly tricky at the moment (or w= as until today, which is mostly being about annual performance reviews :/ i= s managing the Lwt thread pool correctly. (ie., calling Lwt_main.run on the= right things.) (a) does this sound sane or am i missing the point? (b) are you suggesting that each time an input comes in on an Lwt thread i = should spin off a new process and explicitly hand off resources, in order t= o be completely clear what and where are the resources? (primarily IO hand= les and such like i guess?) --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From Richard.Mortier@nottingham.ac.uk Thu Mar 28 12:55:07 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULCMF-0002Rd-Ej (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 12:55:07 +0000 X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1461449 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine03-20433-3.icritical.com ([93.95.15.170]:42892) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1ULCME-0003CW-87 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 12:55:07 +0000 Received: (qmail 26731 invoked from network); 28 Mar 2013 12:51:51 -0000 Received: from localhost (127.0.0.1) by engine03-20433-3.icritical.com with SMTP; 28 Mar 2013 12:51:51 -0000 Received: from engine03-20433-3.icritical.com ([127.0.0.1]) by localhost (engine03-20433-3.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 26330-05 for ; Thu, 28 Mar 2013 12:51:49 +0000 (GMT) Received: (qmail 25205 invoked by uid 599); 28 Mar 2013 12:50:18 -0000 Received: from unknown (HELO smtp3.nottingham.ac.uk) (128.243.44.55) by engine03-20433-3.icritical.com (qpsmtpd/0.28) with ESMTP; Thu, 28 Mar 2013 12:50:18 +0000 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp3.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1ULCFp-0000B3-Bm for cl-mirage@lists.cam.ac.uk; Thu, 28 Mar 2013 12:48:29 +0000 From: Richard Mortier To: Raphael Proust Date: Thu, 28 Mar 2013 12:48:28 +0000 Subject: Re: mirage + froc = self-scaling? Thread-Topic: mirage + froc = self-scaling? Thread-Index: Ac4rsouPPTvJvXejRmSxtwckxaQR8g== Message-ID: <7DEAC47E-4E69-4DCF-9EE4-8EBC089A81E9@nottingham.ac.uk> References: <120C6BA3-69B2-4FC7-87C3-3BF7EA4AFE7F@nottingham.ac.uk> <3255044E-30C6-428F-AB53-3AEC2EE7DD03@recoil.org> In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine03-20433-3.icritical.com Cc: "cl-mirage@lists.cam.ac.uk List" , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 12:55:07 -0000 Content-Length: 1098 Lines: 27 On 28 Mar 2013, at 11:33, Raphael Proust wrote: > They are in a repo of yours: https://github.com/avsm/ocaml-ld >=20 > The following comment (in lib/froc_lwt) is interesting. It list things > that should be avoided when mixing the two paradigms > (cooperative-threading and reactive). interesting - thanks! i'll take a look. --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From felipe.franciosi@citrix.com Thu Mar 28 12:51:27 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULCIh-0002L4-U0 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 12:51:27 +0000 X-Cam-SpamDetails: score -1.4 from SpamAssassin-3.3.2-1461449 * -1.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from smtp.eu.citrix.com ([46.33.159.39]:21458) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1ULCIh-0001sj-7H (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 12:51:27 +0000 X-IronPort-AV: E=Sophos;i="4.84,925,1355097600"; d="scan'208";a="3008947" Received: from lonpex01cl02.citrite.net ([10.30.203.102]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/AES128-SHA; 28 Mar 2013 12:51:26 +0000 Received: from LONPEX01CL03.citrite.net ([169.254.3.114]) by LONPEX01CL02.citrite.net ([10.30.203.102]) with mapi id 14.02.0342.003; Thu, 28 Mar 2013 12:51:26 +0000 From: Felipe Franciosi To: 'Anil Madhavapeddy' Subject: RE: [Xen-devel] Request for input: Extended event channel support Thread-Topic: [Xen-devel] Request for input: Extended event channel support Thread-Index: AQHOKt2+U4fNIrGUlk6DqMyIlegjyJi57xaAgAEgY9A= Date: Thu, 28 Mar 2013 12:51:25 +0000 Message-ID: <9F2C4E7DFB7839489C89757A66C5AD6205F056@LONPEX01CL03.citrite.net> References: <8E3B77A0-C082-4F98-995E-CC1514631D49@recoil.org> In-Reply-To: <8E3B77A0-C082-4F98-995E-CC1514631D49@recoil.org> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.30.203.1] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "xen-users@lists.xen.org" , George Dunlap , "cl-mirage@lists.cam.ac.uk List" , "xen-devel@lists.xen.org" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 12:51:28 -0000 Content-Length: 534 Lines: 17 -----Original Message----- From: xen-devel-bounces@lists.xen.org [mailto:xen-devel-bounces@lists.xen.o= rg] On Behalf Of Anil Madhavapeddy Sent: 27 March 2013 19:37 To: George Dunlap Cc: xen-users@lists.xen.org; cl-mirage@lists.cam.ac.uk List; xen-devel@list= s.xen.org Subject: Re: [Xen-devel] Request for input: Extended event channel support > If I'm not mistaken, every guest needs at least 2 event channels (console= , xenstore) and probably has two more for a net and disk device. Presumably for vCPUs as well IINM? Felipe From anil@recoil.org Thu Mar 28 12:54:33 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULCLh-0002Qo-6o (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 12:54:33 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461449 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:11125 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1ULCLg-0001iw-q3 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 12:54:33 +0000 Received: (qmail 31103 invoked by uid 634); 28 Mar 2013 12:54:31 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 28 Mar 2013 12:54:31 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: [Xen-devel] Request for input: Extended event channel support From: Anil Madhavapeddy In-Reply-To: <9F2C4E7DFB7839489C89757A66C5AD6205F056@LONPEX01CL03.citrite.net> Date: Thu, 28 Mar 2013 12:54:28 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <8E3B77A0-C082-4F98-995E-CC1514631D49@recoil.org> <9F2C4E7DFB7839489C89757A66C5AD6205F056@LONPEX01CL03.citrite.net> To: Felipe Franciosi X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "xen-users@lists.xen.org" , George Dunlap , "cl-mirage@lists.cam.ac.uk List" , "xen-devel@lists.xen.org" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 12:54:33 -0000 Content-Length: 880 Lines: 29 On 28 Mar 2013, at 12:51, Felipe Franciosi = wrote: > -----Original Message----- > From: xen-devel-bounces@lists.xen.org = [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of Anil Madhavapeddy > Sent: 27 March 2013 19:37 > To: George Dunlap > Cc: xen-users@lists.xen.org; cl-mirage@lists.cam.ac.uk List; = xen-devel@lists.xen.org > Subject: Re: [Xen-devel] Request for input: Extended event channel = support >=20 >=20 >> If I'm not mistaken, every guest needs at least 2 event channels = (console, xenstore) and probably has two more for a net and disk device. >=20 > Presumably for vCPUs as well IINM? Yes, except that in Mirage's case we're single vCPU only, and use = multiple VMs to act as parallel processes with explicit message passing. But we would still need an event channel for the vchan shared ring, in = this case too... -anil From felipe.franciosi@citrix.com Thu Mar 28 13:02:20 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULCTE-0002i6-IW (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 13:02:20 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.3 from SpamAssassin-3.3.2-1461449 * -1.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from smtp.eu.citrix.com ([46.33.159.39]:54925) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1ULCTD-00072E-Sl (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 13:02:20 +0000 X-IronPort-AV: E=Sophos;i="4.84,925,1355097600"; d="scan'208";a="3009337" Received: from lonpex01cl02.citrite.net ([10.30.203.102]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/AES128-SHA; 28 Mar 2013 13:02:20 +0000 Received: from LONPEX01CL03.citrite.net ([169.254.3.114]) by LONPEX01CL02.citrite.net ([10.30.203.102]) with mapi id 14.02.0342.003; Thu, 28 Mar 2013 13:02:19 +0000 From: Felipe Franciosi To: 'Anil Madhavapeddy' Subject: RE: [Xen-devel] Request for input: Extended event channel support Thread-Topic: [Xen-devel] Request for input: Extended event channel support Thread-Index: AQHOKt2+U4fNIrGUlk6DqMyIlegjyJi57xaAgAEgY9CAAAGPAIAAAOsQ Date: Thu, 28 Mar 2013 13:02:18 +0000 Message-ID: <9F2C4E7DFB7839489C89757A66C5AD6205F0F0@LONPEX01CL03.citrite.net> References: <8E3B77A0-C082-4F98-995E-CC1514631D49@recoil.org> <9F2C4E7DFB7839489C89757A66C5AD6205F056@LONPEX01CL03.citrite.net> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.30.203.1] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "xen-users@lists.xen.org" , George Dunlap , "cl-mirage@lists.cam.ac.uk List" , "xen-devel@lists.xen.org" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 13:02:20 -0000 Content-Length: 815 Lines: 26 -----Original Message----- From: Anil Madhavapeddy [mailto:anil@recoil.org]=20 Sent: 28 March 2013 12:54 To: Felipe Franciosi Cc: xen-users@lists.xen.org; George Dunlap; cl-mirage@lists.cam.ac.uk List;= xen-devel@lists.xen.org Subject: Re: [Xen-devel] Request for input: Extended event channel support >> Presumably for vCPUs as well IINM? >Yes, except that in Mirage's case we're single vCPU only, and use multiple= VMs to act as parallel processes with explicit message passing. There's also the buffered IO event channel, but I'm pretty sure this is onl= y for HVM so shouldn't affect the Mirage use case. Just mentioning in case there is someone out there reading this and working= out numbers for HVM guests. :) http://lists.xen.org/archives/html/xen-changelog/2011-11/msg00139.html Cheers, Felipe From anil@recoil.org Thu Mar 28 13:28:40 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULCsi-0003MA-ME (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 13:28:40 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461449 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:10697 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1ULCsh-0006LD-qv (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 13:28:40 +0000 Received: (qmail 1229 invoked by uid 634); 28 Mar 2013 13:28:39 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 28 Mar 2013 13:28:38 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: mirage + froc = self-scaling? From: Anil Madhavapeddy In-Reply-To: <74F9747D-E408-491E-8182-A5C108B94B6F@nottingham.ac.uk> Date: Thu, 28 Mar 2013 13:28:36 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <120C6BA3-69B2-4FC7-87C3-3BF7EA4AFE7F@nottingham.ac.uk> <3255044E-30C6-428F-AB53-3AEC2EE7DD03@recoil.org> <80FCD823-A8D6-472E-8F8B-3ADBFBEF8CB6@recoil.org> <74F9747D-E408-491E-8182-A5C108B94B6F@nottingham.ac.uk> To: Richard Mortier X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" , Raphael Proust X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 13:28:40 -0000 Content-Length: 3872 Lines: 83 On 28 Mar 2013, at 12:47, Richard Mortier = wrote: >=20 > On 28 Mar 2013, at 11:36, Anil Madhavapeddy wrote: >=20 >> to help nail the semantics down. Each Froc cycle should spin up a = new process >> that runs its own Lwt thread, with explicit handoff between them for = resources. >=20 > just trying to understand what you mean by this. i have to this point = been looking at having a Froc graph constructed from multiple cells (a = tuple of Changeable and setter function), and then having each Lwt = thread capture one of the setters so that it can push a new value into = the Froc graph in response to external IO. the bit i'm finding mildly = tricky at the moment (or was until today, which is mostly being about = annual performance reviews :/ is managing the Lwt thread pool correctly. = (ie., calling Lwt_main.run on the right things.) >=20 > (a) does this sound sane or am i missing the point? >=20 > (b) are you suggesting that each time an input comes in on an Lwt = thread i should spin off a new process and explicitly hand off = resources, in order to be completely clear what and where are the = resources? (primarily IO handles and such like i guess?) It's important not to treat Lwt and Froc as black boxes, but understand = precisely what's going under the hood. The libraries are all pretty = simple at their core. - Lwt keeps track of I/O resources (file descriptors) and timers (in a = priority queue). Its inner loop ends up in a select()-style call, and = when something becomes runnable, the associated closure is invoked. - Froc keeps track of configuration variables ("signals") and the = control flow dependencies required to calculate variables that are = dependent on other variables. The update manager needs some = environmental way of retrieving configuration updates that aren't = specified within FROC. The confusion (I think) arises from the fact that both Lwt and Froc are = not reentrant libraries. When you create an Lwt "thread", you're adding = a value to a global structure. This makes it very easy to inadvertently = break the rules that Raphael pointed out in Froc_lwt about crossing the = streams between the two libraries. There are two solutions I see: - Rewrite both Lwt and Froc to accept an explicit context that would = make them both reentrant. This would be quite intrusive, particularly = for Lwt. - Separate them with a hard process barrier, so you're simply unable to = see the Froc module from the subprocess that is running the Lwt I/O = code. =20 The observant among you will notice that this architecture is very = similar to the Mirari module that VincentB is currently working on. You = can consider Mirari to be the 'Froc configuration' end, and the VMs or = UNIX binaries it spawns to be the 'Lwt I/O' end. And the problem we're trying to solve is: "how can a Mirage VM talk back = to Mirari and let it know that some configuration state needs to be = changed. For example, because a resource or runqueue is overloaded?". = In this situation, Mirari will get a notification from the VM about a = config update, will recompile a new VM, spin it up, and execute a = transition function across the two. However, before implementing this = in Mirari and Mirage, it would be *far* easier to prototype it as a = simple UNIX-based standalone library. The semantics behind the transition between the two Lwt processes = resulting from a FROC update are what is currently completely undefined. = Ideally, only genuinely new resources should be moved across, and any = common resources (e.g. a network bridge) should be shared. This is = possibly related to the 'traceable data structure work' work that Jake = wanted to implement in FROC, but never completed: http://www.umut-acar.org/publications/pldi2010.pdf [PLDI 2010 paper] Hope this helps (?) -anil= From anil@recoil.org Thu Mar 28 16:42:33 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULFuL-0003PR-Sh (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 16:42:33 +0000 X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1461449 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:13052 helo=dark.recoil.org) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1ULFuL-0003nS-7H (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 16:42:33 +0000 Received: (qmail 17366 invoked by uid 634); 28 Mar 2013 16:42:32 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 28 Mar 2013 16:42:32 +0000 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Repository format tools Message-Id: <207C0738-581F-4FF1-B58A-07F662F1B84D@recoil.org> Date: Thu, 28 Mar 2013 16:42:29 +0000 To: "opam-devel@lists.ocaml.org" , "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 16:42:34 -0000 Content-Length: 2784 Lines: 71 A number of us are assembling quite a few repositories for proper = open-source releases. In particular, between Mirage and Xen, I think we = have about 40 to work through in the next couple of months. One thing I'd like to have is some tooling to make it easier to = manipulate these groups of repositories. I've started a repository = here: https://github.com/avsm/opam-repo-tools ...which builds an opam-changelog binary that can connect to Github and = parses the CHANGES file from the repository. For example: > $ opam-changelog OCamlPro opam >=20 > 1.0.1 > * By default, turn-on checking of certificates for downloaded = dependency archives: use `./configure --disable-certificate-check` to go = back to the previous behavior > * Check the md5sum of downloaded archives when compiling OPAM > * Improved `opam info` command (more information, non-zero error code = when no patterns match) > * Display OS and OPAM version on internal errors to ease error = reporting > * Fix `opam reinstall` when reinstalling a package wich is a = dependency of installed packages (regression introduced in 0.9.5) > * Export and read `OPAMSWITCH` to be able to call OPAM in different = switches > * `opam-client` can now be used in a toplevel > * `-n` now means `--no-setup` and not `--no-checksums` anymore > * Fix stupid typo in detection of FreeBSD archs > * Fix installation of local compilers with local paths endings with = `.../ocaml/` >=20 > 1.0.0 > * Improve the lexer performance (thx to @oandrieu) > * Fix various typos (thx to @chaudhuri) > * Fix build issue (thx to @avsm) >=20 I wanted to find out what else people want. I'd like to know: * README: What a repository is for * TODO: "There be dragons here" * CONTRIBUTING: Guidelines for sending pull requests, see = https://github.com/blog/1184-contributing-guidelines * CHANGES: See above -- this would be particularly useful to display = after an OPAM update, for example. Some of these are subsumed by various metadata files such as _oasis, but = not all the repositories use OASIS, so I'd prefer simple text files in = Markdown where possible. So, does anyone have any strong opinions on this? I'm inclined to go = for the simple CHANGES format that OPAM currently uses (and several = Mirage libraries). But I also notice the Debian changelog format is = quite simple and we could adopt that: = http://www.debian.org/doc/debian-policy/ch-source.html package (version) distribution(s); urgency=3Durgency [optional blank line(s), stripped] * change details more change details =20 [blank line(s), included in output of dpkg-parsechangelog] * even more change details [optional blank line(s), stripped] -- maintainer name [two spaces] date -anil= From vb@luminar.eu.org Thu Mar 28 17:28:42 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULGd0-0004Ym-B9 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 17:28:42 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.3 from SpamAssassin-3.3.2-1461449 * -1.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:41200) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1ULGcz-0004EA-sh (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 28 Mar 2013 17:28:42 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id A9E5CBF422 for ; Thu, 28 Mar 2013 18:28:41 +0100 (CET) Message-ID: <51547DC9.3090505@luminar.eu.org> Date: Thu, 28 Mar 2013 17:28:41 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: [ANN] mirage-fs-0.5.0 and mirari-0.9.2 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 17:28:42 -0000 Content-Length: 999 Lines: 26 I=92m pleased to announce the release of the two aforementionned versions= =20 of mirage-fs and mirari. * mirage-fs (mir-crunch) now has a -o option to write its output in a=20 file instead of in stdout. It was needed when using the autoswitch=20 feature of mirari: indeed, opam config exec "program arg1 ... argn"=20 --switch=3Dfoo does not spawns the line inside "" in a subshell, but=20 directly runs the program and its arguments, and that was breaking mirari= . * mirari now has a mirari clean target that does the cleaning after a=20 mirari build. It has also the "autoswitch" feature: manual switching is=20 not needed anymore, and --xen and --unix switches are available to build=20 the mirage-kernel for backends xen and unix-direct respectively. So, the workflow of developing mirage with mirari does not involve=20 switching anymore. Just use the --xen or --unix arguments to commands=20 passed to mirari (while in your usual switch) to use the selected backend= . Cheers, Vincent From anil@recoil.org Fri Mar 29 10:07:39 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULWDj-0003Jj-SB (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 29 Mar 2013 10:07:39 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461994 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:31818 helo=dark.recoil.org) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1ULWDj-0000ZH-R7 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 29 Mar 2013 10:07:39 +0000 Received: (qmail 4753 invoked by uid 634); 29 Mar 2013 10:07:39 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.198]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 29 Mar 2013 10:07:38 +0000 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: [ANN] mirage-fs-0.5.0 and mirari-0.9.2 From: Anil Madhavapeddy In-Reply-To: <51547DC9.3090505@luminar.eu.org> Date: Fri, 29 Mar 2013 10:07:38 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <2CE129E9-2E7F-46A5-A12C-9B6285836D08@recoil.org> References: <51547DC9.3090505@luminar.eu.org> To: Vincent Bernardoff X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Mar 2013 10:07:39 -0000 Content-Length: 1250 Lines: 34 Nice work Vincent! This all makes it so much more pleasant to work with = mirage-www. Looking forward to the next "mirari run" step next! -anil On 28 Mar 2013, at 17:28, Vincent Bernardoff wrote: > I=92m pleased to announce the release of the two aforementionned = versions of mirage-fs and mirari. >=20 > * mirage-fs (mir-crunch) now has a -o option to write its output in a = file instead of in stdout. It was needed when using the autoswitch = feature of mirari: indeed, opam config exec "program arg1 ... argn" = --switch=3Dfoo does not spawns the line inside "" in a subshell, but = directly runs the program and its arguments, and that was breaking = mirari. >=20 > * mirari now has a mirari clean target that does the cleaning after a = mirari build. It has also the "autoswitch" feature: manual switching is = not needed anymore, and --xen and --unix switches are available to build = the mirage-kernel for backends xen and unix-direct respectively. >=20 >=20 > So, the workflow of developing mirage with mirari does not involve = switching anymore. Just use the --xen or --unix arguments to commands = passed to mirari (while in your usual switch) to use the selected = backend. >=20 > Cheers, >=20 > Vincent >=20 From Richard.Mortier@nottingham.ac.uk Fri Mar 29 18:50:01 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULeNF-0001qJ-QN (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 29 Mar 2013 18:50:01 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461994 * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine03-20433-10.icritical.com ([93.159.202.53]:38341) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1ULeNE-0007X1-Y2 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 29 Mar 2013 18:50:01 +0000 Received: (qmail 32508 invoked from network); 29 Mar 2013 18:49:55 -0000 Received: from localhost (127.0.0.1) by engine03-20433-10.icritical.com with SMTP; 29 Mar 2013 18:49:55 -0000 Received: from engine03-20433-10.icritical.com ([127.0.0.1]) by localhost (engine03-20433-10.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 32016-09 for ; Fri, 29 Mar 2013 18:49:52 +0000 (GMT) Received: (qmail 32456 invoked by uid 599); 29 Mar 2013 18:49:51 -0000 Received: from unknown (HELO smtp3.nottingham.ac.uk) (128.243.44.55) by engine03-20433-10.icritical.com (qpsmtpd/0.28) with ESMTP; Fri, 29 Mar 2013 18:49:51 +0000 Received: from uiwexhub01.ad.nottingham.ac.uk ([128.243.15.133]) by smtp3.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1ULeN9-0002s9-CC for cl-mirage@lists.cam.ac.uk; Fri, 29 Mar 2013 18:49:55 +0000 From: Richard Mortier To: Anil Madhavapeddy , Vincent Bernardoff Date: Fri, 29 Mar 2013 18:49:54 +0000 Subject: Re: Net.Manager Thread-Topic: Net.Manager Thread-Index: Ac4srjPqtRvUhCXFRo2HvoK45yV3yw== Message-ID: <60C29D48-8275-4CA3-998C-C0DF4DA3A3BF@nottingham.ac.uk> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> <5152D688.3080706@luminar.eu.org> <5152D727.9000500@luminar.eu.org> <5152DBA5.9010104@luminar.eu.org> <9C4CC087-F27E-4281-8A1F-48036FE6FA49@recoil.org> In-Reply-To: <9C4CC087-F27E-4281-8A1F-48036FE6FA49@recoil.org> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine03-20433-10.icritical.com Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Mar 2013 18:50:01 -0000 Content-Length: 2116 Lines: 52 On 27 Mar 2013, at 11:46, Anil Madhavapeddy wrote: > On 27 Mar 2013, at 11:44, Vincent Bernardoff wrote: >=20 >> On 27/03/2013 11:36, Haris Rotsos wrote: >>> sorry replied too late to the first mail. Ok tell me when the code is >>> ready for the tap case to see the resulting API and we can see then how >>> we can fit something in a more modular way. >>>=20 >>=20 >> For now I made that: >> https://github.com/vbmithr/ocaml-tuntap >>=20 >> This is a Linux/Mac library that covers only creating/attaching tun/tap = interfaces, extracted out of the mirage tab stubs and completed for Linux. >=20 > Mort: does this fix your weird Linux routing problem with the old tap cod= e? ok, tried this, sort of, i think. short answer: no joy. that is,=20 - i pulled and built the latest ocaml-tuntap, producing tunctl.native. - i ran "sudo ./tunctl.native add tap0 ; sudo ./tunctl.native inet tap0 10.= 0.0.2" - i ran "make run" in mirage-www ...and i still get the same results, i.e., i see the outgoing SYN on lo get= ting a RST in response. i also see pings to 10.0.0.2 being received on and = replied to via lo. i tried creating tap1 with address 10.0.0.3 which worked= fine, but same results -- trying to get packets (eg., ping) to 10.0.0.3 ca= uses them to be visible on lo and not on tap1. or am i missing something obvious? --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From vb@luminar.eu.org Fri Mar 29 19:47:52 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULfHE-00025H-GO (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 29 Mar 2013 19:47:52 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -1.3 from SpamAssassin-3.3.2-1461994 * -1.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:53272) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1ULfHB-0001rE-qC (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 29 Mar 2013 19:47:52 +0000 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id E1DB0BF701 for ; Fri, 29 Mar 2013 20:47:48 +0100 (CET) Message-ID: <5155EFE4.8060201@luminar.eu.org> Date: Fri, 29 Mar 2013 19:47:48 +0000 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Net.Manager References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> <5152D688.3080706@luminar.eu.org> <5152D727.9000500@luminar.eu.org> <5152DBA5.9010104@luminar.eu.org> <9C4CC087-F27E-4281-8A1F-48036FE6FA49@recoil.org> <60C29D48-8275-4CA3-998C-C0DF4DA3A3BF@nottingham.ac.uk> In-Reply-To: <60C29D48-8275-4CA3-998C-C0DF4DA3A3BF@nottingham.ac.uk> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Mar 2013 19:47:52 -0000 Content-Length: 1656 Lines: 52 On 29/03/2013 18:49, Richard Mortier wrote: > > On 27 Mar 2013, at 11:46, Anil Madhavapeddy wrote: > >> On 27 Mar 2013, at 11:44, Vincent Bernardoff wrote= : >> >>> On 27/03/2013 11:36, Haris Rotsos wrote: >>>> sorry replied too late to the first mail. Ok tell me when the code i= s >>>> ready for the tap case to see the resulting API and we can see then = how >>>> we can fit something in a more modular way. >>>> >>> >>> For now I made that: >>> https://github.com/vbmithr/ocaml-tuntap >>> >>> This is a Linux/Mac library that covers only creating/attaching tun/t= ap interfaces, extracted out of the mirage tab stubs and completed for Li= nux. >> >> Mort: does this fix your weird Linux routing problem with the old tap = code? > > ok, tried this, sort of, i think. short answer: no joy. > > that is, > > - i pulled and built the latest ocaml-tuntap, producing tunctl.native. > - i ran "sudo ./tunctl.native add tap0 ; sudo ./tunctl.native inet tap0= 10.0.0.2" > - i ran "make run" in mirage-www > > ...and i still get the same results, i.e., i see the outgoing SYN on lo= getting a RST in response. i also see pings to 10.0.0.2 being received o= n and replied to via lo. i tried creating tap1 with address 10.0.0.3 whic= h worked fine, but same results -- trying to get packets (eg., ping) to 1= 0.0.0.3 causes them to be visible on lo and not on tap1. > > or am i missing something obvious? > The code in mirage-platform is not using ocaml-tuntap for now=85 :( Since= =20 we want to change the design, I did not bother updating the code for=20 now, sorry. I could make the changes quite quickly, if you want. Cheers Vincent From Richard.Mortier@nottingham.ac.uk Fri Mar 29 20:06:19 2013 Received: from ppsw-42.csi.cam.ac.uk ([131.111.8.142]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULfZ5-0002Im-Db (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 29 Mar 2013 20:06:19 +0000 X-Cam-SpamDetails: score -0.1 from SpamAssassin-3.3.2-1461994 * -0.1 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine01-20433-7.icritical.com ([151.236.209.33]:43348) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1ULfZ4-0007Oy-6X (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 29 Mar 2013 20:06:19 +0000 Received: (qmail 5120 invoked from network); 29 Mar 2013 20:06:13 -0000 Received: from localhost (127.0.0.1) by engine01-20433-7.icritical.com with SMTP; 29 Mar 2013 20:06:13 -0000 Received: from engine01-20433-7.icritical.com ([127.0.0.1]) by localhost (engine01-20433-7.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 04647-09 for ; Fri, 29 Mar 2013 20:05:59 +0000 (GMT) Received: (qmail 5040 invoked by uid 599); 29 Mar 2013 20:05:59 -0000 Received: from unknown (HELO smtp4.nottingham.ac.uk) (128.243.220.65) by engine01-20433-7.icritical.com (qpsmtpd/0.28) with ESMTP; Fri, 29 Mar 2013 20:05:59 +0000 Received: from uiwexhub02.ad.nottingham.ac.uk ([128.243.15.132]) by smtp4.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1ULfYn-0002PY-G7 for cl-mirage@lists.cam.ac.uk; Fri, 29 Mar 2013 20:06:01 +0000 From: Richard Mortier To: Vincent Bernardoff Date: Fri, 29 Mar 2013 20:05:59 +0000 Subject: Re: Net.Manager Thread-Topic: Net.Manager Thread-Index: Ac4suNVT0bDeJp6QTt+8JnRDHzRiUQ== Message-ID: <15B27FAF-8744-4B5F-BC85-328C8E7388D7@nottingham.ac.uk> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> <5152D688.3080706@luminar.eu.org> <5152D727.9000500@luminar.eu.org> <5152DBA5.9010104@luminar.eu.org> <9C4CC087-F27E-4281-8A1F-48036FE6FA49@recoil.org> <60C29D48-8275-4CA3-998C-C0DF4DA3A3BF@nottingham.ac.uk> <5155EFE4.8060201@luminar.eu.org> In-Reply-To: <5155EFE4.8060201@luminar.eu.org> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine01-20433-7.icritical.com Cc: "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Mar 2013 20:06:19 -0000 Content-Length: 1770 Lines: 37 On 29 Mar 2013, at 19:47, Vincent Bernardoff wrote: > The code in mirage-platform is not using ocaml-tuntap for now=85 :( Since= =20 > we want to change the design, I did not bother updating the code for=20 > now, sorry. I could make the changes quite quickly, if you want. ah- no, that's fine. tbh given what i understand of mirage use of tuntap an= d ocaml-tuntap, i'd be surprised if it made much difference switching to oc= aml-tuntap -- they're using largely the same calls/parameters aren't they? = it still seems to me more likely that it's something to do with bridging/fo= rwarding/routing that's been set incorrectly... :( =20 fwiw i've also tried creating a bridge interface with brctl and adding tap0= to that; no difference.=20 if you have mirage-www working using unix-direct on a linux box, i don't su= ppose you could mail me the output of a few commands? eg., "route -n", "if= config -a", "ip route list table all" -- might help me spot whatever idiocy= is breaking this... :/ --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From anil@recoil.org Fri Mar 29 20:08:50 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULfbW-0002L0-3X (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 29 Mar 2013 20:08:50 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1461994 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:14417 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1ULfbV-0007xY-Xx (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 29 Mar 2013 20:08:49 +0000 Received: (qmail 23928 invoked by uid 634); 29 Mar 2013 20:08:49 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 29 Mar 2013 20:08:48 +0000 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Net.Manager From: Anil Madhavapeddy In-Reply-To: <15B27FAF-8744-4B5F-BC85-328C8E7388D7@nottingham.ac.uk> Date: Fri, 29 Mar 2013 20:08:46 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <1CD4770E-BFDF-406C-A727-635822579433@recoil.org> References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> <5152D688.3080706@luminar.eu.org> <5152D727.9000500@luminar.eu.org> <5152DBA5.9010104@luminar.eu.org> <9C4CC087-F27E-4281-8A1F-48036FE6FA49@recoil.org> <60C29D48-8275-4CA3-998C-C0DF4DA3A3BF@nottingham.ac.uk> <5155EFE4.8060201@luminar.eu.org> <15B27FAF-8744-4B5F-BC85-328C8E7388D7@nottingham.ac.uk> To: Richard Mortier X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk" , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Mar 2013 20:08:50 -0000 Content-Length: 1074 Lines: 28 On 29 Mar 2013, at 20:05, Richard Mortier = wrote: >=20 > On 29 Mar 2013, at 19:47, Vincent Bernardoff wrote: >=20 >> The code in mirage-platform is not using ocaml-tuntap for now=85 :( = Since=20 >> we want to change the design, I did not bother updating the code for=20= >> now, sorry. I could make the changes quite quickly, if you want. >=20 > ah- no, that's fine. tbh given what i understand of mirage use of = tuntap and ocaml-tuntap, i'd be surprised if it made much difference = switching to ocaml-tuntap -- they're using largely the same = calls/parameters aren't they? it still seems to me more likely that it's = something to do with bridging/forwarding/routing that's been set = incorrectly... :( =20 The purpose of splitting out the tuntap code into a separate repository = is so that you can create a regression test for it to ensure that the = basic network bridging is working. Create a really simple ping test using only ocaml-tuntap before you even = try Mirage (and put it in lib_test in ocaml-tuntap too). -anil= From daniel.buenzli@erratique.ch Sat Mar 30 15:08:25 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1ULxOL-0004zn-6h (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 30 Mar 2013 15:08:25 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamScore: ss X-Cam-SpamDetails: score 2.0 from SpamAssassin-3.3.2-1462428 * 2.0 RCVD_IN_MAPS_RBL RBL: Relay in RBL, * http://www.mail-abuse.com/enduserinfo_rbl.html * [178.197.232.85 listed in rbl-plus.mail-abuse.ja.net] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail6.webfaction.com ([74.55.86.74]:48881 helo=smtp.webfaction.com) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1ULxOI-0000UD-qw (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 30 Mar 2013 15:08:25 +0000 Received: from [172.20.10.2] (85-232.197-178.cust.bluewin.ch [178.197.232.85]) by smtp.webfaction.com (Postfix) with ESMTP id 59E5A20BE080; Sat, 30 Mar 2013 15:08:19 +0000 (UTC) Date: Sat, 30 Mar 2013 16:08:21 +0100 From: =?utf-8?Q?Daniel_B=C3=BCnzli?= To: Anil Madhavapeddy Message-ID: <970493CCD3174FB1AF2A6C74F312AB52@erratique.ch> In-Reply-To: <207C0738-581F-4FF1-B58A-07F662F1B84D@recoil.org> References: <207C0738-581F-4FF1-B58A-07F662F1B84D@recoil.org> Subject: Re: [opam-devel] Repository format tools X-Mailer: sparrow 1.6.4 (build 1178) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Cc: "=?utf-8?Q?opam-devel=40lists.ocaml.org?=" , "=?utf-8?Q?cl-mirage=40lists.cam.ac.uk_List?=" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Mar 2013 15:08:25 -0000 Content-Length: 2017 Lines: 54 Le jeudi, 28 mars 2013 =C3=A0 17:42, Anil Madhavapeddy a =C3=A9crit : > I wanted to find out what else people want. I'd like to know: > =20 > * README: What a repository is for > * TODO: =22There be dragons here=22 > * CONTRIBUTING: Guidelines for sending pull requests, see https://githu= b.com/blog/1184-contributing-guidelines > * CHANGES: See above -- this would be particularly useful to display af= ter an OPAM update, for example. I also sometimes add a DEVEL file for developer instructions (e.g. https:= //github.com/dbuenzli/uunf/blob/master/DEVEL). Do you think this should g= o in CONTRIBUTING =3F (I'd still prefer DEVEL it's less intrusive when yo= u =60ls=60). =20 > Some of these are subsumed by various metadata files such as =5Foasis, = but not all the repositories use OASIS, so I'd prefer simple text files i= n Markdown where possible. Seems a good idea to me. I currently use the metadata of =5Foasis to gene= rate my READMEs, but it is suboptimal on github. The thing is that I don'= t want to repeat myself. Maybe the extraction should be done the other wa= y round. =20 =20 > So, does anyone have any strong opinions on this=3F I'm inclined to go = for the simple CHANGES format that OPAM currently uses (and several Mirag= e libraries). I think it should be tweaked to at least: 1) Be fully markdown renderable. That means I'd use proper sections (=23)= for releases. 2) Use the yyyy-mm-dd time stamp format (rfc 3339). This format has a lot= of good properties (e.g. ascii order gives you time order, clear month v= s day order, etc.). =20 3) (That's really a pointless obsession of mine) I like to give the physi= cal location I'm in when I release. =46or me it gives a little human touc= h to all these virtual bits. Example, https://github.com/dbuenzli/xmlm/blob/master/CHANGES (with a rel= ease made from =E5=A4=A7=E8=B6=B3=E5=8E=BF=E2=80=A6, I can precisely reme= mber that hotel room with a good wired connection, worked on Vg for a few= days there). Best, Daniel From anil@recoil.org Sat Mar 30 19:05:27 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UM15j-0006qm-Bc (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 30 Mar 2013 19:05:27 +0000 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1462428 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:25930 helo=dark.recoil.org) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UM15h-0001js-ZS (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 30 Mar 2013 19:05:27 +0000 Received: (qmail 26894 invoked by uid 634); 30 Mar 2013 19:05:25 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sat, 30 Mar 2013 19:05:25 +0000 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: [opam-devel] Repository format tools From: Anil Madhavapeddy In-Reply-To: <970493CCD3174FB1AF2A6C74F312AB52@erratique.ch> Date: Sat, 30 Mar 2013 19:05:21 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <3517D216-4E0A-48E3-9168-08503A02DCD1@recoil.org> References: <207C0738-581F-4FF1-B58A-07F662F1B84D@recoil.org> <970493CCD3174FB1AF2A6C74F312AB52@erratique.ch> To: =?iso-8859-1?Q?Daniel_B=FCnzli?= X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "opam-devel@lists.ocaml.org" , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Mar 2013 19:05:27 -0000 Content-Length: 3094 Lines: 75 On 30 Mar 2013, at 15:08, Daniel B=FCnzli = wrote: >=20 > Le jeudi, 28 mars 2013 =E0 17:42, Anil Madhavapeddy a =E9crit : >=20 >> I wanted to find out what else people want. I'd like to know: >>=20 >> * README: What a repository is for >> * TODO: "There be dragons here" >> * CONTRIBUTING: Guidelines for sending pull requests, see = https://github.com/blog/1184-contributing-guidelines >> * CHANGES: See above -- this would be particularly useful to display = after an OPAM update, for example. >=20 > I also sometimes add a DEVEL file for developer instructions (e.g. = https://github.com/dbuenzli/uunf/blob/master/DEVEL). Do you think this = should go in CONTRIBUTING ? (I'd still prefer DEVEL it's less intrusive = when you `ls`). CONTRIBUTING is used by Github to add help to a pull request, so it's = not quite the same as DEVEL. I like the idea of a file that tells you = how to build and hack on a repository -- for instance, which the = recommended build system is, what autogenerated files will present (e.g. = from ATDgen) and so forth. I'll add DEVEL to the skeleton repository... >> Some of these are subsumed by various metadata files such as _oasis, = but not all the repositories use OASIS, so I'd prefer simple text files = in Markdown where possible. > Seems a good idea to me. I currently use the metadata of _oasis to = generate my READMEs, but it is suboptimal on github. The thing is that I = don't want to repeat myself. Maybe the extraction should be done the = other way round. Unfortunately, not all projects can use OASIS for various reasons, = usually because they are large repositories that use OMake and need the = faster build speed, or have special compilation requirements (some of = the Mirage ones). I agree this all needs to converge, but it'll take some time... >> So, does anyone have any strong opinions on this? I'm inclined to go = for the simple CHANGES format that OPAM currently uses (and several = Mirage libraries). >=20 > I think it should be tweaked to at least: >=20 > 1) Be fully markdown renderable. That means I'd use proper sections = (#) for releases. >=20 > 2) Use the yyyy-mm-dd time stamp format (rfc 3339). This format has a = lot of good properties (e.g. ascii order gives you time order, clear = month vs day order, etc.). =20 >=20 Yeah, this is easily parseable too. Thomas, any objections to using = something like this for OPAM and friends? We'd need to call the file = 'CHANGES.md' so that Github will render it as a Markdown file. > 3) (That's really a pointless obsession of mine) I like to give the = physical location I'm in when I release. For me it gives a little human = touch to all these virtual bits. Heh, I once correlated the GPS traces from my phone to my Git commit = times to figure out where I was most productive writing code (answer: = Ray's Jazz Cafe at Foyles). Anything after the version and header would ben fine to leave as = user-defined metadata in our format then. I agree that having a = Markdown format is probably better than the Debian metadata. -anil= From vb@luminar.eu.org Sun Mar 31 14:14:58 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UMI66-00019k-61 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 31 Mar 2013 14:14:58 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.3 from SpamAssassin-3.3.2-1462718 * -2.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:58186) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UMI65-0007I2-sA (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 31 Mar 2013 14:14:58 +0100 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id 5D11DBF6E9 for ; Sun, 31 Mar 2013 15:14:57 +0200 (CEST) Message-ID: <515836D1.2080401@luminar.eu.org> Date: Sun, 31 Mar 2013 14:14:57 +0100 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Mirari template Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Mar 2013 13:14:58 -0000 Content-Length: 745 Lines: 24 Hello, Today I tried to use templates to generate the main.ml with Mirari=20 instead of putting the source code as strings in the program. You can checkout git://github.com/vbmithr/mirari branch template. Basically, I added a template/main.ml file which is an OCaml file (the=20 former generated main.ml) with optcomp directives in it. Mirari has been modified to output a settings.ml file instead, that is=20 included by the templated main.ml. I don't know if it is useful or not, I just started that because I=20 didn=92t like very much the idea to have the generated code embedded in=20 mirari.ml, and that I needed to modify it to experiment a bit with the=20 UNIX backend. What do you think about it, if anything ? Cheers, Vincent From vb@luminar.eu.org Sun Mar 31 14:22:51 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UMIDj-0001DQ-OH (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 31 Mar 2013 14:22:51 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.3 from SpamAssassin-3.3.2-1462718 * -2.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:35033) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UMIDj-0003Ol-R9 (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 31 Mar 2013 14:22:51 +0100 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id DDC68BF6E9 for ; Sun, 31 Mar 2013 15:22:50 +0200 (CEST) Message-ID: <515838AA.6040208@luminar.eu.org> Date: Sun, 31 Mar 2013 14:22:50 +0100 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Mirari template References: <515836D1.2080401@luminar.eu.org> In-Reply-To: <515836D1.2080401@luminar.eu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Mar 2013 13:22:51 -0000 Content-Length: 1069 Lines: 34 On 31/03/2013 14:14, Vincent Bernardoff wrote: > Hello, > > Today I tried to use templates to generate the main.ml with Mirari > instead of putting the source code as strings in the program. > > You can checkout git://github.com/vbmithr/mirari branch template. > > Basically, I added a template/main.ml file which is an OCaml file (the > former generated main.ml) with optcomp directives in it. > > Mirari has been modified to output a settings.ml file instead, that is > included by the templated main.ml. > > I don't know if it is useful or not, I just started that because I > didn=92t like very much the idea to have the generated code embedded in > mirari.ml, and that I needed to modify it to experiment a bit with the > UNIX backend. > > What do you think about it, if anything ? > > Cheers, > > Vincent > Just remembered why I did that in the first place: I want the=20 autogenerated main.ml have a different code according to the backend as=20 well, just in case it would be needed. For example, I don=92t want to use= =20 the Net.Manager for UNIX. Vincent From vb@luminar.eu.org Sun Mar 31 14:41:37 2013 Received: from ppsw-51.csi.cam.ac.uk ([131.111.8.151]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UMIVt-0001NX-45 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 31 Mar 2013 14:41:37 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.3 from SpamAssassin-3.3.2-1462718 * -2.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:45371) by ppsw-51.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UMIVs-000339-Yo (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 31 Mar 2013 14:41:37 +0100 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id 6EC2EBF6E9 for ; Sun, 31 Mar 2013 15:41:36 +0200 (CEST) Message-ID: <51583D10.2010004@luminar.eu.org> Date: Sun, 31 Mar 2013 14:41:36 +0100 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Mirari template References: <515836D1.2080401@luminar.eu.org> <515838AA.6040208@luminar.eu.org> In-Reply-To: <515838AA.6040208@luminar.eu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Mar 2013 13:41:37 -0000 Content-Length: 1198 Lines: 41 On 31/03/2013 14:22, Vincent Bernardoff wrote: > On 31/03/2013 14:14, Vincent Bernardoff wrote: >> Hello, >> >> Today I tried to use templates to generate the main.ml with Mirari >> instead of putting the source code as strings in the program. >> >> You can checkout git://github.com/vbmithr/mirari branch template. >> >> Basically, I added a template/main.ml file which is an OCaml file (the >> former generated main.ml) with optcomp directives in it. >> >> Mirari has been modified to output a settings.ml file instead, that is >> included by the templated main.ml. >> >> I don't know if it is useful or not, I just started that because I >> didn=92t like very much the idea to have the generated code embedded i= n >> mirari.ml, and that I needed to modify it to experiment a bit with the >> UNIX backend. >> >> What do you think about it, if anything ? >> >> Cheers, >> >> Vincent >> > > Just remembered why I did that in the first place: I want the > autogenerated main.ml have a different code according to the backend as > well, just in case it would be needed. For example, I don=92t want to u= se > the Net.Manager for UNIX. > > Vincent > Ok to use Core in mirari/mirage, BTW ? Vincent From vb@luminar.eu.org Sun Mar 31 16:00:41 2013 Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UMJkP-0001vs-8y (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 31 Mar 2013 16:00:41 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score -2.3 from SpamAssassin-3.3.2-1462718 * -2.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:43714) by ppsw-41.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UMJkO-0004Ln-Sb (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 31 Mar 2013 16:00:41 +0100 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id A0F74BF6FD for ; Sun, 31 Mar 2013 17:00:40 +0200 (CEST) Message-ID: <51584F98.5070401@luminar.eu.org> Date: Sun, 31 Mar 2013 16:00:40 +0100 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130311 Thunderbird/17.0.4 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Net.Manager References: <514A0148.1090501@luminar.eu.org> <27EB5BFB-6958-4F0F-B970-3EE6D58218E6@nottingham.ac.uk> <514C3C26.8080409@luminar.eu.org> <514C7C33.1050902@luminar.eu.org> <5152D4CF.4030002@luminar.eu.org> <5152D688.3080706@luminar.eu.org> <5152D727.9000500@luminar.eu.org> <5152DBA5.9010104@luminar.eu.org> <9C4CC087-F27E-4281-8A1F-48036FE6FA49@recoil.org> <60C29D48-8275-4CA3-998C-C0DF4DA3A3BF@nottingham.ac.uk> <5155EFE4.8060201@luminar.eu.org> <15B27FAF-8744-4B5F-BC85-328C8E7388D7@nottingham.ac.uk> <1CD4770E-BFDF-406C-A727-635822579433@recoil.org> In-Reply-To: <1CD4770E-BFDF-406C-A727-635822579433@recoil.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Mar 2013 15:00:41 -0000 Content-Length: 626 Lines: 17 On 29/03/2013 20:08, Anil Madhavapeddy wrote: > The purpose of splitting out the tuntap code into a separate repository= is so that you can create a regression test for it to ensure that the ba= sic network bridging is working. > > Create a really simple ping test using only ocaml-tuntap before you eve= n try Mirage (and put it in lib_test in ocaml-tuntap too). > > -anil I just fixed a bug in set_ipv4, it was pretty bad, set_ipv4 should be=20 working now. It is very strange that I though it was "working" before=85=20 guess I really have to write unit tests. Anyway, it is in=20 vbmithr/ocaml-tuntap, tag 0.2 Vincent From anil@recoil.org Sun Mar 31 23:47:40 2013 Received: from ppsw-50.csi.cam.ac.uk ([131.111.8.150]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UMR2K-0006Ju-OS (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 31 Mar 2013 23:47:40 +0100 X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1462718 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:19680 helo=dark.recoil.org) by ppsw-50.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UMR2J-0006Xe-rY (Exim 4.72) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 31 Mar 2013 23:47:40 +0100 Received: (qmail 12627 invoked by uid 634); 31 Mar 2013 22:47:39 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sun, 31 Mar 2013 23:47:38 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Mirari template From: Anil Madhavapeddy In-Reply-To: <51583D10.2010004@luminar.eu.org> Date: Sun, 31 Mar 2013 23:47:15 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <515836D1.2080401@luminar.eu.org> <515838AA.6040208@luminar.eu.org> <51583D10.2010004@luminar.eu.org> To: Vincent Bernardoff , Jeremy Yallop X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Mar 2013 22:47:40 -0000 Content-Length: 1742 Lines: 62 Sure; Jeremy (CCed) is improving Cohttp+Async support, so depending on = it for the host tools is fine. Best use it everywhere in Mirari though, if you use it at all. The one missing thing in Cohttp/Async is SSL support, which we'll need = to add via an stunnel wrapper for now. Dave, is the existing Xapi stunnel = code in a library, or do we need to extract it? -anil On 31 Mar 2013, at 14:41, Vincent Bernardoff wrote: > On 31/03/2013 14:22, Vincent Bernardoff wrote: >> On 31/03/2013 14:14, Vincent Bernardoff wrote: >>> Hello, >>>=20 >>> Today I tried to use templates to generate the main.ml with Mirari >>> instead of putting the source code as strings in the program. >>>=20 >>> You can checkout git://github.com/vbmithr/mirari branch template. >>>=20 >>> Basically, I added a template/main.ml file which is an OCaml file = (the >>> former generated main.ml) with optcomp directives in it. >>>=20 >>> Mirari has been modified to output a settings.ml file instead, that = is >>> included by the templated main.ml. >>>=20 >>> I don't know if it is useful or not, I just started that because I >>> didn=92t like very much the idea to have the generated code embedded = in >>> mirari.ml, and that I needed to modify it to experiment a bit with = the >>> UNIX backend. >>>=20 >>> What do you think about it, if anything ? >>>=20 >>> Cheers, >>>=20 >>> Vincent >>>=20 >>=20 >> Just remembered why I did that in the first place: I want the >> autogenerated main.ml have a different code according to the backend = as >> well, just in case it would be needed. For example, I don=92t want to = use >> the Net.Manager for UNIX. >>=20 >> Vincent >>=20 >=20 > Ok to use Core in mirari/mirage, BTW ? >=20 > Vincent >=20