From anil@recoil.org Wed May 01 09:11:33 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXS8T-0000pe-C3 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 09:11:33 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1477509 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:15015 helo=dark.recoil.org) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1UXS8S-0004Y5-8t (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 09:11:33 +0100 Received: (qmail 11737 invoked by uid 634); 1 May 2013 08:11:32 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.48]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 01 May 2013 09:11:31 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: OpenFlow version hell From: Anil Madhavapeddy In-Reply-To: <20130430172146.GA14240@damage> Date: Wed, 1 May 2013 09:11:30 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20130430172146.GA14240@damage> To: Prashanth Mundkur X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 08:11:33 -0000 Content-Length: 2217 Lines: 70 Thanks for this summary, Prashanth. I looked around for 1.3 = implementations, and came up with just this switch: https://github.com/CPqD/ofsoftswitch13 This does convince me that having continuing to develop the controller = and switch library would very useful, since we could use it to mix-and-match different deployment scenarios, but also understand the performance = impact of different data structures for use within the switch itself... -anil On 30 Apr 2013, at 18:21, Prashanth Mundkur = wrote: >=20 > As mentioned on today's Mirage call, here are some pointers to > OpenFlow versioning issues. >=20 > https://github.com/horms/openvswitch/blob/master/OPENFLOW-1.1%2B#L38 >=20 > This document underplays the issues, especially for OpenFlow 1.1. > There are major changes in the semantics for flow matching. >=20 > - Wildcards vs. exact matching >=20 > = https://www.opennetworking.org/images/stories/downloads/sdn-resources/onf-= specifications/openflow/openflow-spec-v1.0.0.pdf > OF 1.0, Section 3.4: >=20 > Packets are matched against flow entries based on prioritization. An > entry that specifies an exact match (i.e., it has no wildcards) is > always the highest priority. All wildcard entries have a priority > associated with them. Higher priority entries must match before > lower priority ones. >=20 > = https://www.opennetworking.org/images/stories/downloads/sdn-resources/onf-= specifications/openflow/openflow-spec-v1.3.1.pdf > OF 1.3.1, Section 5.3: >=20 > The packet is matched against the table and only the highest priority > flow entry that matches the packet must be selected. >=20 > (The language is a little more ambiguous in OF 1.1, but the intent is > clarified in 1.3.1 above. >=20 > = https://www.opennetworking.org/images/stories/downloads/sdn-resources/onf-= specifications/openflow/openflow-spec-v1.1.0.pdf > Section OF 1.1, Section 4.4: >=20 > The switch should apply the instruction set and update the > associated counters of only the highest-priority flow entry > matching the packet. > ) >=20 > - VLAN tag matching >=20 > Summarized here: >=20 > https://github.com/horms/openvswitch/blob/master/DESIGN#L155 >=20 > --prashanth >=20 From h.rotsos@gmail.com Wed May 01 11:56:10 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXUhm-0002oG-1I (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 11:56:10 +0100 X-Cam-SpamDetails: score -0.6 from SpamAssassin-3.3.2-1477509 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.219.42 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (h.rotsos[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-oa0-f42.google.com ([209.85.219.42]:60992) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UXUhk-0002Tq-8t (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 11:56:10 +0100 Received: by mail-oa0-f42.google.com with SMTP id i10so1343861oag.15 for ; Wed, 01 May 2013 03:56:08 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.60.178.139 with SMTP id cy11mr505509oec.120.1367405767944; Wed, 01 May 2013 03:56:07 -0700 (PDT) Sender: h.rotsos@gmail.com Received: by 10.60.79.161 with HTTP; Wed, 1 May 2013 03:56:07 -0700 (PDT) In-Reply-To: References: <20130430172146.GA14240@damage> Date: Wed, 1 May 2013 11:56:07 +0100 X-Google-Sender-Auth: HO-Hw47iKzZ9bKDnhCFhKkOO8Ko Message-ID: Subject: Re: OpenFlow version hell From: Haris Rotsos To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=089e011774f70ff1e804dba5f7da Cc: Mirage List , Prashanth Mundkur X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 10:56:10 -0000 Content-Length: 16800 Lines: 352 --089e011774f70ff1e804dba5f7da Content-Type: text/plain; charset=UTF-8 Hi all, This is a mail I send a while ago to Nirav Dave and Andrew W. Moore in order to summarise the differences between the versions of the OpenFlow protocol. It might come in handy in this discussion. "So far I have used only the 1.0 specification in my projects and the 1.1 specifications, although they are assumed as the current default, there is no adoption for them yet. Just to organise things, I will group changes based on functionality: 1) Tables and switch state: In the 1.1 specifications there are two main changes on the processing packet pipeline. Firstly, there is a new table type which is called group table and by using an identifier, it can group together multiple flow definitions. Additionally, the lookup process changes and instead of a parallel search in all flow tables it now looks up flows among tables in a pipeline fashion. Assume n tables exist in the switch, then the packet will be matched against table 0 inittially. If a match is found then the actions are applied on the packet, and a new lookup is performed on to the next table of the switch etc. As a result a matching flow on table 1 may change the operation applied from the match of the 1st table etc. For the group table, my understanding is that they try to create a hack in the processing stack that allows 2 things: Firstly, process special packets like multicast packets in a homogenous way, without having to go through the pipeline and, secondly, reduce space to store actions by accumulating then in the group table. 2) matching tuple: Introduce bitmask matches on the mac address of a packet. Introduce a metadata field in the tuple that can store partial results during the processing of a packet. The basic idea is that the metadata field can store control defined state and allow to propagate state between the tables in the processing pipeline. This way you can create state machines between the various processing tables. The matching process can now extract also mpls, 802.3 and sctp packet headers. 3) Packet processing: As I already mentioned, the packet processing occurs now in a pipeline fashion. Additionally, during the processing of a packet, a list of actions is assigned to the packet. This list contains the transient list of operation that will be applied at the end on the packet. Each table can add actions in the list or invalidate previous one. A number of new actions are introduced that allow to handle mpls tags, ecn handling and copy or decrement ttl field (Although ttl is not exposed in the matching tupple). I think this summarizes the differences in the protocol. It appears that the protocol has been mildly bloated and as a result I am not aware of any hwimplementation so far. " By the way, I was wondering if I could take a look in the bluespec code of the switch you presented last week in SRI. I am asking mainly because I would like to see a bit how bluespec code looks like. hope my answer helped you a bit. If you have any further question feel free to contact me. On 1 May 2013 09:11, Anil Madhavapeddy wrote: > Thanks for this summary, Prashanth. I looked around for 1.3 > implementations, > and came up with just this switch: > https://github.com/CPqD/ofsoftswitch13 > > This does convince me that having continuing to develop the controller and > switch library would very useful, since we could use it to mix-and-match > different deployment scenarios, but also understand the performance impact > of different data structures for use within the switch itself... > > -anil > > On 30 Apr 2013, at 18:21, Prashanth Mundkur > wrote: > > > > > As mentioned on today's Mirage call, here are some pointers to > > OpenFlow versioning issues. > > > > https://github.com/horms/openvswitch/blob/master/OPENFLOW-1.1%2B#L38 > > > > This document underplays the issues, especially for OpenFlow 1.1. > > There are major changes in the semantics for flow matching. > > > > - Wildcards vs. exact matching > > > > > https://www.opennetworking.org/images/stories/downloads/sdn-resources/onf-specifications/openflow/openflow-spec-v1.0.0.pdf > > OF 1.0, Section 3.4: > > > > Packets are matched against flow entries based on prioritization. An > > entry that specifies an exact match (i.e., it has no wildcards) is > > always the highest priority. All wildcard entries have a priority > > associated with them. Higher priority entries must match before > > lower priority ones. > > > > > https://www.opennetworking.org/images/stories/downloads/sdn-resources/onf-specifications/openflow/openflow-spec-v1.3.1.pdf > > OF 1.3.1, Section 5.3: > > > > The packet is matched against the table and only the highest priority > > flow entry that matches the packet must be selected. > > > > (The language is a little more ambiguous in OF 1.1, but the intent is > > clarified in 1.3.1 above. > > > > > https://www.opennetworking.org/images/stories/downloads/sdn-resources/onf-specifications/openflow/openflow-spec-v1.1.0.pdf > > Section OF 1.1, Section 4.4: > > > > The switch should apply the instruction set and update the > > associated counters of only the highest-priority flow entry > > matching the packet. > > ) > > > > - VLAN tag matching > > > > Summarized here: > > > > https://github.com/horms/openvswitch/blob/master/DESIGN#L155 > > > > --prashanth > > > > > -- Charalampos Rotsos PhD student The University of Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD Phone: +44-(0) 1223 767032 Email: cr409@cl.cam.ac.uk --089e011774f70ff1e804dba5f7da Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi all,

This is a mail I send a while ago to Nirav Dave and Andre= w W. Moore in order to=C2=A0summarise=C2=A0the differences between the vers= ions of the OpenFlow protocol. It might come in handy in this discussion.= =C2=A0

"S= o far I have used only the 1.0 specification in my projects and the 1.1 spe= cifications, although they are assumed as the current default, there is no = adoption for them yet. Just to=C2=A0organise=C2=A0things, I will group chan= ges based on functionality:

1) Tables and switch state:

In the 1.1 specifications there are two main changes on the proc= essing packet pipeline. Firstly, there is a new table type which is called= =C2=A0group table=C2=A0and by using an identifier, it can group together mu= ltiple flow definitions. Additionally, the lookup process changes and inste= ad of a parallel search in all flow tables it now looks up flows among tabl= es in a pipeline fashion. Assume=C2=A0n=C2=A0tables exist in the switch, th= en the packet will be matched against table 0=C2=A0inittially. If a match i= s found then the actions are applied=C2=A0on=C2=A0the packet, and a new loo= kup is performed on to the next table of the switch etc. As a result a matc= hing=C2=A0flow=C2=A0on table 1 may change the operation applied from the ma= tch=C2=A0of=C2=A0the 1st table etc. For the group table, my understanding i= s that they try to create a hack in the processing stack that allows 2 thin= gs: Firstly, process special packets like multicast packets in a homogenous= way, without having to go through the pipeline and, secondly, reduce space= to store actions by accumulating=C2=A0then=C2=A0in the group table.=C2=A0<= /div>

2)=C2=A0matching=C2=A0tuple:=C2=A0

Introduce=C2=A0bitmask=C2=A0matches on the=C2=A0mac=C2=A0address= of a packet.=C2=A0
Introduce a metadata field in the tuple that can store partial results duri= ng the processing of a packet. The basic idea is that the metadata field ca= n store control defined state and allow to propagate state between the tabl= es in the processing pipeline.=C2=A0This way you can create state machines = between the various processing tables.=C2=A0
The matching process can now extract also=C2=A0mpls,=C2=A0802.3 and=C2=A0sctp=C2=A0packet headers.=C2=A0

3) Packet processing:

As I already mentioned, the packet processing occurs now in a pi= peline fashion. Additionally, during the processing of a packet, a list of = actions is =C2=A0assigned to the packet. This list contains the transient l= ist of=C2=A0operation=C2=A0that will be applied at the end=C2=A0on=C2=A0the= packet. Each table can add actions in the list or invalidate previous one.= =C2=A0
A number of new actions are introduced that allow to handle=C2=A0mpls=C2= =A0tags,=C2=A0ecn=C2=A0handling and copy or decrement=C2=A0ttl=C2=A0field (= Although=C2=A0ttl=C2=A0is not exposed in the matching=C2=A0tupple).=C2=A0

I think this summarizes the differences in the protocol. It app= ears that the protocol has been mildly bloated and as a result I am not awa= re of any=C2=A0hwimplementation so far. "

By the way, I was wondering if I could take a look=C2=A0in=C2= =A0the=C2=A0bluespec=C2=A0code of the switch you presented last week in SRI= . I am asking mainly because I would like to see a bit how=C2=A0bluespec=C2= =A0code looks like.=C2=A0

hope=C2=A0my answer helped you a bit. If you have any further q= uestion feel free to contact me.=C2=A0


O= n 1 May 2013 09:11, Anil Madhavapeddy <anil@recoil.org> wrote:=
Thanks for this summary, Prashanth. =C2=A0I looked around for 1.3 implement= ations,
and came up with just this switch:
https:= //github.com/CPqD/ofsoftswitch13

This does convince me that having continuing to develop the controller and<= br> switch library would very useful, since we could use it to mix-and-match different deployment scenarios, but also understand the performance impact<= br> of different data structures for use within the switch itself...

-anil

On 30 Apr 2013, at 18:21, Prashanth Mundkur <pmundkur.ocaml@gmail.com> wrote:

>
> As mentioned on today's Mirage call, here are some pointers to
> OpenFlow versioning issues.
>
> https://github.com/horms/openvswitch/blob/mast= er/OPENFLOW-1.1%2B#L38
>
> This document underplays the issues, especially for OpenFlow 1.1.
> There are major changes in the semantics for flow matching.
>
> - Wildcards vs. exact matching
>
> https://www.opennetworking.org/images/stories/downloads/sdn-resourc= es/onf-specifications/openflow/openflow-spec-v1.0.0.pdf
> OF 1.0, Section 3.4:
>
> =C2=A0Packets are matched against flow entries based on prioritization= . An
> =C2=A0entry that specifies an exact match (i.e., it has no wildcards) = is
> =C2=A0always the highest priority. =C2=A0All wildcard entries have a p= riority
> =C2=A0associated with them. Higher priority entries must match before<= br> > =C2=A0lower priority ones.
>
> https://www.opennetworking.org/images/stories/downloads/sdn-resourc= es/onf-specifications/openflow/openflow-spec-v1.3.1.pdf
> OF 1.3.1, Section 5.3:
>
> The packet is matched against the table and only the highest priority<= br> > flow entry that matches the packet must be selected.
>
> (The language is a little more ambiguous in OF 1.1, but the intent is<= br> > clarified in 1.3.1 above.
>
> https://www.opennetworking.org/images/stories/downloads/sdn-resourc= es/onf-specifications/openflow/openflow-spec-v1.1.0.pdf
> Section OF 1.1, Section 4.4:
>
> =C2=A0 The switch should apply the instruction set and update the
> =C2=A0 associated counters of only the highest-priority flow entry
> =C2=A0 matching the packet.
> )
>
> - VLAN tag matching
>
> Summarized here:
>
> https://github.com/horms/openvswitch/blob/master/DESIG= N#L155
>
> --prashanth
>





--
= Charalampos Rotsos
PhD student
The University of Cambridge
Compute= r Laboratory
William Gates Building
JJ Thomson Avenue
Cambridge CB3 0FD

Phone: +44-(0) 1223 767032
Email: cr409@cl.cam.ac.uk
--089e011774f70ff1e804dba5f7da-- From vb@luminar.eu.org Wed May 01 19:04:13 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXbO1-0004cQ-6M (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 19:04:13 +0100 X-Cam-SpamDetails: score -2.6 from SpamAssassin-3.3.2-1477509 * -2.6 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:43929) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UXbO0-0006Hj-Fo (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 19:04:13 +0100 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id 8D74BBF9B9 for ; Wed, 1 May 2013 20:04:11 +0200 (CEST) Message-ID: <5181591B.1070004@luminar.eu.org> Date: Wed, 01 May 2013 19:04:11 +0100 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130403 Thunderbird/17.0.5 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: https://polarssl.org/ Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 18:04:13 -0000 Content-Length: 134 Lines: 7 Another embeddded SSL lib, I just found out that mini-os is using it. Might be worth a look if not already done ? Cheers, Vincent From anil@recoil.org Wed May 01 19:07:58 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXbRe-0004ha-27 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 19:07:58 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1477509 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:11520 helo=dark.recoil.org) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UXbRd-00077A-FE (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 19:07:58 +0100 Received: (qmail 31028 invoked by uid 634); 1 May 2013 18:07:57 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 01 May 2013 19:07:56 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: https://polarssl.org/ From: Anil Madhavapeddy In-Reply-To: <5181591B.1070004@luminar.eu.org> Date: Wed, 1 May 2013 19:07:55 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5181591B.1070004@luminar.eu.org> To: Vincent Bernardoff X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 18:07:58 -0000 Content-Length: 628 Lines: 18 On 1 May 2013, at 19:04, Vincent Bernardoff wrote: > Another embeddded SSL lib, I just found out that mini-os is using it. = Might be worth a look if not already done ? >=20 It's also dual commercial GPLv2 licensed, unfortunately, and so roughly = the same as MatrixSSL. I've not looked in more depth though, as = MatrixSSL has a really nice low-level API that is perfect for our = binding needs... Jon, is your WIP on Github anywhere? I do think it would be easier to = have something on UNIX ahead of the mirage-platform version, if only to = run it through valgrind and bind to Async too... -anil From kosmo.zb@gmail.com Wed May 01 19:15:03 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXbYV-0004oN-H7 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 19:15:03 +0100 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-lb0-f178.google.com ([209.85.217.178]:46001) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UXbYV-0003vE-hS (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 19:15:03 +0100 Received: by mail-lb0-f178.google.com with SMTP id w10so1652567lbi.23 for ; Wed, 01 May 2013 11:15:03 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.112.162.130 with SMTP id ya2mr1486796lbb.122.1367432102997; Wed, 01 May 2013 11:15:02 -0700 (PDT) Received: by 10.112.19.10 with HTTP; Wed, 1 May 2013 11:15:02 -0700 (PDT) In-Reply-To: References: <5181591B.1070004@luminar.eu.org> Date: Wed, 1 May 2013 19:15:02 +0100 Message-ID: Subject: Re: https://polarssl.org/ From: David Sheets To: Anil Madhavapeddy Content-Type: text/plain; charset=ISO-8859-1 Cc: Mirage List , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 18:15:03 -0000 Content-Length: 804 Lines: 17 On Wed, May 1, 2013 at 7:07 PM, Anil Madhavapeddy wrote: > On 1 May 2013, at 19:04, Vincent Bernardoff wrote: > >> Another embeddded SSL lib, I just found out that mini-os is using it. Might be worth a look if not already done ? >> > > It's also dual commercial GPLv2 licensed, unfortunately, and so roughly the same as MatrixSSL. I've not looked in more depth though, as MatrixSSL has a really nice low-level API that is perfect for our binding needs... Perhaps you can avoid the GPL? It's not clear to me. > Jon, is your WIP on Github anywhere? I do think it would be easier to have something on UNIX ahead of the mirage-platform version, if only to run it through valgrind and bind to Async too... > > -anil > > From anil@recoil.org Wed May 01 19:18:18 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXbbe-0004sC-Tp (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 19:18:18 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1477509 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:21970 helo=dark.recoil.org) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UXbbe-0001VN-Dy (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 19:18:18 +0100 Received: (qmail 17989 invoked by uid 634); 1 May 2013 18:18:18 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 01 May 2013 19:18:17 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: https://polarssl.org/ From: Anil Madhavapeddy In-Reply-To: Date: Wed, 1 May 2013 19:18:16 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <58A3F02B-BA98-4C45-ADAC-06C1D3309402@recoil.org> References: <5181591B.1070004@luminar.eu.org> To: David Sheets X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Mirage List , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 18:18:19 -0000 Content-Length: 1012 Lines: 29 On 1 May 2013, at 19:15, David Sheets wrote: > On Wed, May 1, 2013 at 7:07 PM, Anil Madhavapeddy = wrote: >> On 1 May 2013, at 19:04, Vincent Bernardoff = wrote: >>=20 >>> Another embeddded SSL lib, I just found out that mini-os is using = it. Might be worth a look if not already done ? >>>=20 >>=20 >> It's also dual commercial GPLv2 licensed, unfortunately, and so = roughly the same as MatrixSSL. I've not looked in more depth though, as = MatrixSSL has a really nice low-level API that is perfect for our = binding needs... >=20 > Perhaps you can avoid > the GPL? It's not clear to me. Doesnt make any practical difference afaict: "You obey the GPL in all respects for the Program and all portions = (including modifications) of the Program included in the Derivative Work = (provided that this condition does not apply to Independent Works);" So, uh... it's effectively GPL then, right? -anil= From kosmo.zb@gmail.com Wed May 01 19:39:15 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXbvv-00056F-65 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 19:39:15 +0100 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-lb0-f172.google.com ([209.85.217.172]:63979) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UXbvv-00018u-gJ (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 19:39:15 +0100 Received: by mail-lb0-f172.google.com with SMTP id o10so1700999lbi.3 for ; Wed, 01 May 2013 11:39:14 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.152.3.137 with SMTP id c9mr1390884lac.5.1367433554640; Wed, 01 May 2013 11:39:14 -0700 (PDT) Received: by 10.112.19.10 with HTTP; Wed, 1 May 2013 11:39:14 -0700 (PDT) In-Reply-To: <58A3F02B-BA98-4C45-ADAC-06C1D3309402@recoil.org> References: <5181591B.1070004@luminar.eu.org> <58A3F02B-BA98-4C45-ADAC-06C1D3309402@recoil.org> Date: Wed, 1 May 2013 19:39:14 +0100 Message-ID: Subject: Re: https://polarssl.org/ From: David Sheets To: Anil Madhavapeddy Content-Type: text/plain; charset=ISO-8859-1 Cc: Mirage List , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 18:39:15 -0000 Content-Length: 1162 Lines: 24 On Wed, May 1, 2013 at 7:18 PM, Anil Madhavapeddy wrote: > On 1 May 2013, at 19:15, David Sheets wrote: > >> On Wed, May 1, 2013 at 7:07 PM, Anil Madhavapeddy wrote: >>> On 1 May 2013, at 19:04, Vincent Bernardoff wrote: >>> >>>> Another embeddded SSL lib, I just found out that mini-os is using it. Might be worth a look if not already done ? >>>> >>> >>> It's also dual commercial GPLv2 licensed, unfortunately, and so roughly the same as MatrixSSL. I've not looked in more depth though, as MatrixSSL has a really nice low-level API that is perfect for our binding needs... >> >> Perhaps you can avoid >> the GPL? It's not clear to me. > > Doesnt make any practical difference afaict: > > "You obey the GPL in all respects for the Program and all portions (including modifications) of the Program included in the Derivative Work (provided that this condition does not apply to Independent Works);" > > So, uh... it's effectively GPL then, right? "Program" is their library. I believe this is essentially a linking exception. > -anil From Dave.Scott@eu.citrix.com Wed May 01 20:10:00 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXcPg-0005qc-N9 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 20:10:00 +0100 X-Cam-SpamDetails: score -2.6 from SpamAssassin-3.3.2-1477509 * -2.6 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from smtp.eu.citrix.com ([46.33.159.39]:53193) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UXcPg-0004vn-0h (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 20:10:00 +0100 X-IronPort-AV: E=Sophos;i="4.87,552,1363132800"; d="scan'208";a="4128335" Received: from lonpmailmx01.citrite.net ([10.30.203.162]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/RC4-MD5; 01 May 2013 19:09:20 +0000 Received: from LONPMAILBOX01.citrite.net ([10.30.224.160]) by LONPMAILMX01.citrite.net ([10.30.203.162]) with mapi; Wed, 1 May 2013 20:09:58 +0100 From: Dave Scott To: David Sheets Date: Wed, 1 May 2013 20:09:57 +0100 Subject: Re: https://polarssl.org/ Thread-Topic: https://polarssl.org/ Thread-Index: Ac5Gn3i7qvim2oNdSmWm1x6ziiFigQ== Message-ID: References: <5181591B.1070004@luminar.eu.org> <58A3F02B-BA98-4C45-ADAC-06C1D3309402@recoil.org> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Vincent Bernardoff , Mirage List , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 19:10:00 -0000 Content-Length: 1637 Lines: 44 On May 1, 2013, at 7:39 PM, "David Sheets" wrote: > On Wed, May 1, 2013 at 7:18 PM, Anil Madhavapeddy wrote= : >> On 1 May 2013, at 19:15, David Sheets wrote: >>=20 >>> On Wed, May 1, 2013 at 7:07 PM, Anil Madhavapeddy wro= te: >>>> On 1 May 2013, at 19:04, Vincent Bernardoff wrote: >>>>=20 >>>>> Another embeddded SSL lib, I just found out that mini-os is using it.= Might be worth a look if not already done ? >>>>>=20 >>>>=20 >>>> It's also dual commercial GPLv2 licensed, unfortunately, and so roughl= y the same as MatrixSSL. I've not looked in more depth though, as MatrixSS= L has a really nice low-level API that is perfect for our binding needs... >>>=20 >>> Perhaps you can avoid >>> the GPL? It's not clear to me. >>=20 >> Doesnt make any practical difference afaict: >>=20 >> "You obey the GPL in all respects for the Program and all portions (incl= uding modifications) of the Program included in the Derivative Work (provid= ed that this condition does not apply to Independent Works);" >>=20 >> So, uh... it's effectively GPL then, right? >=20 > "Program" is their library. I believe this is essentially a linking excep= tion. >=20 We've talked a bit about SSL offload before (particularly to hardware). We = could link one of these libs with minios and make a C SSL proxy using libvc= han. Or we could a concept of a "process" to mirage -- perhaps one static p= rocess per vCPU -- and use inter process communication to communicate with = the ocaml code. Cheers, Dave From anil@recoil.org Wed May 01 21:07:30 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXdJK-0006lK-I8 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 21:07:30 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1477509 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:37974 helo=dark.recoil.org) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1UXdJK-0007Sj-6p (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 21:07:30 +0100 Received: (qmail 6470 invoked by uid 634); 1 May 2013 20:07:28 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 01 May 2013 21:07:24 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: mirage + froc = self-scaling? From: Anil Madhavapeddy In-Reply-To: Date: Wed, 1 May 2013 21:07:23 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <251A6A69-DB78-4598-AF34-0CE39B2CE8F6@recoil.org> References: <27C5F743E5AA481BB4A16B403F1842A3@erratique.ch> <4F6CF319-0E99-4CF3-A65D-993B26E7F77E@nottingham.ac.uk> <59296051-131D-425E-B59F-24F97CF4A51E@nottingham.ac.uk> <4ABA2037-0D63-460A-8CCC-202E0682745D@nottingham.ac.uk> To: Ashish Agarwal X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: =?iso-8859-1?Q?Daniel_B=FCnzli?= , Richard Mortier , Raphael Proust , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 20:07:30 -0000 Content-Length: 2716 Lines: 67 Just to pick up on this point, the issue is that I think we don't really = know if FRP can actually be used at scale. Or at least, noone has tried = yet in OCaml (some of the Javascript versions are reasonably well used = for UI reactive programming). So while I agree with the general need for tutorials, I'd probably focus = on much safer ground such as Lwt/Async ahead of putting FRP on = ocaml.org. However, a short tutorial on React and Lwt_react would = certainly be nice to have, if only to understand how to build = interesting data structures out of the primitives that are exposed. -anil On 10 Apr 2013, at 14:30, Ashish Agarwal wrote: > Slightly off-topic, but I'd like to encourage experts in topics like = this to contribute to ocaml.org. For example, in this case, it would be = great to add a page about FRP, where at the minimum the different FRP = libraries are listed. This isn't much work. Spending more time, it would = be nice to have tutorials for each of the libraries, a comparison = between them, and maybe references to the literature when appropriate. = This takes more work but is the kind of thing that can really help OCaml = be more widely used. You can always submit an issue with your text, and = we'll figure out how to integrate it to the right place. >=20 >=20 > On Wed, Apr 10, 2013 at 7:59 AM, Richard Mortier = wrote: >=20 > On 10 Apr 2013, at 12:57, Mortier Richard wrote: >=20 > > so to summarise, React's benefits: are specified denotational = semantics, and no global data structures; but React can only guarantee = no memory leaks in environments that support weak references (which does = not include JS until ECMAScript 6 is adopted). >=20 > sorry- i should, of course, have included far more complete and = up-to-date documentation as another benefit for React :) >=20 > -- > Cheers, >=20 > R. >=20 >=20 >=20 >=20 > This message and any attachment are intended solely for the addressee = and may contain confidential information. If you have received this = message in error, please send it back to me, and immediately delete it. = Please do not use, copy or disclose the information contained in this = message or in any attachment. Any views or opinions expressed by the = author of this email do not necessarily reflect the views of the = University of Nottingham. >=20 > This message has been checked for viruses but the contents of an = attachment > may still contain software viruses which could damage your computer = system: > you are advised to perform your own checks. Email communications with = the > University of Nottingham may be monitored as permitted by UK = legislation. >=20 >=20 From anil@recoil.org Wed May 01 21:25:30 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXdak-0006w2-Pm (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 21:25:30 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1477509 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:24252 helo=dark.recoil.org) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1UXdak-0002sZ-7c (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 01 May 2013 21:25:30 +0100 Received: (qmail 12818 invoked by uid 634); 1 May 2013 20:25:30 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 01 May 2013 21:25:29 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: https://polarssl.org/ From: Anil Madhavapeddy In-Reply-To: Date: Wed, 1 May 2013 21:25:29 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <9BA486E2-18A4-414B-BC41-68FD73DE2781@recoil.org> References: <5181591B.1070004@luminar.eu.org> <58A3F02B-BA98-4C45-ADAC-06C1D3309402@recoil.org> To: Dave Scott X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: David Sheets , Mirage List , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 20:25:30 -0000 Content-Length: 704 Lines: 16 On 1 May 2013, at 20:09, Dave Scott wrote: > We've talked a bit about SSL offload before (particularly to = hardware). We could link one of these libs with minios and make a C SSL = proxy using libvchan. Or we could a concept of a "process" to mirage -- = perhaps one static process per vCPU -- and use inter process = communication to communicate with the ocaml code. I'd really like to keep the SSL bindings outside of the main OCaml = process, so this is a good place to start doing so. Also, it's a = natural tie-in to the actor library, as those communication channels are = intended to either be vchan (xenserver) or internal tcp (EC2, where we = cant vchan). -anil From pierre.chambart@laposte.net Thu May 02 08:40:14 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXo7i-0001YD-7M (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 08:40:14 +0100 X-Cam-SpamDetails: score -2.5 from SpamAssassin-3.3.2-1477933 * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (pierre.chambart[at]laposte.net) * -2.5 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no * trust * [193.253.67.229 listed in list.dnswl.dnsbl.ja.net] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from smtpout4.laposte.net ([193.253.67.229]:44830 helo=smtpout.laposte.net) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UXo7h-000629-is (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 08:40:14 +0100 Received: from localhost ([81.64.134.21]) by mwinf8507-out with ME id WvgD1l0020Trr3l03vgD6U; Thu, 02 May 2013 09:40:13 +0200 Date: Thu, 2 May 2013 09:40:12 +0200 From: Pierre Chambart To: "cl-mirage@lists.cam.ac.uk List" Subject: Re: Mirari template Message-ID: <20130502094012.1446665c@laposte.net> In-Reply-To: References: <515836D1.2080401@luminar.eu.org> <515838AA.6040208@luminar.eu.org> <51583D10.2010004@luminar.eu.org> X-Mailer: Claws Mail 3.8.1 (GTK+ 2.24.10; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 07:40:14 -0000 Content-Length: 539 Lines: 15 Le Sun, 31 Mar 2013 23:47:15 +0100, Anil Madhavapeddy a =C3=A9crit : > The one missing thing in Cohttp/Async is SSL support, which we'll > need to add via an stunnel wrapper for now. Dave, is the existing > Xapi stunnel code in a library, or do we need to extract it? Wouldn't it be easier to do it using ocamlssl like lwt_ssl does ? It is quite easy. You need to be able to wake up threads when a file descriptor is ready for read/write, I haven't verified, but it should probably be possible in Async. --=20 Pierre From anil@recoil.org Thu May 02 08:44:53 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXoCD-0001g0-I1 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 08:44:53 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1477933 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:23246 helo=dark.recoil.org) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UXoCC-0000Du-1I (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 08:44:53 +0100 Received: (qmail 469 invoked by uid 634); 2 May 2013 07:44:52 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 02 May 2013 08:44:51 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Mirari template From: Anil Madhavapeddy In-Reply-To: <20130502094012.1446665c@laposte.net> Date: Thu, 2 May 2013 08:44:50 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <0DDB5FF0-1C9D-47F7-9F57-B76869068461@recoil.org> References: <515836D1.2080401@luminar.eu.org> <515838AA.6040208@luminar.eu.org> <51583D10.2010004@luminar.eu.org> <20130502094012.1446665c@laposte.net> To: Pierre Chambart X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 07:44:53 -0000 Content-Length: 1089 Lines: 28 On 2 May 2013, at 08:40, Pierre Chambart = wrote: > Le Sun, 31 Mar 2013 23:47:15 +0100, > Anil Madhavapeddy a =E9crit : >=20 >> The one missing thing in Cohttp/Async is SSL support, which we'll >> need to add via an stunnel wrapper for now. Dave, is the existing >> Xapi stunnel code in a library, or do we need to extract it? >=20 > Wouldn't it be easier to do it using ocamlssl like lwt_ssl does ? It = is > quite easy. You need to be able to wake up threads when a file > descriptor is ready for read/write, I haven't verified, but it should > probably be possible in Async. I basically don't trust those bindings. It would be far more robust to have low-level SSL bindings, and do the higher-level async handling, entropy generation and certificate callbacks in pure OCaml. OpenSSL is also a bit of a beast to compile for an embedded environment like = Mirage. (The long-term aim is to replace the SSL library with a pure OCaml one, and gradually rewriting the existing bindings as we go along seems like a good approach). -anil= From Dave.Scott@eu.citrix.com Thu May 02 09:08:06 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXoYg-0002ca-G7 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 09:08:06 +0100 X-Cam-SpamDetails: score -2.5 from SpamAssassin-3.3.2-1477933 * -2.5 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * 0.0 HTML_MESSAGE BODY: HTML included in message X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from smtp.eu.citrix.com ([46.33.159.39]:56040) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UXoYe-0002Xr-9d (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 09:08:06 +0100 X-IronPort-AV: E=Sophos;i="4.87,552,1363132800"; d="scan'208,217";a="4145524" Received: from lonpmailmx01.citrite.net ([10.30.203.162]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/RC4-MD5; 02 May 2013 08:07:02 +0000 Received: from LONPMAILBOX01.citrite.net ([10.30.224.160]) by LONPMAILMX01.citrite.net ([10.30.203.162]) with mapi; Thu, 2 May 2013 09:08:04 +0100 From: Dave Scott To: "cl-mirage@lists.cam.ac.uk" Date: Thu, 2 May 2013 09:08:02 +0100 Subject: Fwd: [ovs-discuss] Open vSwitch 1.10.0 Available Thread-Topic: [ovs-discuss] Open vSwitch 1.10.0 Available Thread-Index: Ac5HDCvEqK9/lI5qRyi3dtVO6lKl7A== Message-ID: References: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_A7268EEA504A409787E2566289153076eucitrixcom_" MIME-Version: 1.0 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 08:08:06 -0000 Content-Length: 8700 Lines: 122 --_000_A7268EEA504A409787E2566289153076eucitrixcom_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 RllJLCBvcGVudnN3aXRjaCBub3cgaGFzIGV4cGVyaW1lbnRhbCBzdXBwb3J0IGZvciBvcGVuZmxv dyB2ZXJzaW9ucyA+IDEuMA0KDQotLQ0KRGF2ZSBTY290dA0KWGVuU2VydmVyIFN5c3RlbSBBcmNo aXRlY3QNCg0KQmVnaW4gZm9yd2FyZGVkIG1lc3NhZ2U6DQoNCkZyb206IEp1c3RpbiBQZXR0aXQg PGpwZXR0aXRAbmljaXJhLmNvbTxtYWlsdG86anBldHRpdEBuaWNpcmEuY29tPj4NCkRhdGU6IE1h eSAyLCAyMDEzLCA4OjMzOjQzIEFNIEdNVCswMTowMA0KVG86ICJkaXNjdXNzQG9wZW52c3dpdGNo Lm9yZzxtYWlsdG86ZGlzY3Vzc0BvcGVudnN3aXRjaC5vcmc+IGRpc2N1c3MiIDxkaXNjdXNzQG9w ZW52c3dpdGNoLm9yZzxtYWlsdG86ZGlzY3Vzc0BvcGVudnN3aXRjaC5vcmc+PiwgImFubm91bmNl QG9wZW52c3dpdGNoLm9yZzxtYWlsdG86YW5ub3VuY2VAb3BlbnZzd2l0Y2gub3JnPiIgPGFubm91 bmNlQG9wZW52c3dpdGNoLm9yZzxtYWlsdG86YW5ub3VuY2VAb3BlbnZzd2l0Y2gub3JnPj4NClN1 YmplY3Q6IFtvdnMtZGlzY3Vzc10gT3BlbiB2U3dpdGNoIDEuMTAuMCBBdmFpbGFibGUNCg0KVGhl IE9wZW4gdlN3aXRjaCB0ZWFtIGlzIHBsZWFzZWQgdG8gYW5ub3VuY2UgdGhlIHJlbGVhc2Ugb2Yg T3BlbiB2U3dpdGNoIDEuMTAuMDoNCg0KICAgaHR0cDovL29wZW52c3dpdGNoLm9yZy9yZWxlYXNl cy9vcGVudnN3aXRjaC0xLjEwLjAudGFyLmd6DQoNClRoaXMgcmVsZWFzZSBjb250YWlucyBuZXcg ZmVhdHVyZXMsIHBlcmZvcm1hbmNlIGltcHJvdmVtZW50cywgYW5kIGEgc2lnbmlmaWNhbnQgcmV3 b3JraW5nIG9mIGNvcmUgY29tcG9uZW50cy4gIEZlYXR1cmUgaGlnaGxpZ2h0cyBvZiAxLjEwLjAg aW5jbHVkZToNCg0KICAgKiBTdXBwb3J0IGZvciB0aGUgVlhMQU4gdHVubmVsIHByb3RvY29sLg0K DQogICAqIEV4cGVyaW1lbnRhbCBzdXBwb3J0IGZvciBuZXdlciB2ZXJzaW9ucyBvZiBPcGVuRmxv dy4NCg0KICAgKiBXaXRoIHRoZSBMaW51eCBkYXRhcGF0aCwgcGFja2V0cyBmb3IgbmV3IGZsb3dz IGFyZSBub3cgcXVldWVkDQogICAgIHNlcGFyYXRlbHkgb24gYSBwZXItcG9ydCBiYXNpcywgc28g aXQgc2hvdWxkIG5vIGxvbmdlciBiZQ0KICAgICBwb3NzaWJsZSBmb3IgYSBsYXJnZSBudW1iZXIg b2YgbmV3IGZsb3dzIGFycml2aW5nIG9uIG9uZSBwb3J0IHRvDQogICAgIHByZXZlbnQgbmV3IGZs b3dzIGZyb20gYmVpbmcgcHJvY2Vzc2VkIG9uIG90aGVyIHBvcnRzLg0KDQogICAqIFBhdGNoIHBv cnRzIG5vIGxvbmdlciByZXF1aXJlIGtlcm5lbCBzdXBwb3J0LCBzbyB0aGV5IG5vdyB3b3JrDQog ICAgIHdpdGggRnJlZUJTRCBhbmQgdGhlIGtlcm5lbCBtb2R1bGUgYnVpbHQgaW50byBMaW51eCAz LjMgYW5kIGxhdGVyLg0KDQogICAqIFRoZSBtYXhpbXVtIHNpemUgb2YgdGhlIE1BQyBsZWFybmlu ZyB0YWJsZSBpcyBub3cgY29uZmlndXJhYmxlLg0KDQogICAqIEl0IGlzIHBvc3NpYmxlIHRvIHJl cXVlc3QgdGhlIE9wZW5GbG93IHBvcnQgbnVtYmVyIHdpdGggdGhlDQogICAgICJvZnBvcnRfcmVx dWVzdCIgY29sdW1uIGluIHRoZSBJbnRlcmZhY2UgdGFibGUuDQoNCiAgICogQW5kIG1hbnkgb3Ro ZXJzLiAgU2VlIHRoZSBmdWxsIGNoYW5nZSBsb2cgaGVyZToNCg0KICAgICAgIGh0dHA6Ly9vcGVu dnN3aXRjaC5vcmcvcmVsZWFzZXMvTkVXUy0xLjEwLjANCg0KRW5qb3khDQoNCi0tVGhlIE9wZW4g dlN3aXRjaCBUZWFtDQoNCi0tLS0tLS0tLS0tLS0tLS0tLS0tDQpPcGVuIHZTd2l0Y2ggaXMgYSBw cm9kdWN0aW9uIHF1YWxpdHksIG11bHRpbGF5ZXIgb3BlbiBzb3VyY2UgdmlydHVhbCBzd2l0Y2gu ICBJdCBpcyBkZXNpZ25lZCB0byBlbmFibGUgbWFzc2l2ZSBuZXR3b3JrIGF1dG9tYXRpb24gdGhy b3VnaCBwcm9ncmFtbWF0aWMgZXh0ZW5zaW9uLCB3aGlsZSBzdGlsbCBzdXBwb3J0aW5nIHN0YW5k YXJkIG1hbmFnZW1lbnQgaW50ZXJmYWNlcy4gT3BlbiB2U3dpdGNoIGNhbiBvcGVyYXRlIGJvdGgg YXMgYSBzb2Z0IHN3aXRjaCBydW5uaW5nIHdpdGhpbiB0aGUgaHlwZXJ2aXNvciwgYW5kIGFzIHRo ZSBjb250cm9sIHN0YWNrIGZvciBzd2l0Y2hpbmcgc2lsaWNvbi4gSXQgaGFzIGJlZW4gcG9ydGVk IHRvIG11bHRpcGxlIHZpcnR1YWxpemF0aW9uIHBsYXRmb3JtcyBhbmQgc3dpdGNoaW5nIGNoaXBz ZXRzLg0KDQoNCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f DQpkaXNjdXNzIG1haWxpbmcgbGlzdA0KZGlzY3Vzc0BvcGVudnN3aXRjaC5vcmc8bWFpbHRvOmRp c2N1c3NAb3BlbnZzd2l0Y2gub3JnPg0KaHR0cDovL29wZW52c3dpdGNoLm9yZy9tYWlsbWFuL2xp c3RpbmZvL2Rpc2N1c3MNCg== --_000_A7268EEA504A409787E2566289153076eucitrixcom_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PGh0bWw+PGhlYWQ+PG1ldGEgaHR0cC1lcXVpdj0iY29udGVudC10eXBlIiBjb250ZW50PSJ0ZXh0 L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPjwvaGVhZD48Ym9keSBkaXI9ImF1dG8iPjxkaXY+RllJLCBv cGVudnN3aXRjaCBub3cgaGFzIGV4cGVyaW1lbnRhbCBzdXBwb3J0IGZvciBvcGVuZmxvdyB2ZXJz aW9ucyAmZ3Q7IDEuMDxicj48YnI+LS0mbmJzcDs8ZGl2PkRhdmUgU2NvdHQ8L2Rpdj48ZGl2Plhl blNlcnZlciBTeXN0ZW0gQXJjaGl0ZWN0PC9kaXY+PC9kaXY+PGRpdj48YnI+QmVnaW4gZm9yd2Fy ZGVkIG1lc3NhZ2U6PGJyPjxicj48L2Rpdj48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48ZGl2Pjxi PkZyb206PC9iPiBKdXN0aW4gUGV0dGl0ICZsdDs8YSBocmVmPSJtYWlsdG86anBldHRpdEBuaWNp cmEuY29tIj5qcGV0dGl0QG5pY2lyYS5jb208L2E+Jmd0Ozxicj48Yj5EYXRlOjwvYj4gTWF5IDIs IDIwMTMsIDg6MzM6NDMgQU0gR01UKzAxOjAwPGJyPjxiPlRvOjwvYj4gIjxhIGhyZWY9Im1haWx0 bzpkaXNjdXNzQG9wZW52c3dpdGNoLm9yZyI+ZGlzY3Vzc0BvcGVudnN3aXRjaC5vcmc8L2E+IGRp c2N1c3MiICZsdDs8YSBocmVmPSJtYWlsdG86ZGlzY3Vzc0BvcGVudnN3aXRjaC5vcmciPmRpc2N1 c3NAb3BlbnZzd2l0Y2gub3JnPC9hPiZndDssICI8YSBocmVmPSJtYWlsdG86YW5ub3VuY2VAb3Bl bnZzd2l0Y2gub3JnIj5hbm5vdW5jZUBvcGVudnN3aXRjaC5vcmc8L2E+IiAmbHQ7PGEgaHJlZj0i bWFpbHRvOmFubm91bmNlQG9wZW52c3dpdGNoLm9yZyI+YW5ub3VuY2VAb3BlbnZzd2l0Y2gub3Jn PC9hPiZndDs8YnI+PGI+U3ViamVjdDo8L2I+IDxiPltvdnMtZGlzY3Vzc10gT3BlbiB2U3dpdGNo IDEuMTAuMCBBdmFpbGFibGU8L2I+PGJyPjxicj48L2Rpdj48L2Jsb2NrcXVvdGU+PGJsb2NrcXVv dGUgdHlwZT0iY2l0ZSI+PGRpdj48c3Bhbj5UaGUgT3BlbiB2U3dpdGNoIHRlYW0gaXMgcGxlYXNl ZCB0byBhbm5vdW5jZSB0aGUgcmVsZWFzZSBvZiBPcGVuIHZTd2l0Y2ggMS4xMC4wOjwvc3Bhbj48 YnI+PHNwYW4+PC9zcGFuPjxicj48c3Bhbj4gJm5ic3A7Jm5ic3A7Jm5ic3A7PGEgaHJlZj0iaHR0 cDovL29wZW52c3dpdGNoLm9yZy9yZWxlYXNlcy9vcGVudnN3aXRjaC0xLjEwLjAudGFyLmd6Ij5o dHRwOi8vb3BlbnZzd2l0Y2gub3JnL3JlbGVhc2VzL29wZW52c3dpdGNoLTEuMTAuMC50YXIuZ3o8 L2E+PC9zcGFuPjxicj48c3Bhbj48L3NwYW4+PGJyPjxzcGFuPlRoaXMgcmVsZWFzZSBjb250YWlu cyBuZXcgZmVhdHVyZXMsIHBlcmZvcm1hbmNlIGltcHJvdmVtZW50cywgYW5kIGEgc2lnbmlmaWNh bnQgcmV3b3JraW5nIG9mIGNvcmUgY29tcG9uZW50cy4gJm5ic3A7RmVhdHVyZSBoaWdobGlnaHRz IG9mIDEuMTAuMCBpbmNsdWRlOjwvc3Bhbj48YnI+PHNwYW4+PC9zcGFuPjxicj48c3Bhbj4gJm5i c3A7Jm5ic3A7Jm5ic3A7KiBTdXBwb3J0IGZvciB0aGUgVlhMQU4gdHVubmVsIHByb3RvY29sLjwv c3Bhbj48YnI+PHNwYW4+PC9zcGFuPjxicj48c3Bhbj4gJm5ic3A7Jm5ic3A7Jm5ic3A7KiBFeHBl cmltZW50YWwgc3VwcG9ydCBmb3IgbmV3ZXIgdmVyc2lvbnMgb2YgT3BlbkZsb3cuPC9zcGFuPjxi cj48c3Bhbj48L3NwYW4+PGJyPjxzcGFuPiAmbmJzcDsmbmJzcDsmbmJzcDsqIFdpdGggdGhlIExp bnV4IGRhdGFwYXRoLCBwYWNrZXRzIGZvciBuZXcgZmxvd3MgYXJlIG5vdyBxdWV1ZWQ8L3NwYW4+ PGJyPjxzcGFuPiAmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtzZXBhcmF0ZWx5IG9uIGEg cGVyLXBvcnQgYmFzaXMsIHNvIGl0IHNob3VsZCBubyBsb25nZXIgYmU8L3NwYW4+PGJyPjxzcGFu PiAmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtwb3NzaWJsZSBmb3IgYSBsYXJnZSBudW1i ZXIgb2YgbmV3IGZsb3dzIGFycml2aW5nIG9uIG9uZSBwb3J0IHRvPC9zcGFuPjxicj48c3Bhbj4g Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7cHJldmVudCBuZXcgZmxvd3MgZnJvbSBiZWlu ZyBwcm9jZXNzZWQgb24gb3RoZXIgcG9ydHMuPC9zcGFuPjxicj48c3Bhbj48L3NwYW4+PGJyPjxz cGFuPiAmbmJzcDsmbmJzcDsmbmJzcDsqIFBhdGNoIHBvcnRzIG5vIGxvbmdlciByZXF1aXJlIGtl cm5lbCBzdXBwb3J0LCBzbyB0aGV5IG5vdyB3b3JrPC9zcGFuPjxicj48c3Bhbj4gJm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7d2l0aCBGcmVlQlNEIGFuZCB0aGUga2VybmVsIG1vZHVsZSBi dWlsdCBpbnRvIExpbnV4IDMuMyBhbmQgbGF0ZXIuPC9zcGFuPjxicj48c3Bhbj48L3NwYW4+PGJy PjxzcGFuPiAmbmJzcDsmbmJzcDsmbmJzcDsqIFRoZSBtYXhpbXVtIHNpemUgb2YgdGhlIE1BQyBs ZWFybmluZyB0YWJsZSBpcyBub3cgY29uZmlndXJhYmxlLjwvc3Bhbj48YnI+PHNwYW4+PC9zcGFu Pjxicj48c3Bhbj4gJm5ic3A7Jm5ic3A7Jm5ic3A7KiBJdCBpcyBwb3NzaWJsZSB0byByZXF1ZXN0 IHRoZSBPcGVuRmxvdyBwb3J0IG51bWJlciB3aXRoIHRoZTwvc3Bhbj48YnI+PHNwYW4+ICZuYnNw OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyJvZnBvcnRfcmVxdWVzdCIgY29sdW1uIGluIHRoZSBJ bnRlcmZhY2UgdGFibGUuPC9zcGFuPjxicj48c3Bhbj48L3NwYW4+PGJyPjxzcGFuPiAmbmJzcDsm bmJzcDsmbmJzcDsqIEFuZCBtYW55IG90aGVycy4gJm5ic3A7U2VlIHRoZSBmdWxsIGNoYW5nZSBs b2cgaGVyZTo8L3NwYW4+PGJyPjxzcGFuPjwvc3Bhbj48YnI+PHNwYW4+ICZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOzxhIGhyZWY9Imh0dHA6Ly9vcGVudnN3aXRjaC5v cmcvcmVsZWFzZXMvTkVXUy0xLjEwLjAiPmh0dHA6Ly9vcGVudnN3aXRjaC5vcmcvcmVsZWFzZXMv TkVXUy0xLjEwLjA8L2E+PC9zcGFuPjxicj48c3Bhbj48L3NwYW4+PGJyPjxzcGFuPkVuam95ITwv c3Bhbj48YnI+PHNwYW4+PC9zcGFuPjxicj48c3Bhbj4tLVRoZSBPcGVuIHZTd2l0Y2ggVGVhbTwv c3Bhbj48YnI+PHNwYW4+PC9zcGFuPjxicj48c3Bhbj4tLS0tLS0tLS0tLS0tLS0tLS0tLTwvc3Bh bj48YnI+PHNwYW4+T3BlbiB2U3dpdGNoIGlzIGEgcHJvZHVjdGlvbiBxdWFsaXR5LCBtdWx0aWxh eWVyIG9wZW4gc291cmNlIHZpcnR1YWwgc3dpdGNoLiAmbmJzcDtJdCBpcyBkZXNpZ25lZCB0byBl bmFibGUgbWFzc2l2ZSBuZXR3b3JrIGF1dG9tYXRpb24gdGhyb3VnaCBwcm9ncmFtbWF0aWMgZXh0 ZW5zaW9uLCB3aGlsZSBzdGlsbCBzdXBwb3J0aW5nIHN0YW5kYXJkIG1hbmFnZW1lbnQgaW50ZXJm YWNlcy4gT3BlbiB2U3dpdGNoIGNhbiBvcGVyYXRlIGJvdGggYXMgYSBzb2Z0IHN3aXRjaCBydW5u aW5nIHdpdGhpbiB0aGUgaHlwZXJ2aXNvciwgYW5kIGFzIHRoZSBjb250cm9sIHN0YWNrIGZvciBz d2l0Y2hpbmcgc2lsaWNvbi4gSXQgaGFzIGJlZW4gcG9ydGVkIHRvIG11bHRpcGxlIHZpcnR1YWxp emF0aW9uIHBsYXRmb3JtcyBhbmQgc3dpdGNoaW5nIGNoaXBzZXRzLjwvc3Bhbj48YnI+PHNwYW4+ PC9zcGFuPjxicj48c3Bhbj48L3NwYW4+PGJyPjxzcGFuPl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fPC9zcGFuPjxicj48c3Bhbj5kaXNjdXNzIG1haWxpbmcg bGlzdDwvc3Bhbj48YnI+PHNwYW4+PGEgaHJlZj0ibWFpbHRvOmRpc2N1c3NAb3BlbnZzd2l0Y2gu b3JnIj5kaXNjdXNzQG9wZW52c3dpdGNoLm9yZzwvYT48L3NwYW4+PGJyPjxzcGFuPjxhIGhyZWY9 Imh0dHA6Ly9vcGVudnN3aXRjaC5vcmcvbWFpbG1hbi9saXN0aW5mby9kaXNjdXNzIj5odHRwOi8v b3BlbnZzd2l0Y2gub3JnL21haWxtYW4vbGlzdGluZm8vZGlzY3VzczwvYT48L3NwYW4+PGJyPjwv ZGl2PjwvYmxvY2txdW90ZT48L2JvZHk+PC9odG1sPg== --_000_A7268EEA504A409787E2566289153076eucitrixcom_-- From daniel.buenzli@erratique.ch Thu May 02 09:39:01 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXp2b-0004fV-Cu (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 09:39:01 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1477933 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail6.webfaction.com ([74.55.86.74]:56559 helo=smtp.webfaction.com) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UXp2a-0004GJ-1G (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 09:39:01 +0100 Received: from [172.20.10.2] (92.40.254.83.threembb.co.uk [92.40.254.83]) by smtp.webfaction.com (Postfix) with ESMTP id 0264A20EB8D5; Thu, 2 May 2013 08:38:57 +0000 (UTC) Date: Thu, 2 May 2013 09:38:53 +0100 From: =?utf-8?Q?Daniel_B=C3=BCnzli?= To: Anil Madhavapeddy Message-ID: <446AC07ECF9E4F538C253AE1F964209B@erratique.ch> In-Reply-To: <251A6A69-DB78-4598-AF34-0CE39B2CE8F6@recoil.org> References: <27C5F743E5AA481BB4A16B403F1842A3@erratique.ch> <4F6CF319-0E99-4CF3-A65D-993B26E7F77E@nottingham.ac.uk> <59296051-131D-425E-B59F-24F97CF4A51E@nottingham.ac.uk> <4ABA2037-0D63-460A-8CCC-202E0682745D@nottingham.ac.uk> <251A6A69-DB78-4598-AF34-0CE39B2CE8F6@recoil.org> Subject: Re: mirage + froc =?utf-8?Q?=3D_?=self-scaling? X-Mailer: sparrow 1.6.4 (build 1178) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Cc: Ashish Agarwal , Richard Mortier , Raphael Proust , "=?utf-8?Q?cl-mirage=40lists.cam.ac.uk_List?=" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 08:39:01 -0000 Content-Length: 3258 Lines: 67 Le mercredi, 1 mai 2013 =C3=A0 21:07, Anil Madhavapeddy a =C3=A9crit : > Just to pick up on this point, the issue is that I think we don't reall= y know if =46RP can actually be used at scale. Or at least, noone has tri= ed yet in OCaml (some of the Javascript versions are reasonably well used= for UI reactive programming). To be honest I don't see why it couldn't actually scale. Sure it's going = to be slower than pure imperative programming, the datastructure tracking= the dependencies is not free -- but still even that is not entirely clea= r since frp may mean that you compute less because of the very fine depen= dency tracking. =20 However if you actually need an observer mechanism in your system, I woul= d highly suggest that you use frp, as the na=C3=AFve OO way of doing this= is usually unsound with what you want --- glitches, needless computation= and too coarse dependency tracking that means that you have to program d= efensively against spurious updates (recheck the state of the system befo= re proceeding when hollywood calls you). One thing to keep in mind though is that =46RP is really a different kind= of programming, it's dataflow programming and you really need a differen= t mindset (see e.g. the example in this presentation slides where we are = interested in defining the =60elapsed=60 variable of a countdown ui =5B1=5D= ). But at least you program with values that are well defined and composi= tional, not some kind of reference mutated by a sour callback soup. =20 On my side because of my work on Vg, I have been thinking a little bit ab= out React w.r.t. to frp's original motivation -- functional reactive anim= ation =5B1=5D (animation =3D Vg.image React.S.t). And I think React may n= eed to evolve to support that idea in a non toy manner. =20 The problem is that React has the premise that there can be no two *primi= tive* signals/events (inputs to the system) that happen at the same insta= ntaneous time. Each change to a primitive triggers an update cycle that y= ou have to consider atomically. In animation you are usually not interest= ed in updating that image signal beyond 60Hz, so if you change say three = colors of a picture between two frames you don't actually want to recompu= te trice the picture. Now you can circumvent this problem within react itself: let on=5Fframe, redraw =3D E.create () =20 let resample prim =3D E.hold (S.value prim) (E.sample on=5Fframe prim) =20 but it feels artificial, you'd have to =60resample=60 all your primitives= before using them to define your image, and could become inefficient if = you have many primitives. =20 The good news is that this limitation is purely artificial and react can = perfectly be adapted to support simultaneous primitive updates, I just ne= ed to devise an api that exposes an abstract type to some parts of react'= s internal mechanism. But again I need to think more about this and write= code. =20 This with the other goal I have to make React leak less in the browser an= d change the signature of the signal switching combinator could lead to a= n (only slightly) incompatible 1.0 release or React. =20 Best, Daniel =5B1=5D http://erratique.ch/talks/ocamlum-2010 =5B2=5D http://conal.net/papers/icfp97/ From raphlalou@gmail.com Thu May 02 10:03:51 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXpQd-0006RF-4L (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 10:03:51 +0100 X-Cam-SpamDetails: score -0.6 from SpamAssassin-3.3.2-1477933 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.210.169 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (raphlalou[at]gmail.com) * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ia0-f169.google.com ([209.85.210.169]:33144) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UXpQb-0006Dc-0X (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 10:03:51 +0100 Received: by mail-ia0-f169.google.com with SMTP id l29so295399iag.14 for ; Thu, 02 May 2013 02:03:47 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.50.112.103 with SMTP id ip7mr3316930igb.77.1367485427796; Thu, 02 May 2013 02:03:47 -0700 (PDT) Sender: raphlalou@gmail.com Received: by 10.42.41.131 with HTTP; Thu, 2 May 2013 02:03:47 -0700 (PDT) In-Reply-To: <446AC07ECF9E4F538C253AE1F964209B@erratique.ch> References: <27C5F743E5AA481BB4A16B403F1842A3@erratique.ch> <4F6CF319-0E99-4CF3-A65D-993B26E7F77E@nottingham.ac.uk> <59296051-131D-425E-B59F-24F97CF4A51E@nottingham.ac.uk> <4ABA2037-0D63-460A-8CCC-202E0682745D@nottingham.ac.uk> <251A6A69-DB78-4598-AF34-0CE39B2CE8F6@recoil.org> <446AC07ECF9E4F538C253AE1F964209B@erratique.ch> Date: Thu, 2 May 2013 10:03:47 +0100 X-Google-Sender-Auth: MMC7aPv8OPYVe3QCnjHVvR04AMs Message-ID: Subject: Re: mirage + froc = self-scaling? From: Raphael Proust To: =?UTF-8?Q?Daniel_B=C3=BCnzli?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Ashish Agarwal , Richard Mortier , "cl-mirage@lists.cam.ac.uk List" , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 09:03:51 -0000 Content-Length: 2777 Lines: 83 On Thu, May 2, 2013 at 9:38 AM, Daniel B=C3=BCnzli wrote: > Le mercredi, 1 mai 2013 =C3=A0 21:07, Anil Madhavapeddy a =C3=A9crit : >> [..] > [..] > The problem is that React has the premise that there can be no two *primi= tive* signals/events (inputs to the system) that happen at the same instant= aneous time. Each change to a primitive triggers an update cycle that you h= ave to consider atomically. In animation you are usually not interested in = updating that image signal beyond 60Hz, so if you change say three colors o= f a picture between two frames you don't actually want to recompute trice t= he picture. > > Now you can circumvent this problem within react itself: > > let on_frame, redraw =3D E.create () > let resample prim =3D E.hold (S.value prim) (E.sample on_frame prim) > > but it feels artificial, you'd have to `resample` all your primitives bef= ore using them to define your image, and could become inefficient if you ha= ve many primitives. > > The good news is that this limitation is purely artificial and react can = perfectly be adapted to support simultaneous primitive updates, I just need= to devise an api that exposes an abstract type to some parts of react's in= ternal mechanism. But again I need to think more about this and write code. The way froc handles this is not completely satisfactory. It doesn't behave well in concurrent programs. In froc, one can register new values for as many primitives as one wants and then call a `sync` function. I don't remember the exact names for the functions but basically it looks like: ~~~~~~~ create: unit -> (('a -> unit) * 'a t) (* the returned closure does not trigger an update cycle *) sync: unit -> unit (* triggers an update cycle *) ~~~~~~~ Good thing one can do: - erase stall values before they were propagated in the dependency graph, - change several values at the same time Bad things that can happen: - another thread might erase values you registered, - another thread might start an update cycle before you registered all the values you wanted What I'd like it to look like: ~~~~~~~ create: unit -> ('a pusher * 'a t) write: (\exist 'a . ('a * 'a pusher)) list -> unit (* push several values on several primitive signals instantaneously *) ~~~~~~~ Good thing one can do: - erase stall values before pushing (i.e. prepare the push-list by both appending and removing/replacing entries), - change several values at the same time Bad thing one can do: - get confused with existential types (Although existential types are not really needed. ~~~~~~~ create: unit -> ('a pusher * 'a t) type poly_pusher val poly: 'a pusher -> 'a -> poly_pusher val sync: poly_pusher list -> unit ~~~~~~~ ) Ciao, --=20 ______________ Rapha=C3=ABl Proust From daniel.buenzli@erratique.ch Thu May 02 11:15:07 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXqXb-0003Li-9x (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 11:15:07 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1477933 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail6.webfaction.com ([74.55.86.74]:39428 helo=smtp.webfaction.com) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UXqXa-0007Yt-7y (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 11:15:07 +0100 Received: from [10.80.118.32] (firewall.ctxuk.citrix.com [46.33.159.2]) by smtp.webfaction.com (Postfix) with ESMTP id A838F20EBCC4; Thu, 2 May 2013 10:15:04 +0000 (UTC) Date: Thu, 2 May 2013 11:15:00 +0100 From: =?utf-8?Q?Daniel_B=C3=BCnzli?= To: Raphael Proust Message-ID: <78049995E85B43BBB0A4889ADB371BFA@erratique.ch> In-Reply-To: References: <27C5F743E5AA481BB4A16B403F1842A3@erratique.ch> <4F6CF319-0E99-4CF3-A65D-993B26E7F77E@nottingham.ac.uk> <59296051-131D-425E-B59F-24F97CF4A51E@nottingham.ac.uk> <4ABA2037-0D63-460A-8CCC-202E0682745D@nottingham.ac.uk> <251A6A69-DB78-4598-AF34-0CE39B2CE8F6@recoil.org> <446AC07ECF9E4F538C253AE1F964209B@erratique.ch> Subject: Re: mirage + froc =?utf-8?Q?=3D_?=self-scaling? X-Mailer: sparrow 1.6.4 (build 1178) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Ashish Agarwal , Richard Mortier , "=?utf-8?Q?cl-mirage=40lists.cam.ac.uk_List?=" , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 10:15:07 -0000 Content-Length: 2583 Lines: 41 Thanks for your insights. > What I'd like it to look like: > > ~~~~~~~ > create: unit -> ('a pusher * 'a t) > write: (\exist 'a . ('a * 'a pusher)) list -> unit (* push several > values on several primitive signals instantaneously *) > ~~~~~~~ Actually I was more going towards that kind interface. The thing is that React, contrary to Froc doesn't use global data structures, so I was for sure not going to have a froc-like interface. The data structure used during and update cycle needs to be witnessed somewhere. So the idea was to expose a `step` type that represents an update cycle: module Step : sig type t create : unit -> step perform : step -> unit end val S.create : unit -> 'a signal * (?step:Step.t -> 'a -> unit) with the idea that if you use the ~step arguments is schedules the value of the primitive to be one provided on this step. This may not be optimal maybe Step should be a "Steps", so that you can definitively plugin a Steps structure on the primitive setter and forget. This could also be an opportunity to build in a mechanism for primitive update from the update cycle -- so far I always told people to put the updates in a queue and dequeue after the step. In fact if the signal setter function no longer automatically create an update cycle that is we change the signal setter signature to a concrete Step.t -> 'a -> unit, this becomes trivial. But I need to think about what happens with fixed point operators, as those automatically create steps. It may also make libraries less compositional (if they impose their step). Regarding the thread-safety rule [1] I think it would still be rather simple: Let uset(s) be the set of events and signals *including the primitive ones* that need to be updated whenever `Step.perform s` is invoked. Performing two steps s and s' concurrently is only allowed if uset(s) and uset(s') are disjoint, otherwise the steps executions must be serialized. So maybe the idea of react that the setter function automatically creates an update cycle was a bad one. Having an explicit steps type to drive the system may be better. But it's a change I need to get right, so I won't rush. I'm however very interested in talking in person with you Raphael (or anybody who has some experience or toughts about frp in Cambridge) about this during the following two months. Best, Daniel P.S. This becomes a little bit OT for mirage maybe we should try to find another venue to discuss these things. Maybe an ocaml-frp mailing list ? [1] http://erratique.ch/software/react/doc/React#update From agarwal1975@gmail.com Thu May 02 12:44:07 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXrvj-0007Iv-Sf (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 12:44:07 +0100 X-Cam-SpamDetails: score 0.9 from SpamAssassin-3.3.2-1477933 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.210.178 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (agarwal1975[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in * digit (agarwal1975[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 1.2 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ia0-f178.google.com ([209.85.210.178]:51685) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UXrvj-0004vq-6Z (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 12:44:07 +0100 Received: by mail-ia0-f178.google.com with SMTP id t29so379658iag.37 for ; Thu, 02 May 2013 04:44:06 -0700 (PDT) X-Received: by 10.50.22.3 with SMTP id z3mr3658275ige.32.1367495046278; Thu, 02 May 2013 04:44:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.56.7 with HTTP; Thu, 2 May 2013 04:43:46 -0700 (PDT) In-Reply-To: <78049995E85B43BBB0A4889ADB371BFA@erratique.ch> References: <27C5F743E5AA481BB4A16B403F1842A3@erratique.ch> <4F6CF319-0E99-4CF3-A65D-993B26E7F77E@nottingham.ac.uk> <59296051-131D-425E-B59F-24F97CF4A51E@nottingham.ac.uk> <4ABA2037-0D63-460A-8CCC-202E0682745D@nottingham.ac.uk> <251A6A69-DB78-4598-AF34-0CE39B2CE8F6@recoil.org> <446AC07ECF9E4F538C253AE1F964209B@erratique.ch> <78049995E85B43BBB0A4889ADB371BFA@erratique.ch> From: Ashish Agarwal Date: Thu, 2 May 2013 07:43:46 -0400 Message-ID: Subject: Re: mirage + froc = self-scaling? To: =?ISO-8859-1?Q?Daniel_B=FCnzli?= Content-Type: multipart/alternative; boundary=047d7b10caff7781b904dbbac0aa Cc: Richard Mortier , Raphael Proust , "cl-mirage@lists.cam.ac.uk List" , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 11:44:07 -0000 Content-Length: 7387 Lines: 187 --047d7b10caff7781b904dbbac0aa Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable My original suggestion was to summarize concepts like FRP and write tutorials about them on ocaml.org. Whether FRP scales or not is unimportant. Maybe it doesn't, but still FRP is a widely discussed concept and there are multiple OCaml libraries supporting it. Thus, I think ocaml.org could include content about it. Yes.. this is way off topic. The main OCaml List is sufficient for this topic. On Thu, May 2, 2013 at 6:15 AM, Daniel B=FCnzli wrote: > Thanks for your insights. > > > What I'd like it to look like: > > > > ~~~~~~~ > > create: unit -> ('a pusher * 'a t) > > write: (\exist 'a . ('a * 'a pusher)) list -> unit (* push several > > values on several primitive signals instantaneously *) > > ~~~~~~~ > > > Actually I was more going towards that kind interface. The thing is that > React, contrary to Froc doesn't use global data structures, so I was for > sure not going to have a froc-like interface. The data structure used > during and update cycle needs to be witnessed somewhere. So the idea was = to > expose a `step` type that represents an update cycle: > > module Step : sig > type t > create : unit -> step > perform : step -> unit > end > > val S.create : unit -> 'a signal * (?step:Step.t -> 'a -> unit) > > with the idea that if you use the ~step arguments is schedules the value > of the primitive to be one provided on this step. This may not be optimal > maybe Step should be a "Steps", so that you can definitively plugin a Ste= ps > structure on the primitive setter and forget. > > This could also be an opportunity to build in a mechanism for primitive > update from the update cycle -- so far I always told people to put the > updates in a queue and dequeue after the step. In fact if the signal sett= er > function no longer automatically create an update cycle that is we change > the signal setter signature to a concrete Step.t -> 'a -> unit, this > becomes trivial. But I need to think about what happens with fixed point > operators, as those automatically create steps. It may also make librarie= s > less compositional (if they impose their step). > > Regarding the thread-safety rule [1] I think it would still be rather > simple: Let uset(s) be the set of events and signals *including the > primitive ones* that need to be updated whenever `Step.perform s` is > invoked. Performing two steps s and s' concurrently is only allowed if > uset(s) and uset(s') are disjoint, otherwise the steps executions must be > serialized. > > So maybe the idea of react that the setter function automatically creates > an update cycle was a bad one. Having an explicit steps type to drive the > system may be better. > > But it's a change I need to get right, so I won't rush. I'm however very > interested in talking in person with you Raphael (or anybody who has some > experience or toughts about frp in Cambridge) about this during the > following two months. > > Best, > > Daniel > > P.S. This becomes a little bit OT for mirage maybe we should try to find > another venue to discuss these things. Maybe an ocaml-frp mailing list ? > > [1] http://erratique.ch/software/react/doc/React#update > > > --047d7b10caff7781b904dbbac0aa Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable My original suggestion was to summarize concepts like FRP and write tutoria= ls about them on ocaml.org. Whether FRP sc= ales or not is unimportant. Maybe it doesn't, but still FRP is a widely= discussed concept and there are multiple OCaml libraries supporting it. Th= us, I think ocaml.org could include conten= t about it.

Yes.. this is way off topic. The main OCaml List is sufficie= nt for this topic.


On Thu, May= 2, 2013 at 6:15 AM, Daniel B=FCnzli <daniel.buenzli@erratique.c= h> wrote:
Thanks for your insights.

> What I'd like it to look like:
>
> ~~~~~~~
> create: unit -> ('a pusher * 'a t)
> write: (\exist 'a . ('a * 'a pusher)) list -> unit (* p= ush several
> values on several primitive signals instantaneously *)
> ~~~~~~~


Actually I was more going towards that kind interface. The thing is t= hat React, contrary to Froc doesn't use global data structures, so I wa= s for sure not going to have a froc-like interface. The data structure used= during and update cycle needs to be witnessed somewhere. So the idea was t= o expose a `step` type that represents an update cycle:

=A0 module Step : sig
=A0 =A0 type t
=A0 =A0 create : unit -> step
=A0 =A0 perform : step =A0-> unit
=A0 end

=A0 val S.create : unit -> 'a signal * (?step:Step.t -> 'a -&= gt; unit)

with the idea that if you use the ~step arguments is schedules the value of= the primitive to be one provided on this step. This may not be optimal may= be Step should be a "Steps", so that you can definitively plugin = a Steps structure on the primitive setter and forget.

This could also be an opportunity to build in a mechanism for primitive upd= ate from the update cycle -- so far I always told people to put the updates= in a queue and dequeue after the step. In fact if the signal setter functi= on no longer automatically create an update cycle that is we change the sig= nal setter signature to a concrete Step.t -> 'a -> unit, this bec= omes trivial. But I need to think about what happens with fixed point opera= tors, as those automatically create steps. It may also make libraries less = compositional (if they impose their step).

Regarding the thread-safety rule [1] I think it would still be rather simpl= e: Let uset(s) be the set of events and signals *including the primitive on= es* that need to be updated whenever `Step.perform s` is invoked. Performin= g two steps s and s' concurrently is only allowed if uset(s) and uset(s= ') are disjoint, otherwise the steps executions must be serialized.

So maybe the idea of react that the setter function automatically creates a= n update cycle was a bad one. Having an explicit steps type to drive the sy= stem may be better.

But it's a change I need to get right, so I won't rush. I'm how= ever very interested in talking in person with you Raphael (or anybody who = has some experience or toughts about frp in Cambridge) about this during th= e following two months.

Best,

Daniel

P.S. This becomes a little bit OT for mirage maybe we should try to find an= other venue to discuss these things. Maybe an ocaml-frp mailing list ?

[1] http://erratique.ch/software/react/doc/React#update



--047d7b10caff7781b904dbbac0aa-- From pmundkur.ocaml@gmail.com Thu May 02 18:41:12 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UXxVI-0004iI-LZ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 18:41:12 +0100 X-Cam-SpamDetails: score 0.6 from SpamAssassin-3.3.2-1477933 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.220.42 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (pmundkur.ocaml[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 1.2 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-pa0-f42.google.com ([209.85.220.42]:38035) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UXxVH-0005yF-1h (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 02 May 2013 18:41:12 +0100 Received: by mail-pa0-f42.google.com with SMTP id bj3so478555pad.1 for ; Thu, 02 May 2013 10:41:10 -0700 (PDT) X-Received: by 10.68.232.42 with SMTP id tl10mr10040772pbc.72.1367516470585; Thu, 02 May 2013 10:41:10 -0700 (PDT) Received: from damage (visnet-156.csl.sri.com. [130.107.98.156]) by mx.google.com with ESMTPSA id em2sm8151621pbb.0.2013.05.02.10.41.08 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Thu, 02 May 2013 10:41:09 -0700 (PDT) Date: Thu, 2 May 2013 10:41:06 -0700 From: Prashanth Mundkur To: Dave Scott Subject: Re: Fwd: [ovs-discuss] Open vSwitch 1.10.0 Available Message-ID: <20130502174106.GA17287@damage> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 17:41:12 -0000 Content-Length: 1236 Lines: 30 On 09:08 Thu 02 May, Dave Scott wrote: > FYI, openvswitch now has experimental support for openflow versions > > 1.0 Interesting, thanks. There is more information here: http://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob;f=FAQ;h=1f0f8414cf6bb8508badfdfd71bc62dde28d6055;hb=HEAD#l931 Q: What versions of OpenFlow does Open vSwitch support? A: Open vSwitch 1.9 and earlier support only OpenFlow 1.0 (plus extensions that bring in many of the features from later versions of OpenFlow). Open vSwitch versions 1.10 and later will have experimental support for OpenFlow 1.2 and 1.3. On these versions of Open vSwitch, the following command enables OpenFlow 1.0, 1.2, and 1.3 on bridge br0: ovs-vsctl set bridge br0 protocols=OpenFlow10,OpenFlow12,OpenFlow13 Support for OpenFlow 1.1 is incomplete enough that it cannot yet be enabled, even experimentally. Support for OpenFlow 1.2 and 1.3 is still incomplete. Work to be done is tracked in OPENFLOW-1.1+ in the Open vSwitch source tree (also via http://openvswitch.org/development/openflow-1-x-plan/). When support for a given OpenFlow version is solidly implemented, Open vSwitch will enable that version by default. From stedolan@stedolan.net Fri May 03 12:39:49 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UYEL7-0004SF-Lu (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 03 May 2013 12:39:49 +0100 X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1478301 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.215.46 listed in list.dnswl.dnsbl.ja.net] * 0.0 HTML_MESSAGE BODY: HTML included in message X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-la0-f46.google.com ([209.85.215.46]:34885) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UYEL6-0006dC-Ey (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 03 May 2013 12:39:49 +0100 Received: by mail-la0-f46.google.com with SMTP id fk20so1432925lab.19 for ; Fri, 03 May 2013 04:39:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:x-originating-ip:in-reply-to:references :date:message-id:subject:from:to:cc:content-type:x-gm-message-state; bh=86uOH2BU5iQv/C5mV98ZBcDfnTUJ8XYpN4TQmd400ao=; b=CS18bCqSow2lkgTnAmO+B3q0uCDphGlVDI9WyzOmmPVPtZ/ti/eeWQWi43EZ3NC3EH 9jscFE0s2JxPx/bBP5+CfRFn2E7H9h1rFxrCY+Qyuf60AMSkAsPdH6I/xBBEO8bbAwpd gI5sk7THXuN5sihV2HkrZDOFvinoeDORSeuqN2bh5BZ52plwv6zal4CtbVUUx7L8sJVv 3LP9j6dVOQPs8RXaCCPeLteSHBc8jtqheorGBhbBuLluOmQ0PUjhgAIc10V1ZOi81VkM Yba06iPPk7ceJ+N4Xml5KTOAaYeu7k1emWjQCKcT2FGz5opZasLIWfT09vadHWKO+DXh OQAQ== MIME-Version: 1.0 X-Received: by 10.112.151.3 with SMTP id um3mr499583lbb.102.1367581187999; Fri, 03 May 2013 04:39:47 -0700 (PDT) Received: by 10.114.160.136 with HTTP; Fri, 3 May 2013 04:39:47 -0700 (PDT) X-Originating-IP: [94.116.119.52] In-Reply-To: <9BA486E2-18A4-414B-BC41-68FD73DE2781@recoil.org> References: <5181591B.1070004@luminar.eu.org> <58A3F02B-BA98-4C45-ADAC-06C1D3309402@recoil.org> <9BA486E2-18A4-414B-BC41-68FD73DE2781@recoil.org> Date: Fri, 3 May 2013 12:39:47 +0100 Message-ID: Subject: Re: https://polarssl.org/ From: Stephen Dolan To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=047d7b343c2ae99a6604dbcece7e X-Gm-Message-State: ALoCoQmJcNgZkl4UJg8wqkgpl4vPjtL8CwXfXzoeVKY/pAksjGD/7McKw5tIkoCAGmifUzCv7uPW Cc: David Sheets , Dave Scott , Mirage List , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2013 11:39:49 -0000 Content-Length: 2766 Lines: 67 --047d7b343c2ae99a6604dbcece7e Content-Type: text/plain; charset=ISO-8859-1 Incidentally, PolarSSL is a fork of the (now unmaintained) BSD/GPL dual-licensed XySSL. Not that I'm actually recommending it; the thought of an unmaintained SSL library fills me with dread. Stephen On Wed, May 1, 2013 at 9:25 PM, Anil Madhavapeddy wrote: > On 1 May 2013, at 20:09, Dave Scott wrote: > > We've talked a bit about SSL offload before (particularly to hardware). > We could link one of these libs with minios and make a C SSL proxy using > libvchan. Or we could a concept of a "process" to mirage -- perhaps one > static process per vCPU -- and use inter process communication to > communicate with the ocaml code. > > I'd really like to keep the SSL bindings outside of the main OCaml > process, so this is a good place to start doing so. Also, it's a natural > tie-in to the actor library, as those communication channels are intended > to either be vchan (xenserver) or internal tcp (EC2, where we cant vchan). > > -anil > > > --047d7b343c2ae99a6604dbcece7e Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Incidentally, PolarSSL is a fork of the (now unmaintained)= BSD/GPL dual-licensed XySSL. Not that I'm actually recommending it; th= e thought of an unmaintained SSL library fills me with dread.

Stephen


On Wed, May 1, 2013 at 9:25 PM, Anil Madhavapeddy <anil@recoil.or= g> wrote:
On 1 May 2013, at 20:09, D= ave Scott <Dave.Scott@eu.cit= rix.com> wrote:
> We've talked a bit about SSL offload before (particularly to hardw= are). We could link one of these libs with minios and make a C SSL proxy us= ing libvchan. Or we could a concept of a "process" to mirage -- p= erhaps one static process per vCPU -- and use inter process communication t= o communicate with the ocaml code.

I'd really like to keep the SSL bindings outside of the main OCam= l process, so this is a good place to start doing so. =A0Also, it's a n= atural tie-in to the actor library, as those communication channels are int= ended to either be vchan (xenserver) or internal tcp (EC2, where we cant vc= han).

-anil



--047d7b343c2ae99a6604dbcece7e-- From anil@recoil.org Fri May 03 13:47:49 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UYFOv-0006qU-3J (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 03 May 2013 13:47:49 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1478301 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:26843 helo=dark.recoil.org) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UYFOu-0002ca-ht (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 03 May 2013 13:47:49 +0100 Received: (qmail 23841 invoked by uid 634); 3 May 2013 12:47:48 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED,TVD_RCVD_IP X-Spam-Check-By: dark.recoil.org Received: from 207-38-128-64.c3-0.avec-ubr2.nyr-avec.ny.cable.rcn.com (HELO [192.168.1.125]) (207.38.128.64) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 03 May 2013 13:47:47 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: https://polarssl.org/ From: Anil Madhavapeddy In-Reply-To: Date: Fri, 3 May 2013 08:47:47 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5181591B.1070004@luminar.eu.org> <58A3F02B-BA98-4C45-ADAC-06C1D3309402@recoil.org> <9BA486E2-18A4-414B-BC41-68FD73DE2781@recoil.org> To: Stephen Dolan X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: David Sheets , Dave Scott , Mirage List , Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2013 12:47:49 -0000 Content-Length: 1225 Lines: 35 I notice that XySSL is used in Adobe Flash. I can see how this is going = to play out... http://xkcd.com/424/ -anil On 3 May 2013, at 07:39, Stephen Dolan wrote: > Incidentally, PolarSSL is a fork of the (now unmaintained) BSD/GPL = dual-licensed XySSL. Not that I'm actually recommending it; the thought = of an unmaintained SSL library fills me with dread. >=20 > Stephen >=20 >=20 > On Wed, May 1, 2013 at 9:25 PM, Anil Madhavapeddy = wrote: > On 1 May 2013, at 20:09, Dave Scott wrote: > > We've talked a bit about SSL offload before (particularly to = hardware). We could link one of these libs with minios and make a C SSL = proxy using libvchan. Or we could a concept of a "process" to mirage -- = perhaps one static process per vCPU -- and use inter process = communication to communicate with the ocaml code. >=20 > I'd really like to keep the SSL bindings outside of the main OCaml = process, so this is a good place to start doing so. Also, it's a = natural tie-in to the actor library, as those communication channels are = intended to either be vchan (xenserver) or internal tcp (EC2, where we = cant vchan). >=20 > -anil >=20 >=20 >=20 From anil@recoil.org Fri May 03 14:45:47 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UYGJ1-0000vt-S4 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 03 May 2013 14:45:47 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1478301 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:12124 helo=dark.recoil.org) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1UYGJ0-0006bb-9d (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 03 May 2013 14:45:47 +0100 Received: (qmail 2981 invoked by uid 634); 3 May 2013 13:45:46 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED,TVD_RCVD_IP X-Spam-Check-By: dark.recoil.org Received: from 207-38-128-64.c3-0.avec-ubr2.nyr-avec.ny.cable.rcn.com (HELO [192.168.1.125]) (207.38.128.64) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 03 May 2013 14:45:37 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Reminder: Mirage weekly call - 30 Apr at 1600 BST From: Anil Madhavapeddy In-Reply-To: Date: Fri, 3 May 2013 09:45:36 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <3547678E-4F38-4F89-99F3-2DBFB1D4D088@recoil.org> References: <49CADCC7-33A1-4E43-B51E-2DF266E50E0A@gmail.com> <9E08CA48-929E-4BB9-88C6-312B379A088A@recoil.org> To: "cl-mirage@lists.cam.ac.uk List" , Jon Ludlam X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2013 13:45:47 -0000 Content-Length: 7965 Lines: 240 The minutes are now online at: = http://www.openmirage.org/wiki/weekly-2013-04-30 Amir and I'll be doing a blog post soon on the OSCon talk and Xen.org = incubation, so I'll link to these from there (even though they're quite = raw, better than nothing). Vincent/Dave: perhaps a Mirari blog post once the EC2 bindings work, = too? The call next week is cancelled due to several of us travelling next = week in the US, but will be resumed as normal on the 14th May! -anil On 30 Apr 2013, at 04:41, Anil Madhavapeddy wrote: > The Mirage weekly call is today. Dave will be over at the CL so we'll = call in from there, so Citrix folk may want to find their own meeting = room in 101 rather than hunting for Dave. >=20 > Agenda: > - Moving devices drivers out into userspace (Dave) > - Cross-compiling MatrixSSL (if Jon is available) > - Js_of_ocaml storage (Anil) > - Actor libraries and the new wg-parallel working group (Anil) > - Any other updates >=20 > Thomas is on vacation this week, so he won't be dialing in. >=20 > -- Go To Meeting details -- >=20 > You can either use the online GoToMeeting or dial in via the numbers = below: >=20 > 1. Please join my meeting. > https://www1.gotomeeting.com/join/211547328 >=20 > 2. Use your microphone and speakers (VoIP) - a headset is = recommended. Or, call in using your telephone. >=20 > United States: +1 (224) 649-0001 > Argentina (toll-free): 0 800 266 1382 > Australia (toll-free): 1 800 193 385 > Australia: +61 2 8355 1020 > Austria (toll-free): 0 800 202148 > Austria: +43 (0) 7 2088 1047 > Bahrain (toll-free): 800 81 111 > Belarus (toll-free): 8 820 0011 0214 > Belgium (toll-free): 0 800 26116 > Belgium: +32 (0) 28 08 4368 > Brazil (toll-free): 0 800 047 4906 > Canada (toll-free): 1 888 455 1389 > Canada: +1 (647) 497-9353 > China (toll-free): 4008 811084 > Czech Republic (toll-free): 800 500448 > Denmark (toll-free): 8090 1924 > Denmark: +45 (0) 69 91 89 28 > Finland (toll-free): 80094507 > Finland: +358 (0) 942 59 7850 > France (toll-free): 0 805 541 047 > France: +33 (0) 170 950 594 > Germany (toll-free): 0 800 723 5270 > Germany: +49 (0) 811 8899 6903 > Hong Kong (toll-free): 30713169 > Iceland (toll-free): 800 9869 > India (toll-free): 000 800 100 7855 > Indonesia (toll-free): 007 803 011 0395 > Ireland (toll-free): 1 800 946 538 > Ireland: +353 (0) 19 030 010 > Israel (toll-free): 1 809 212 875 > Italy (toll-free): 800 906959 > Italy: +39 0 247 92 13 01 > Japan (toll-free): 00 120 663 800 > Korea, Republic of (toll-free): 806150880 > Luxembourg (toll-free): 800 22104 > Malaysia (toll-free): 1 800 81 5373 > Mexico (toll-free): 01 800 925 0372 > Netherlands (toll-free): 0 800 265 8469 > Netherlands: +31 (0) 208 080 219 > New Zealand (toll-free): 0 800 45 2202 > New Zealand: +64 (0) 9 280 6302 > Norway (toll-free): 800 30 257 > Norway: +47 75 80 32 07 > Panama (toll-free): 00 800 226 8832 > Peru (toll-free): 0 800 54682 > Philippines (toll-free): 1 800 1651 0716 > Poland (toll-free): 00 800 1213979 > Portugal (toll-free): 800 784 461 > Russian Federation (toll-free): 810 800 29674011 > Saudi Arabia (toll-free): +9668008443633 > Singapore (toll-free): 800 120 5615 > South Africa (toll-free): 0 800 983 867 > Spain (toll-free): 0 800 900 582 > Spain: +34 911 82 9906 > Sweden (toll-free): 020 980 772 > Sweden: +46 (0) 852 500 186 > Switzerland (toll-free): 0 800 740 393 > Switzerland: +41 (0) 435 0167 13 > Taiwan (toll-free): 00 800 666 854 > Thailand (toll-free): 001 800 658 131 > Turkey (toll-free): +90800448823683 > Ukraine (toll-free): 0 800 50 0641 > United Arab Emirates (toll-free): +97180004440439 > United Kingdom (toll-free): 0 808 168 0229 > United Kingdom: +44 (0) 207 151 1853 > United States (toll-free): 1 877 309 2073 > Uruguay (toll-free): 000 413 598 4110 > Viet Nam (toll-free): 120 65 157 >=20 > Access Code: 211-547-328 > Audio PIN: Shown after joining the meeting >=20 > Meeting ID: 211-547-328 > On 26 Apr 2013, at 19:57, Anil Madhavapeddy wrote: >=20 >> The minutes from this call are now up at: >> http://www.openmirage.org/wiki/weekly-2013-04-23 >>=20 >> Next week: >> * Solidify the story with actor libraries and concurrency from the = 1.0 release checklist. >> * Figure out the SSL options for 1.0. >>=20 >> I'm continuing to add to the repo list at: >> http://www.openmirage.org/wiki/dev-preview-checklist >> ...as we work through the list. >>=20 >> -anil >>=20 >> On 22 Apr 2013, at 15:29, Amir Chaudhry wrote: >>=20 >>> Hi folks, >>>=20 >>> Just to remind you about the Mirage Release weekly call. It's = happening tomorrow and the Go To Meeting details are the same as before = (copied here too). =20 >>>=20 >>> Notes from last week's meeting are available at: = http://openmirage.org/wiki/weekly-2013-04-16 >>>=20 >>>=20 >>> -- Go To Meeting details -- >>>=20 >>> You can either use the online GoToMeeting or dial in via the numbers = below: >>>=20 >>> 1. Please join my meeting. >>> https://www1.gotomeeting.com/join/211547328 >>>=20 >>> 2. Use your microphone and speakers (VoIP) - a headset is = recommended. Or, call in using your telephone. >>>=20 >>> United States: +1 (224) 649-0001 >>> Argentina (toll-free): 0 800 266 1382 >>> Australia (toll-free): 1 800 193 385 >>> Australia: +61 2 8355 1020 >>> Austria (toll-free): 0 800 202148 >>> Austria: +43 (0) 7 2088 1047 >>> Bahrain (toll-free): 800 81 111 >>> Belarus (toll-free): 8 820 0011 0214 >>> Belgium (toll-free): 0 800 26116 >>> Belgium: +32 (0) 28 08 4368 >>> Brazil (toll-free): 0 800 047 4906 >>> Canada (toll-free): 1 888 455 1389 >>> Canada: +1 (647) 497-9353 >>> China (toll-free): 4008 811084 >>> Czech Republic (toll-free): 800 500448 >>> Denmark (toll-free): 8090 1924 >>> Denmark: +45 (0) 69 91 89 28 >>> Finland (toll-free): 80094507 >>> Finland: +358 (0) 942 59 7850 >>> France (toll-free): 0 805 541 047 >>> France: +33 (0) 170 950 594 >>> Germany (toll-free): 0 800 723 5270 >>> Germany: +49 (0) 811 8899 6903 >>> Hong Kong (toll-free): 30713169 >>> Iceland (toll-free): 800 9869 >>> India (toll-free): 000 800 100 7855 >>> Indonesia (toll-free): 007 803 011 0395 >>> Ireland (toll-free): 1 800 946 538 >>> Ireland: +353 (0) 19 030 010 >>> Israel (toll-free): 1 809 212 875 >>> Italy (toll-free): 800 906959 >>> Italy: +39 0 247 92 13 01 >>> Japan (toll-free): 00 120 663 800 >>> Korea, Republic of (toll-free): 806150880 >>> Luxembourg (toll-free): 800 22104 >>> Malaysia (toll-free): 1 800 81 5373 >>> Mexico (toll-free): 01 800 925 0372 >>> Netherlands (toll-free): 0 800 265 8469 >>> Netherlands: +31 (0) 208 080 219 >>> New Zealand (toll-free): 0 800 45 2202 >>> New Zealand: +64 (0) 9 280 6302 >>> Norway (toll-free): 800 30 257 >>> Norway: +47 75 80 32 07 >>> Panama (toll-free): 00 800 226 8832 >>> Peru (toll-free): 0 800 54682 >>> Philippines (toll-free): 1 800 1651 0716 >>> Poland (toll-free): 00 800 1213979 >>> Portugal (toll-free): 800 784 461 >>> Russian Federation (toll-free): 810 800 29674011 >>> Saudi Arabia (toll-free): +9668008443633 >>> Singapore (toll-free): 800 120 5615 >>> South Africa (toll-free): 0 800 983 867 >>> Spain (toll-free): 0 800 900 582 >>> Spain: +34 911 82 9906 >>> Sweden (toll-free): 020 980 772 >>> Sweden: +46 (0) 852 500 186 >>> Switzerland (toll-free): 0 800 740 393 >>> Switzerland: +41 (0) 435 0167 13 >>> Taiwan (toll-free): 00 800 666 854 >>> Thailand (toll-free): 001 800 658 131 >>> Turkey (toll-free): +90800448823683 >>> Ukraine (toll-free): 0 800 50 0641 >>> United Arab Emirates (toll-free): +97180004440439 >>> United Kingdom (toll-free): 0 808 168 0229 >>> United Kingdom: +44 (0) 207 151 1853 >>> United States (toll-free): 1 877 309 2073 >>> Uruguay (toll-free): 000 413 598 4110 >>> Viet Nam (toll-free): 120 65 157 >>>=20 >>> Access Code: 211-547-328 >>> Audio PIN: Shown after joining the meeting >>>=20 >>> Meeting ID: 211-547-328 >>>=20 >>>=20 >>> Best wishes, >>> Amir >>>=20 >>>=20 >>>=20 >>>=20 >>=20 >>=20 >=20 >=20 From Richard.Mortier@nottingham.ac.uk Wed May 08 20:02:26 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Ua9dC-0000P0-2w (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 08 May 2013 20:02:26 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1479814 * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine01-20433-1.icritical.com ([93.95.15.169]:47141) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1Ua9dB-0004ZN-78 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 08 May 2013 20:02:26 +0100 Received: (qmail 8172 invoked from network); 8 May 2013 19:02:22 -0000 Received: from localhost (127.0.0.1) by engine01-20433-1.icritical.com with SMTP; 8 May 2013 19:02:22 -0000 Received: from engine01-20433-1.icritical.com ([127.0.0.1]) by localhost (engine01-20433-1.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 07593-06 for ; Wed, 8 May 2013 20:02:21 +0100 (BST) Received: (qmail 8112 invoked by uid 599); 8 May 2013 19:02:18 -0000 Received: from unknown (HELO smtp4.nottingham.ac.uk) (128.243.220.65) by engine01-20433-1.icritical.com (qpsmtpd/0.28) with ESMTP; Wed, 08 May 2013 20:02:18 +0100 Received: from uiwexhub01.ad.nottingham.ac.uk ([128.243.15.133]) by smtp4.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1Ua9d5-00024I-HP; Wed, 08 May 2013 20:02:19 +0100 From: Richard Mortier To: Udita Gangwal Date: Wed, 8 May 2013 20:01:51 +0100 Subject: Re: tun/tap issue Thread-Topic: tun/tap issue Thread-Index: Ac5MHoCFR3ZGM5zXRU+RXuzscBctBA== Message-ID: <8626FE6A-D81D-4541-8A78-98ADFFFDDE4F@nottingham.ac.uk> References: In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine01-20433-1.icritical.com Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 May 2013 19:02:26 -0000 Content-Length: 3058 Lines: 59 cc'ing the cl-mirage mailing list, in case anyone there has anything to add= /suggest. (i'm not sure whether udita is on the list, so please keep her cc'd. if som= eone could add her to the list, that'd be great.) for the list-- udita is an intern from IIIT who's helping me with some home= work/openflow/mirage related things. her current task is to try to pin down= the effects and then the cause of the bug that i believed i was seeing try= ing to use mirage networking on linux. (i mailed about this problem previou= sly.) On 8 May 2013, at 16:57, Udita Gangwal wrote: > Could you tell me what problem were you exactly facing with the ocaml-tun= tap in Linux? You said that the packets were somehow getting lost somewher= e. Could you please let me know the exact scenario? when i compiled and built mirage-www on linux using 4.00.1+mirage-unix (and= 3.12.1+mirage-unix-direct), i find that i can run the resulting binary, ev= erything appears to work, i can ping the address it claims using the tuntap= code (10.0.0.2) but when i try to "telnet 10.0.0.2 80" to connect to the w= eb server that should be running, it doesn't connect. when i ran tcpdumps o= n all the interfaces, it appeared that, although the route table entry shou= ld have been sending all traffic directed to 10.0.0.0/24 to the tap0 interf= ace, in fact the tcpdumps showed the TCP SYN packets were being sent to the= lo (loopback) interface.=20 all of this worked fine for me on OSX, hence my hypothesis was that this wa= s due to some bug in how the tuntap and/or netfilter apis were being used. = iirc ping is handled on a different path in the kernel, hence was working o= k. vincent has since extracted the tuntap functionality from mirage-net into i= ts own library, ocaml-tuntap. the suggestion was to try writing a couple of= simple test apps, perhaps one using ping and one that just runs a simple t= cp server of some kind that you can try to connect to (make it an echo serv= er if you're feeling fancy :) if both of those "Just Work" for you on linu= x, then it would seem that this is my problem alone. if either doesn't work= , then we can start to track down the bug. either way, having a couple of s= imple examples for ocaml-tuntap will be useful :) anil, vincent -- anything to add? --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From no-reply@fc-group.co.uk Thu May 09 09:10:44 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UaLw4-0007Do-Jy (Exim 4.70) (return-path ); Thu, 09 May 2013 09:10:44 +0100 X-Cam-SpamScore: sssssssss X-Cam-SpamDetails: score 9.6 from SpamAssassin-3.3.2-1480184 * 1.7 DEAR_SOMETHING BODY: Contains 'Dear (something)' * 1.2 MISSING_HEADERS Missing To: header * 0.7 HTML_IMAGE_ONLY_28 BODY: HTML: images with 2400-2800 bytes of words * 0.0 HTML_MESSAGE BODY: HTML included in message * 1.1 MIME_HTML_ONLY BODY: Message only has text/html MIME parts * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars * 1.6 RCVD_IN_BRBL_LASTEXT RBL: RCVD_IN_BRBL_LASTEXT * [66.187.101.211 listed in bb.barracudacentral.org] * 1.9 REPLYTO_WITHOUT_TO_CC REPLYTO_WITHOUT_TO_CC * 1.3 RDNS_NONE Delivered to internal network by a host with no rDNS X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from [66.187.101.211] (port=57680 helo=server.smiliegames.com) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UaLvy-0003xe-6s (Exim 4.80_167-5a66dd3) (return-path ); Thu, 09 May 2013 09:10:44 +0100 Received: from [205.204.94.17] (port=58911 helo=fc-group.co.uk) by server.smiliegames.com with esmtpa (Exim 4.77) (envelope-from ) id 1UaLvW-0005mx-IR; Thu, 09 May 2013 08:10:10 +0000 From: "HMRC" Subject: Notice of tax return Date: 09 May 2013 01:10:58 -0700 Message-ID: <20130509011058.36761236388F3D12@fc-group.co.uk> MIME-Version: 1.0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.smiliegames.com X-AntiAbuse: Original Domain - lists.cam.ac.uk X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - fc-group.co.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list Reply-To: no-reply@fc-group.co.uk List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 May 2013 08:10:44 -0000 Content-Length: 2983 Lines: 89 New Page 11=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20




Date: 9 may 2013
Our Ref.: C/123345/13
Your Ref.: 19B/0231/13

NOTICE OF TAX RETURN FOR YEAR 2012

=20=20=20=20=20=20=20=20=20=20=20
Dear Sir/Madam,

I am sending this email to announce: After the last annual calcu= lation of your fiscal activity we have determined that you are eligible to r= eceive a tax return of:

£288.87

To receive your return, you need to create a Government Gateway acco= unt.

Click here to Re= gister


Our head office address can be found on our web site at HM Revenue &= amp; Customs: www.hmrc.gov.uk

=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20

The contents of this email and any attachments = are confidential and as applicable, copyright in these is reserved to HM Rev= enue & Customs. Unless expressly authorised by us, any further dissemina= tion or distribution of this email or its attachments is prohibited.

From anil@recoil.org Sat May 11 00:49:42 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uax4I-000081-Bc (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 11 May 2013 00:49:42 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1480571 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:20372 helo=dark.recoil.org) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1Uax4H-0002lv-9B (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 11 May 2013 00:49:42 +0100 Received: (qmail 24748 invoked by uid 634); 10 May 2013 23:49:41 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from 104.sub-70-192-64.myvzw.com (HELO [192.168.1.4]) (70.192.64.104) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sat, 11 May 2013 00:49:41 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: ANN: cohttp-0.9.7 and github-0.5.0 Date: Fri, 10 May 2013 19:49:36 -0400 Message-Id: To: "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Yaron Minsky , Jeremy Yallop X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 May 2013 23:49:42 -0000 Content-Length: 2039 Lines: 46 I've uploaded new versions of Cohttp and Github to OPAM. Thanks in = particular to David Sheets for testing I haven't broken anything in = Cohttp (too badly). Jeremy: I pulled the parameterised monad bits out of this release until = the interfaces are a little more baked (there's no particular rush on = this front). http://github.com/mirage/ocaml-cohttp Cohttp 0.9.7 (2013-05-10): * Attach a GC finaliser to the Lwt client to ensure that even an HTTP = body isnt consumed, the socket will eventually be closed (#11). * Add an Async.Server interface, and revise the Client interface to be = more in line with Core standards. * Add 422 Unprocessable Entity code. * Refactor modules better across Lwt/Async, but incompatible with = earlier releases for Async (Lwt is unchanged at present). * Add user agent string and User-Agent header helper function * The git history of this release is full of adventures in parameterised = monads and refactoring, but this isn't in the actual release. Yet. http://github.com/avsm/ocaml-github Github 0.5.0 (2013-05-10): * Force `-j-std` to ATDgen to always use standards-compliant JSON (#11). * Rename `Github.Issues` to `Github.Issue` to parallel other submodules. * Rename `Github.Issue.edit` to `Github.Issue.update` to parallel other = CRUD interfaces. * Reorder named parameters to raw API submodule functions. * Add Pull Request API. * Add Hook API (generic "web" hooks only). * Add Statuses API. * Add structured semantic errors. * Nows sends partially configurable (via `API.set_user_agent`) = User-Agent string. * Add `API.set_token` to bind an access token for subsequent requests. * Declare ocaml-re dependency. * Add anonymous bind operator (>>) to `Github.Monad`. * Add `Github.Token.delete` for revoking GitHub authorization tokens. * Add `Github_cookie_jar.delete` for deleting local token cookies. * Add `revoke` command to git-jar. * Support GitHub cookie jar names with slashes in. * Change the signature of `Github_cookie_jar.init` from `... -> unit` to = `... -> unit Lwt.t`. From Dave.Scott@eu.citrix.com Mon May 13 22:36:31 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uc0Q3-0000yp-AZ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 13 May 2013 22:36:31 +0100 X-Cam-SpamDetails: score -0.6 from SpamAssassin-3.3.2-1481494 * -0.6 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from smtp.eu.citrix.com ([46.33.159.39]:49670) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1Uc0Q1-0000gK-j6 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 13 May 2013 22:36:31 +0100 X-IronPort-AV: E=Sophos;i="4.87,552,1363132800"; d="scan'208";a="4528120" Received: from lonpmailmx02.citrite.net ([10.30.203.163]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/RC4-MD5; 13 May 2013 21:27:18 +0000 Received: from LONPMAILBOX01.citrite.net ([10.30.224.161]) by LONPMAILMX02.citrite.net ([10.30.203.163]) with mapi; Mon, 13 May 2013 22:36:29 +0100 From: Dave Scott To: "cl-mirage@lists.cam.ac.uk" Date: Mon, 13 May 2013 22:36:31 +0100 Subject: "The Kernel is the Problem, Not the Solution" Thread-Topic: "The Kernel is the Problem, Not the Solution" Thread-Index: Ac5QIe1PwseuO6ZUSbiVDMAgdQpgYQ== Message-ID: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2013 21:36:31 -0000 Content-Length: 171 Lines: 9 http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurre= nt-connections-the-kernel-i.html Some quite interesting stuff in there. --=20 Dave Scott= From anil@recoil.org Mon May 13 22:50:21 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uc0dR-000197-1S (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 13 May 2013 22:50:21 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1481494 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:10315 helo=dark.recoil.org) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1Uc0dP-0003ec-iO (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 13 May 2013 22:50:21 +0100 Received: (qmail 6214 invoked by uid 634); 13 May 2013 21:50:19 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Mon, 13 May 2013 22:50:18 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: "The Kernel is the Problem, Not the Solution" From: Anil Madhavapeddy In-Reply-To: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> Date: Mon, 13 May 2013 22:50:21 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> To: Dave Scott X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2013 21:50:21 -0000 Content-Length: 531 Lines: 16 On 13 May 2013, at 22:36, Dave Scott wrote: >=20 > = http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurr= ent-connections-the-kernel-i.html >=20 > Some quite interesting stuff in there. I definitely want to try out our existing TCP stack with a few million = connections and see how we fare. Perhaps we should give this a shot at = the Xen hackathon on Thursday? The hard bit is finding enough = interfaces on the Linux test clients so they don't run out of ports :-P -anil= From stedolan@stedolan.net Tue May 14 00:00:18 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uc1j8-0001ky-Sj (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 00:00:18 +0100 X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1481494 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.217.182 listed in list.dnswl.dnsbl.ja.net] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-lb0-f182.google.com ([209.85.217.182]:53974) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1Uc1j7-0007xe-27 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 00:00:18 +0100 Received: by mail-lb0-f182.google.com with SMTP id r11so6993441lbv.13 for ; Mon, 13 May 2013 16:00:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:sender:x-originating-ip:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding:x-gm-message-state; bh=bSKtvbzotMN5kyBFhvG4lGMHypGkynyvBDSgjHOWMrI=; b=VP1BvM51+pQB91pJF7777b2uH1S/O52BZKUj8P2Ez8bDJDipzR03i5d3Wsb9RteRtl UD1Wniq5pJgRXdYB0X+Sf4ALCrZmKz0z+0z78K+R0cZoq1pA2ueAW3OUpRrsuixGjT6r t4CN0m4xrku1GL++K02Aw+5JSaFOaZkmkOBgUi91e9a+vsTOeN8ddslmsVgMT+n7FkRU TiKtuBKKHA78cRP4vVuf/GM+VqrAk5oXQDqXhJuiYJifVSaQHcJn44O+AJsyu41rJM/z 6pJebHGG8ggAq2dVqogSWNJYmFduu4nbCqKcUz7zhuS1OGDtp91Lf7EnsTah9KKUkKVg v8MQ== MIME-Version: 1.0 X-Received: by 10.112.157.102 with SMTP id wl6mr13886530lbb.65.1368486016953; Mon, 13 May 2013 16:00:16 -0700 (PDT) Sender: stedolan@stedolan.net Received: by 10.114.160.136 with HTTP; Mon, 13 May 2013 16:00:16 -0700 (PDT) X-Originating-IP: [131.111.184.8] In-Reply-To: References: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> Date: Tue, 14 May 2013 00:00:16 +0100 X-Google-Sender-Auth: BWO_mRTi4ANypVHXFPxMHGEQoEU Message-ID: Subject: Re: "The Kernel is the Problem, Not the Solution" From: Stephen Dolan To: Anil Madhavapeddy Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQkTD8j2Vi4Jfvc7LXR6CJsSFghzuaBH6xfSA/8D1n4DviH0RIrLA8LFU9GQo3XaKFbtYul3 Cc: Dave Scott , "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2013 23:00:19 -0000 Content-Length: 836 Lines: 26 You've probably seen this before, but it's really easy to multihome interfaces on linux: for i in $(seq 1 254); do ifconfig eth0:$i 192.168.42.$i; done gives you 2^24 distinct (ip, port) pairs on one box. Stephen On Mon, May 13, 2013 at 10:50 PM, Anil Madhavapeddy wrote= : > On 13 May 2013, at 22:36, Dave Scott wrote: > >> >> http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concu= rrent-connections-the-kernel-i.html >> >> Some quite interesting stuff in there. > > I definitely want to try out our existing TCP stack with a few million co= nnections and see how we fare. Perhaps we should give this a shot at the X= en hackathon on Thursday? The hard bit is finding enough interfaces on the= Linux test clients so they don't run out of ports :-P > > -anil From anil@recoil.org Tue May 14 09:06:17 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcAFV-0003eD-Rt (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 09:06:17 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1481731 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:19023 helo=dark.recoil.org) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UcAFU-0002CV-EY (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 09:06:17 +0100 Received: (qmail 16722 invoked by uid 634); 14 May 2013 08:06:16 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 14 May 2013 09:06:15 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Reminder: Mirage weekly call - 14 May at 1600 BST From: Anil Madhavapeddy In-Reply-To: <3547678E-4F38-4F89-99F3-2DBFB1D4D088@recoil.org> Date: Tue, 14 May 2013 09:06:12 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <49CADCC7-33A1-4E43-B51E-2DF266E50E0A@gmail.com> <9E08CA48-929E-4BB9-88C6-312B379A088A@recoil.org> <3547678E-4F38-4F89-99F3-2DBFB1D4D088@recoil.org> To: "cl-mirage@lists.cam.ac.uk List" , Jon Ludlam X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 08:06:17 -0000 Content-Length: 2991 Lines: 94 Agenda: - Ocamlot update and use in Mirage (Anil, David Sheets) - Actors: Dave's been working on this with his message switch - Performance testing (all!) - Irminsule demo for next week (Thomas/Anil) - Follow-up on various other items in last two discussions as needed -- Go To Meeting details -- You can either use the online GoToMeeting or dial in via the numbers = below: 1. Please join my meeting. https://www1.gotomeeting.com/join/211547328 2. Use your microphone and speakers (VoIP) - a headset is recommended. = Or, call in using your telephone. United States: +1 (224) 649-0001 Argentina (toll-free): 0 800 266 1382 Australia (toll-free): 1 800 193 385 Australia: +61 2 8355 1020 Austria (toll-free): 0 800 202148 Austria: +43 (0) 7 2088 1047 Bahrain (toll-free): 800 81 111 Belarus (toll-free): 8 820 0011 0214 Belgium (toll-free): 0 800 26116 Belgium: +32 (0) 28 08 4368 Brazil (toll-free): 0 800 047 4906 Canada (toll-free): 1 888 455 1389 Canada: +1 (647) 497-9353 China (toll-free): 4008 811084 Czech Republic (toll-free): 800 500448 Denmark (toll-free): 8090 1924 Denmark: +45 (0) 69 91 89 28 Finland (toll-free): 80094507 Finland: +358 (0) 942 59 7850 France (toll-free): 0 805 541 047 France: +33 (0) 170 950 594 Germany (toll-free): 0 800 723 5270 Germany: +49 (0) 811 8899 6903 Hong Kong (toll-free): 30713169 Iceland (toll-free): 800 9869 India (toll-free): 000 800 100 7855 Indonesia (toll-free): 007 803 011 0395 Ireland (toll-free): 1 800 946 538 Ireland: +353 (0) 19 030 010 Israel (toll-free): 1 809 212 875 Italy (toll-free): 800 906959 Italy: +39 0 247 92 13 01 Japan (toll-free): 00 120 663 800 Korea, Republic of (toll-free): 806150880 Luxembourg (toll-free): 800 22104 Malaysia (toll-free): 1 800 81 5373 Mexico (toll-free): 01 800 925 0372 Netherlands (toll-free): 0 800 265 8469 Netherlands: +31 (0) 208 080 219 New Zealand (toll-free): 0 800 45 2202 New Zealand: +64 (0) 9 280 6302 Norway (toll-free): 800 30 257 Norway: +47 75 80 32 07 Panama (toll-free): 00 800 226 8832 Peru (toll-free): 0 800 54682 Philippines (toll-free): 1 800 1651 0716 Poland (toll-free): 00 800 1213979 Portugal (toll-free): 800 784 461 Russian Federation (toll-free): 810 800 29674011 Saudi Arabia (toll-free): +9668008443633 Singapore (toll-free): 800 120 5615 South Africa (toll-free): 0 800 983 867 Spain (toll-free): 0 800 900 582 Spain: +34 911 82 9906 Sweden (toll-free): 020 980 772 Sweden: +46 (0) 852 500 186 Switzerland (toll-free): 0 800 740 393 Switzerland: +41 (0) 435 0167 13 Taiwan (toll-free): 00 800 666 854 Thailand (toll-free): 001 800 658 131 Turkey (toll-free): +90800448823683 Ukraine (toll-free): 0 800 50 0641 United Arab Emirates (toll-free): +97180004440439 United Kingdom (toll-free): 0 808 168 0229 United Kingdom: +44 (0) 207 151 1853 United States (toll-free): 1 877 309 2073 Uruguay (toll-free): 000 413 598 4110 Viet Nam (toll-free): 120 65 157 Access Code: 211-547-328 Audio PIN: Shown after joining the meeting Meeting ID: 211-547-328 From anil@recoil.org Tue May 14 09:07:27 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcAGd-0003jK-QR (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 09:07:27 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1481731 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:8841 helo=dark.recoil.org) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UcAGd-0002iZ-DM (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 09:07:27 +0100 Received: (qmail 21911 invoked by uid 634); 14 May 2013 08:07:27 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 14 May 2013 09:07:25 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: "The Kernel is the Problem, Not the Solution" From: Anil Madhavapeddy In-Reply-To: Date: Tue, 14 May 2013 09:07:23 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> To: Stephen Dolan , Balraj Singh X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Dave Scott , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 08:07:27 -0000 Content-Length: 1265 Lines: 42 Yeah, it's all the other stuff -- managing the iperf instances, making = sure they're not all hogging CPU, splitting it across aliases, = interfaces, bridges and VMs, etc. All a bit tedious. Isn't there some magic load generator that already = helps out with this sort of thing? -anil On 14 May 2013, at 00:00, Stephen Dolan = wrote: > You've probably seen this before, but it's really easy to multihome > interfaces on linux: >=20 > for i in $(seq 1 254); do ifconfig eth0:$i 192.168.42.$i; done >=20 > gives you 2^24 distinct (ip, port) pairs on one box. >=20 > Stephen >=20 > On Mon, May 13, 2013 at 10:50 PM, Anil Madhavapeddy = wrote: >> On 13 May 2013, at 22:36, Dave Scott = wrote: >>=20 >>>=20 >>> = http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurr= ent-connections-the-kernel-i.html >>>=20 >>> Some quite interesting stuff in there. >>=20 >> I definitely want to try out our existing TCP stack with a few = million connections and see how we fare. Perhaps we should give this a = shot at the Xen hackathon on Thursday? The hard bit is finding enough = interfaces on the Linux test clients so they don't run out of ports :-P >>=20 >> -anil >=20 From amc79@cam.ac.uk Tue May 14 09:09:52 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcAIx-0003nu-Vb (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 09:09:52 +0100 X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from host81-149-102-120.in-addr.btopenworld.com ([81.149.102.120]:51508 helo=amirmacbook.home) by ppsw-32.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.156]:587) with esmtpsa (PLAIN:amc79) (TLSv1:AES128-SHA:128) id 1UcAIx-0007mP-2B (Exim 4.80_167-5a66dd3) (return-path ); Tue, 14 May 2013 09:09:51 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: "The Kernel is the Problem, Not the Solution" From: Amir Chaudhry In-Reply-To: Date: Tue, 14 May 2013 09:09:54 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <2FDCE563-C06E-425A-AD48-B5C3327903CC@cam.ac.uk> References: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1503) Cc: Balraj Singh , Stephen Dolan , Dave Scott , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 08:09:52 -0000 Content-Length: 1489 Lines: 51 On 14 May 2013, at 09:07, Anil Madhavapeddy wrote: > Yeah, it's all the other stuff -- managing the iperf instances, making = sure they're not all hogging CPU, splitting it across aliases, = interfaces, bridges and VMs, etc. >=20 > All a bit tedious. Isn't there some magic load generator that already = helps out with this sort of thing? I think that would be the same magical load generator that Euan was = asking for a couple of weeks ago :) ac >=20 > On 14 May 2013, at 00:00, Stephen Dolan = wrote: >=20 >> You've probably seen this before, but it's really easy to multihome >> interfaces on linux: >>=20 >> for i in $(seq 1 254); do ifconfig eth0:$i 192.168.42.$i; done >>=20 >> gives you 2^24 distinct (ip, port) pairs on one box. >>=20 >> Stephen >>=20 >> On Mon, May 13, 2013 at 10:50 PM, Anil Madhavapeddy = wrote: >>> On 13 May 2013, at 22:36, Dave Scott = wrote: >>>=20 >>>>=20 >>>> = http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurr= ent-connections-the-kernel-i.html >>>>=20 >>>> Some quite interesting stuff in there. >>>=20 >>> I definitely want to try out our existing TCP stack with a few = million connections and see how we fare. Perhaps we should give this a = shot at the Xen hackathon on Thursday? The hard bit is finding enough = interfaces on the Linux test clients so they don't run out of ports :-P >>>=20 >>> -anil >>=20 >=20 >=20 From ms705@hermes.cam.ac.uk Tue May 14 09:42:21 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcAoP-0005TU-V5 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 09:42:21 +0100 X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from mourne.cl.cam.ac.uk ([128.232.10.102]:48079) by ppsw-42.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.159]:465) with esmtpsa (PLAIN:ms705) (TLSv1:DHE-RSA-CAMELLIA256-SHA:256) id 1UcAoP-0006q6-9Q (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 09:42:21 +0100 Message-ID: <5191F8ED.7070304@cl.cam.ac.uk> Date: Tue, 14 May 2013 09:42:21 +0100 From: Malte Schwarzkopf User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130330 Thunderbird/17.0.5 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: "The Kernel is the Problem, Not the Solution" References: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> In-Reply-To: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Sender: Malte Schwarzkopf X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 08:42:22 -0000 Content-Length: 2234 Lines: 52 On 13/05/13 22:36, Dave Scott wrote: > > http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html > > Some quite interesting stuff in there. > There's some valid points here that we found with the R2D2 work, too (e.g. raw sockets don't do more than ~1 Mpps), but also a bit of folklore that does not seem right to me: "Don’t scribble data all over memory via pointers. Each time you follow a pointer it will be a cache miss: [hash pointer] -> [Task Control Block] -> [Socket] -> [App]. That’s four cache misses." ... that sounds like someone didn't understand caches, or they're talking about a particularly weird kind of pointer. Normal pointers don't require looking up in the TCB. "The paging table for 32gigs require 64MB of paging tables which doesn’t fit in cache. So you have two caches misses, one for the paging table and one for what it points to. [...] Solutions: compress data; use cache efficient structures instead of binary search tree that has a lot of memory accesses" This seems to ignore the fact that page tables are in HW; if they're talking about the data structures used for maintaining mapped memory regions in the kernel, this is more accurate. In that case, approaches like that in RadixVM (EuroSys 2013; http://dl.acm.org/citation.cfm?id=2465373) seem promising. And there's some misattribution of SDN concepts, I think: "The control plane is left to Linux, for the data plane, nothing. The data plane runs in application code. It never interacts with the kernel. There’s no thread scheduling, no system calls, no interrupts, nothing." Sounds nice, but is largely untrue for Linux. Even user-space networking solutions like NetMap still occasionally make syscalls (though they amortise many operations into one of them). What they are proposing sounds more like the ideas in Arrakis (HotOS 2013) and some super-computing work (e.g. FusedOS at SC 2012) -- or indeed like some ideas we're looking at for DIOS, where cores are "unhooked" from Linux once it boots up, and get to use NIC queues directly (without ever telling the Linux kernel, but having the option to interact with it for resource allocation). Cheers, Malte From stas@FreeBSD.org Tue May 14 09:49:27 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcAvH-0005kf-Cf (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 09:49:27 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1481731 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from backbone.deglitch.com ([78.110.53.255]:27032 helo=mx0.deglitch.com) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UcAvB-0007qJ-0J (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 09:49:27 +0100 Received: from DSPAM-Daemon (localhost [127.0.0.1]) by mx0.deglitch.com (Postfix) with SMTP id 899588FC3A for ; Tue, 14 May 2013 12:49:00 +0400 (MSK) Received: from orion.SpringDaemons.com (unknown [IPv6:2601:9:3d00:f:218:8bff:fe20:cc91]) by mx0.deglitch.com (Postfix) with ESMTPSA id 46BD98FC2B; Tue, 14 May 2013 12:48:51 +0400 (MSK) Received: from orion (localhost [127.0.0.1]) by orion.SpringDaemons.com (Postfix) with SMTP id 09CAA39D29; Tue, 14 May 2013 01:48:59 -0700 (PDT) Date: Tue, 14 May 2013 01:48:58 -0700 From: Stanislav Sedov To: Anil Madhavapeddy Subject: Re: "The Kernel is the Problem, Not the Solution" Message-Id: <20130514014858.f2907d2584b263399d627498@FreeBSD.org> In-Reply-To: References: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> Organization: The FreeBSD Project X-Mailer: carrier-pigeon Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-DSPAM-Result: Innocent X-DSPAM-Processed: Tue May 14 12:49:00 2013 X-DSPAM-Confidence: 1.0000 X-DSPAM-Improbability: 1 in 98689409 chance of being spam X-DSPAM-Probability: 0.0023 X-DSPAM-Signature: 5191fa7c13142827654262 Cc: Balraj Singh , Stephen Dolan , Dave Scott , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 08:49:27 -0000 Content-Length: 737 Lines: 23 On Tue, 14 May 2013 09:07:23 +0100 Anil Madhavapeddy mentioned: > Yeah, it's all the other stuff -- managing the iperf instances, making sure they're not all hogging CPU, splitting it across aliases, interfaces, bridges and VMs, etc. > > All a bit tedious. Isn't there some magic load generator that already helps out with this sort of thing? > I can try to get access to the SwiftTest appliance (the company I worked for and run the test. Do you want to run it agains Mirage in Xen? What kind of protocols do you want to test against? -- Stanislav Sedov ST4096-RIPE () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments !DSPAM:5191fa7c13142827654262! From anil@recoil.org Tue May 14 12:57:27 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcDrD-0006mE-N1 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 12:57:27 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1481731 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:7421 helo=dark.recoil.org) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1UcDrC-0006ar-7b (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 12:57:27 +0100 Received: (qmail 16992 invoked by uid 634); 14 May 2013 11:57:24 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from volstagg-0.srg.cl.cam.ac.uk (HELO [10.0.1.75]) (128.232.32.232) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 14 May 2013 12:57:23 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: "The Kernel is the Problem, Not the Solution" From: Anil Madhavapeddy In-Reply-To: <20130514014858.f2907d2584b263399d627498@FreeBSD.org> Date: Tue, 14 May 2013 12:57:21 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> <20130514014858.f2907d2584b263399d627498@FreeBSD.org> To: Stanislav Sedov X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Balraj Singh , Stephen Dolan , Dave Scott , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 11:57:27 -0000 Content-Length: 1530 Lines: 37 On 14 May 2013, at 09:48, Stanislav Sedov wrote: > On Tue, 14 May 2013 09:07:23 +0100 > Anil Madhavapeddy mentioned: >=20 >> Yeah, it's all the other stuff -- managing the iperf instances, = making sure they're not all hogging CPU, splitting it across aliases, = interfaces, bridges and VMs, etc. >>=20 >> All a bit tedious. Isn't there some magic load generator that already = helps out with this sort of thing? >>=20 >=20 > I can try to get access to the SwiftTest appliance (the company I = worked > for and run the test. Do you want to run it agains Mirage in Xen? > What kind of protocols do you want to test against? While it would be very nice to have access to some 'big iron' tests, what we really need right now is a lightweight way for developers to interactively run these tests and fix the scalability bugs they find along the way. Balraj did this in the early days for particular aspects of TCP (by manually looking at the window plots of traces and fixing scheduling issues), but that's obviously not too sustainable. I wouldn't be surprised at all to find many low-hanging scalability problems from testing the Mirage-net stack against thousands (and then millions) of connections. So we do need a few shell scripts to automate the VM launch and load generation of the clients and server. Once the issues from these are fixed, I would be very interested in using SwiftTest as a regression test (i.e. once 10 million connections work, they should stay working). -anil= From heidihoward360@gmail.com Tue May 14 16:21:27 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcH2d-0006Qt-MJ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 16:21:27 +0100 X-Cam-SpamDetails: score -0.3 from SpamAssassin-3.3.2-1481731 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.128.47 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (heidihoward360[at]gmail.com) * 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in * digit (heidihoward360[at]gmail.com) * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-qe0-f47.google.com ([209.85.128.47]:56314) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UcH2d-0006w8-6r (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 16:21:27 +0100 Received: by mail-qe0-f47.google.com with SMTP id w7so500306qeb.20 for ; Tue, 14 May 2013 08:21:26 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.229.155.201 with SMTP id t9mr5167329qcw.155.1368544886480; Tue, 14 May 2013 08:21:26 -0700 (PDT) Sender: heidihoward360@gmail.com Received: by 10.49.107.36 with HTTP; Tue, 14 May 2013 08:21:26 -0700 (PDT) Date: Tue, 14 May 2013 16:21:26 +0100 X-Google-Sender-Auth: 13gKdX7N5ed6i2BTkPNr-WWhJjU Message-ID: Subject: Problems Installing OPAM From: Heidi Howard To: cl-mirage@lists.cam.ac.uk Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 15:21:27 -0000 Content-Length: 1127 Lines: 36 I'm trying to install OPAM on Ubuntu 12.04 64-bit using the package manager, but I don't understand why its not working, any suggestions ? $ echo "deb [arch=amd64] http://www.recoil.org/~avsm/ wheezy main" >> /etc/apt/sources.list $ sudo apt-get install opam Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed opam 0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded. Need to get 2,598 kB of archives. After this operation, 7,478 kB of additional disk space will be used. WARNING: The following packages cannot be authenticated! opam Install these packages without verification [y/N]? y Get:1 http://www.recoil.org/~avsm/ wheezy/main opam amd64 0.9.6 [2,598 kB] Fetched 2,598 kB in 0s (9,820 kB/s) Selecting previously unselected package opam. (Reading database ... 313487 files and directories currently installed.) Unpacking opam (from .../archives/opam_0.9.6_amd64.deb) ... Processing triggers for man-db ... Setting up opam (0.9.6) ... $ opam init bash: /usr/local/bin/opam: No such file or directory -- Regards Heidi From heidihoward360@gmail.com Tue May 14 16:51:06 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcHVK-0007ZR-4S (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 16:51:06 +0100 X-Cam-SpamDetails: score -0.3 from SpamAssassin-3.3.2-1481731 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.128.50 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (heidihoward360[at]gmail.com) * 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in * digit (heidihoward360[at]gmail.com) * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-qe0-f50.google.com ([209.85.128.50]:50144) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UcHVI-0005Wo-1h (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 16:51:06 +0100 Received: by mail-qe0-f50.google.com with SMTP id k5so542086qej.9 for ; Tue, 14 May 2013 08:51:03 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.229.105.222 with SMTP id u30mr10388865qco.142.1368546663857; Tue, 14 May 2013 08:51:03 -0700 (PDT) Sender: heidihoward360@gmail.com Received: by 10.49.107.36 with HTTP; Tue, 14 May 2013 08:51:03 -0700 (PDT) In-Reply-To: References: Date: Tue, 14 May 2013 16:51:03 +0100 X-Google-Sender-Auth: wusre4QdO-NRVmiob6aczQfuvaM Message-ID: Subject: Re: Problems Installing OPAM From: Heidi Howard To: cl-mirage@lists.cam.ac.uk Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 15:51:06 -0000 Content-Length: 1295 Lines: 45 Fixed, using "hash -r" On 14 May 2013 16:21, Heidi Howard wrote: > I'm trying to install OPAM on Ubuntu 12.04 64-bit using the package > manager, but I don't understand why its not working, any suggestions ? > > > $ echo "deb [arch=amd64] http://www.recoil.org/~avsm/ wheezy main" >> > /etc/apt/sources.list > > $ sudo apt-get install opam > Reading package lists... Done > Building dependency tree > Reading state information... Done > The following NEW packages will be installed > opam > 0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded. > Need to get 2,598 kB of archives. > After this operation, 7,478 kB of additional disk space will be used. > WARNING: The following packages cannot be authenticated! > opam > Install these packages without verification [y/N]? y > Get:1 http://www.recoil.org/~avsm/ wheezy/main opam amd64 0.9.6 [2,598 kB] > Fetched 2,598 kB in 0s (9,820 kB/s) > Selecting previously unselected package opam. > (Reading database ... 313487 files and directories currently installed.) > Unpacking opam (from .../archives/opam_0.9.6_amd64.deb) ... > Processing triggers for man-db ... > Setting up opam (0.9.6) ... > > $ opam init > bash: /usr/local/bin/opam: No such file or directory > > > > -- > Regards > Heidi -- Regards Heidi From thomas.gazagnaire@gmail.com Tue May 14 16:51:44 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcHVw-0007Zn-TS (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 16:51:44 +0100 X-Cam-SpamDetails: score -0.6 from SpamAssassin-3.3.2-1481731 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.82.49 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (thomas.gazagnaire[at]gmail.com) * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-wg0-f49.google.com ([74.125.82.49]:58108) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UcHVw-00012S-gY (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 16:51:44 +0100 Received: by mail-wg0-f49.google.com with SMTP id j13so604611wgh.4 for ; Tue, 14 May 2013 08:51:44 -0700 (PDT) X-Received: by 10.181.13.169 with SMTP id ez9mr7745809wid.8.1368546702199; Tue, 14 May 2013 08:51:42 -0700 (PDT) Received: from [192.168.0.12] (gou06-3-88-170-165-56.fbx.proxad.net. [88.170.165.56]) by mx.google.com with ESMTPSA id dq8sm24705694wib.1.2013.05.14.08.51.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 14 May 2013 08:51:41 -0700 (PDT) Sender: Thomas Gazagnaire From: Thomas Gazagnaire Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: a cool demo of Akka (Scala's actor) monitors Date: Tue, 14 May 2013 17:51:39 +0200 Message-Id: <1F2747C1-F6BF-4A33-92B7-6F028A0C3574@gazagnaire.org> To: Mirage List Mime-Version: 1.0 (Apple Message framework v1085) X-Mailer: Apple Mail (2.1085) X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 15:51:45 -0000 Content-Length: 159 Lines: 9 http://console-demo.typesafe.com/demo@typesafe.com/Demo/ Lots of flashy buttons and graphs Would be cool to have the same thing for mirage apps. -- Thomas From crowcroft@gmail.com Tue May 14 17:02:07 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcHfz-0000Nm-Qj (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 17:02:07 +0100 X-Cam-SpamDetails: score -0.6 from SpamAssassin-3.3.2-1481731 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.83.42 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (crowcroft[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ee0-f42.google.com ([74.125.83.42]:38056) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UcHfy-0006y0-9H (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 17:02:07 +0100 Received: by mail-ee0-f42.google.com with SMTP id c50so435337eek.1 for ; Tue, 14 May 2013 09:02:06 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.14.109.131 with SMTP id s3mr92269396eeg.26.1368547326098; Tue, 14 May 2013 09:02:06 -0700 (PDT) Sender: crowcroft@gmail.com Received: by 10.223.144.134 with HTTP; Tue, 14 May 2013 09:02:05 -0700 (PDT) In-Reply-To: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> References: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> Date: Tue, 14 May 2013 17:02:05 +0100 X-Google-Sender-Auth: w1ozyEUU1xX6buSjisabluf5BcQ Message-ID: Subject: Re: "The Kernel is the Problem, Not the Solution" From: Jon Crowcroft To: Dave Scott Content-Type: multipart/alternative; boundary=001a11c29f7e3b293404dcafc1ff Cc: "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 16:02:07 -0000 Content-Length: 2244 Lines: 55 --001a11c29f7e3b293404dcafc1ff Content-Type: text/plain; charset=ISO-8859-1 so one thing wrong in the article - I don't know how many people read ich Steven's Unix Networked Programming (I'd just use his examples) but TCP/IP Illustrated Volume 2 is far more useful since its a walkthrough of the BSD kernel stack (amongst other things) and contains MANY fine lessons on how to write gofaster code (one example is TCP/P header prediction for example in the packet receive code of TCP, and another is the use of PCTCB caches... On Mon, May 13, 2013 at 10:36 PM, Dave Scott wrote: > > > http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html > > Some quite interesting stuff in there. > > -- > Dave Scott > --001a11c29f7e3b293404dcafc1ff Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
so one thing wrong in the article - I don't know how m= any people read ich Steven's Unix Networked Programming (I'd just u= se his examples) but TCP/IP Illustrated Volume 2 is far more useful since i= ts a walkthrough of the BSD kernel stack (amongst other things) and contain= s MANY fine lessons on how to write gofaster code
(one example is TCP/P header prediction for example in the packet receive c= ode of TCP, and another is the use of PCTCB caches...


On Mon, May 13, 2013 at= 10:36 PM, Dave Scott <Dave.Scott@eu.citrix.com> wrot= e:

http://highs= calability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connectio= ns-the-kernel-i.html

Some quite interesting stuff in there.

--
Dave Scott

--001a11c29f7e3b293404dcafc1ff-- From anil@recoil.org Tue May 14 17:16:18 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcHti-0000ok-4A (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 17:16:18 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1481731 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:17633 helo=dark.recoil.org) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UcHth-0002Vx-hj (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 17:16:18 +0100 Received: (qmail 19806 invoked by uid 634); 14 May 2013 16:16:17 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.242]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 14 May 2013 17:16:16 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: a cool demo of Akka (Scala's actor) monitors From: Anil Madhavapeddy In-Reply-To: <1F2747C1-F6BF-4A33-92B7-6F028A0C3574@gazagnaire.org> Date: Tue, 14 May 2013 17:16:14 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1F2747C1-F6BF-4A33-92B7-6F028A0C3574@gazagnaire.org> To: Thomas Gazagnaire X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: =?iso-8859-1?Q?Daniel_B=FCnzli?= , Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 16:16:18 -0000 Content-Length: 404 Lines: 17 On 14 May 2013, at 16:51, Thomas Gazagnaire = wrote: > http://console-demo.typesafe.com/demo@typesafe.com/Demo/ >=20 > Lots of flashy buttons and graphs >=20 > Would be cool to have the same thing for mirage apps. That's very, very nice looking (I won't comment on its utility until = I've seen it on a real app!) So Daniel, here's the challenge for Vg graphs... :-) -anil= From anil@recoil.org Tue May 14 17:17:45 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcHv7-0000ql-AF (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 17:17:45 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1481731 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:13970 helo=dark.recoil.org) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UcHv6-00037D-hN (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 17:17:45 +0100 Received: (qmail 11395 invoked by uid 634); 14 May 2013 16:17:44 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.242]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 14 May 2013 17:17:44 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Problems Installing OPAM From: Anil Madhavapeddy In-Reply-To: Date: Tue, 14 May 2013 17:17:42 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Heidi Howard X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 16:17:45 -0000 Content-Length: 1753 Lines: 62 This is a new one on me. I'm not sure that OPAM can do anything to = rebuild the shell cache. But it's very very weird that invoking OPAM = from the shell would work, and then a subshell fails to find the binary = that launched it! So, hrm, not sure what we can do to improve this... -anil On 14 May 2013, at 16:51, Heidi Howard wrote: > Fixed, using "hash -r" >=20 > On 14 May 2013 16:21, Heidi Howard wrote: >> I'm trying to install OPAM on Ubuntu 12.04 64-bit using the package >> manager, but I don't understand why its not working, any suggestions = ? >>=20 >>=20 >> $ echo "deb [arch=3Damd64] http://www.recoil.org/~avsm/ wheezy main" = >> >> /etc/apt/sources.list >>=20 >> $ sudo apt-get install opam >> Reading package lists... Done >> Building dependency tree >> Reading state information... Done >> The following NEW packages will be installed >> opam >> 0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded. >> Need to get 2,598 kB of archives. >> After this operation, 7,478 kB of additional disk space will be used. >> WARNING: The following packages cannot be authenticated! >> opam >> Install these packages without verification [y/N]? y >> Get:1 http://www.recoil.org/~avsm/ wheezy/main opam amd64 0.9.6 = [2,598 kB] >> Fetched 2,598 kB in 0s (9,820 kB/s) >> Selecting previously unselected package opam. >> (Reading database ... 313487 files and directories currently = installed.) >> Unpacking opam (from .../archives/opam_0.9.6_amd64.deb) ... >> Processing triggers for man-db ... >> Setting up opam (0.9.6) ... >>=20 >> $ opam init >> bash: /usr/local/bin/opam: No such file or directory >>=20 >>=20 >>=20 >> -- >> Regards >> Heidi >=20 >=20 >=20 > --=20 > Regards > Heidi >=20 From daniel.buenzli@erratique.ch Tue May 14 17:51:56 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcISC-0001Ty-4U (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 17:51:56 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1481731 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail6.webfaction.com ([74.55.86.74]:34649 helo=smtp.webfaction.com) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UcISB-0008Ol-6b (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 17:51:56 +0100 Received: from [10.80.118.32] (firewall.ctxuk.citrix.com [46.33.159.2]) by smtp.webfaction.com (Postfix) with ESMTP id 4E28E20FAF06; Tue, 14 May 2013 16:51:47 +0000 (UTC) Date: Tue, 14 May 2013 17:51:48 +0100 From: =?utf-8?Q?Daniel_B=C3=BCnzli?= To: Anil Madhavapeddy Message-ID: In-Reply-To: References: <1F2747C1-F6BF-4A33-92B7-6F028A0C3574@gazagnaire.org> Subject: Re: a cool demo of Akka (Scala's actor) monitors X-Mailer: sparrow 1.6.4 (build 1178) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Thomas Gazagnaire , Mirage List X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 16:51:56 -0000 Content-Length: 739 Lines: 15 > So Daniel, here's the challenge for Vg graphs... :-) Well in fact the graphing part is quite small in there and could already be handled by Vg's current state. The challenge is more in the interactive aspect and the way you code that interface --- the code behind a very simple ui like that [1] is quite horrible [2] --- I would be really interested in giving a try at building good declarative and composable abstractions for client side web uis with js_of_ocaml/react/vg but I'm afraid that won't fit during my remaining time here at Cambridge (though some steps in that direction will be made). Best, Daniel [1] http://erratique.ch/software/vg/demos/rhtmlc [2] https://github.com/dbuenzli/vg/blob/master/test/rhtmlc.ml#L103 From stas@FreeBSD.org Tue May 14 18:10:39 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcIkJ-0001ix-D0 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 18:10:39 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1481731 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from backbone.deglitch.com ([78.110.53.255]:43445 helo=mx0.deglitch.com) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UcIkD-0008Ts-1A (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 18:10:39 +0100 Received: from [192.168.1.2] (111.sub-70-197-16.myvzw.com [70.197.16.111]) by mx0.deglitch.com (Postfix) with ESMTPSA id 71FD28FC2B; Tue, 14 May 2013 21:10:10 +0400 (MSK) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: "The Kernel is the Problem, Not the Solution" From: Stanislav Sedov In-Reply-To: Date: Tue, 14 May 2013 10:10:04 -0700 Content-Transfer-Encoding: 7bit Message-Id: References: <47BCE060-D312-4F93-9748-F4AB139A0089@eu.citrix.com> <20130514014858.f2907d2584b263399d627498@FreeBSD.org> To: Anil Madhavapeddy X-Mailer: Apple Mail (2.1503) Cc: Balraj Singh , Stephen Dolan , Dave Scott , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 17:10:39 -0000 Content-Length: 1346 Lines: 31 On May 14, 2013, at 4:57 AM, Anil Madhavapeddy wrote: > > I wouldn't be surprised at all to find many low-hanging scalability > problems from testing the Mirage-net stack against thousands (and then > millions) of connections. So we do need a few shell scripts to automate > the VM launch and load generation of the clients and server. Once the > issues from these are fixed, I would be very interested in using > SwiftTest as a regression test (i.e. once 10 million connections work, > they should stay working). > I agree. It's a pity there're no opensource easy-to-use load generators available. I has actually started working on a load generator based on the packet description language presented in your PhD a while ago, but didn't make a lot of progress. The idea was to make a traffic generator that will rely on the packet descriptions as a specification for the protocol (so it is not hardcoded and it will be easy to extend) and make it easy to specify the actions clients make via some kind of JIT (or AOT) compiled scripted language. If something like a netmap backend is used, I believe it will be possible to get a very decent performance out of it (most likely much better than commercial load generators with hardwired protocol implementations and informally coded state machines). -- ST4096-RIPE From Richard.Mortier@nottingham.ac.uk Tue May 14 18:42:13 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcJEr-0002hA-Uq (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 18:42:13 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1481731 * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine03-20433-3.icritical.com ([93.95.15.170]:42893) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1UcJEr-0006rB-7Y (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 18:42:13 +0100 Received: (qmail 14077 invoked from network); 14 May 2013 17:42:08 -0000 Received: from localhost (127.0.0.1) by engine03-20433-3.icritical.com with SMTP; 14 May 2013 17:42:08 -0000 Received: from engine03-20433-3.icritical.com ([127.0.0.1]) by localhost (engine03-20433-3.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 13903-02 for ; Tue, 14 May 2013 18:42:06 +0100 (BST) Received: (qmail 13466 invoked by uid 599); 14 May 2013 17:40:24 -0000 Received: from unknown (HELO smtp4.nottingham.ac.uk) (128.243.220.65) by engine03-20433-3.icritical.com (qpsmtpd/0.28) with ESMTP; Tue, 14 May 2013 18:40:24 +0100 Received: from uiwexhub01.ad.nottingham.ac.uk ([128.243.15.133]) by smtp4.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1UcJD9-0007R6-Gb; Tue, 14 May 2013 18:40:27 +0100 From: Richard Mortier To: Anil Madhavapeddy Date: Tue, 14 May 2013 18:40:26 +0100 Subject: Re: Problems Installing OPAM Thread-Topic: Problems Installing OPAM Thread-Index: Ac5Qyh6b632YS/IPSnW1RxW7rMRoPQ== Message-ID: <0104D85B-66F7-4096-B578-EBAD6F7F4235@nottingham.ac.uk> References: In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine03-20433-3.icritical.com Cc: "cl-mirage@lists.cam.ac.uk" , Heidi Howard X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 17:42:14 -0000 Content-Length: 2781 Lines: 82 looked to me more like it had once been run from a local install in u-l-b a= nd then heidi had installed the .debs which put it in u-b On 14 May 2013, at 17:17, Anil Madhavapeddy wrote: > This is a new one on me. I'm not sure that OPAM can do anything to rebui= ld the shell cache. But it's very very weird that invoking OPAM from the s= hell would work, and then a subshell fails to find the binary that launched= it! >=20 > So, hrm, not sure what we can do to improve this... >=20 > -anil >=20 > On 14 May 2013, at 16:51, Heidi Howard wrote: >=20 >> Fixed, using "hash -r" >>=20 >> On 14 May 2013 16:21, Heidi Howard wrote: >>> I'm trying to install OPAM on Ubuntu 12.04 64-bit using the package >>> manager, but I don't understand why its not working, any suggestions ? >>>=20 >>>=20 >>> $ echo "deb [arch=3Damd64] http://www.recoil.org/~avsm/ wheezy main" >> >>> /etc/apt/sources.list >>>=20 >>> $ sudo apt-get install opam >>> Reading package lists... Done >>> Building dependency tree >>> Reading state information... Done >>> The following NEW packages will be installed >>> opam >>> 0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded. >>> Need to get 2,598 kB of archives. >>> After this operation, 7,478 kB of additional disk space will be used. >>> WARNING: The following packages cannot be authenticated! >>> opam >>> Install these packages without verification [y/N]? y >>> Get:1 http://www.recoil.org/~avsm/ wheezy/main opam amd64 0.9.6 [2,598 = kB] >>> Fetched 2,598 kB in 0s (9,820 kB/s) >>> Selecting previously unselected package opam. >>> (Reading database ... 313487 files and directories currently installed.= ) >>> Unpacking opam (from .../archives/opam_0.9.6_amd64.deb) ... >>> Processing triggers for man-db ... >>> Setting up opam (0.9.6) ... >>>=20 >>> $ opam init >>> bash: /usr/local/bin/opam: No such file or directory >>>=20 >>>=20 >>>=20 >>> -- >>> Regards >>> Heidi >>=20 >>=20 >>=20 >> --=20 >> Regards >> Heidi >>=20 >=20 >=20 --=20 Cheers, R. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From anil@recoil.org Tue May 14 21:21:11 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcLih-00064v-Dp (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 21:21:11 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1481731 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:6224 helo=dark.recoil.org) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UcLig-0000oU-gb (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 21:21:11 +0100 Received: (qmail 12823 invoked by uid 634); 14 May 2013 20:21:09 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.48]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 14 May 2013 21:21:09 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Problems Installing OPAM From: Anil Madhavapeddy In-Reply-To: <0104D85B-66F7-4096-B578-EBAD6F7F4235@nottingham.ac.uk> Date: Tue, 14 May 2013 21:21:08 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <0104D85B-66F7-4096-B578-EBAD6F7F4235@nottingham.ac.uk> To: Richard Mortier X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk" , Heidi Howard X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 20:21:11 -0000 Content-Length: 3493 Lines: 112 Ah yes, that's right. The solution is basically to get the Debian = packages upstreamed properly so that a source install is never required. = This has been blocked due to the 8 month (!) upstream Debian release = freeze, but this ended last week. I've pinged the Debian issue on = Github to understand the current state of play. It seems that getting a = new package into Debian is quite an epic year-long endeavour... https://github.com/OCamlPro/opam/issues/149 -anil On 14 May 2013, at 18:40, Richard Mortier = wrote: > looked to me more like it had once been run from a local install in = u-l-b and then heidi had installed the .debs which put it in u-b >=20 > On 14 May 2013, at 17:17, Anil Madhavapeddy wrote: >=20 >> This is a new one on me. I'm not sure that OPAM can do anything to = rebuild the shell cache. But it's very very weird that invoking OPAM = from the shell would work, and then a subshell fails to find the binary = that launched it! >>=20 >> So, hrm, not sure what we can do to improve this... >>=20 >> -anil >>=20 >> On 14 May 2013, at 16:51, Heidi Howard wrote: >>=20 >>> Fixed, using "hash -r" >>>=20 >>> On 14 May 2013 16:21, Heidi Howard wrote: >>>> I'm trying to install OPAM on Ubuntu 12.04 64-bit using the package >>>> manager, but I don't understand why its not working, any = suggestions ? >>>>=20 >>>>=20 >>>> $ echo "deb [arch=3Damd64] http://www.recoil.org/~avsm/ wheezy = main" >> >>>> /etc/apt/sources.list >>>>=20 >>>> $ sudo apt-get install opam >>>> Reading package lists... Done >>>> Building dependency tree >>>> Reading state information... Done >>>> The following NEW packages will be installed >>>> opam >>>> 0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded. >>>> Need to get 2,598 kB of archives. >>>> After this operation, 7,478 kB of additional disk space will be = used. >>>> WARNING: The following packages cannot be authenticated! >>>> opam >>>> Install these packages without verification [y/N]? y >>>> Get:1 http://www.recoil.org/~avsm/ wheezy/main opam amd64 0.9.6 = [2,598 kB] >>>> Fetched 2,598 kB in 0s (9,820 kB/s) >>>> Selecting previously unselected package opam. >>>> (Reading database ... 313487 files and directories currently = installed.) >>>> Unpacking opam (from .../archives/opam_0.9.6_amd64.deb) ... >>>> Processing triggers for man-db ... >>>> Setting up opam (0.9.6) ... >>>>=20 >>>> $ opam init >>>> bash: /usr/local/bin/opam: No such file or directory >>>>=20 >>>>=20 >>>>=20 >>>> -- >>>> Regards >>>> Heidi >>>=20 >>>=20 >>>=20 >>> --=20 >>> Regards >>> Heidi >>>=20 >>=20 >>=20 >=20 >=20 > --=20 > Cheers, >=20 > R. >=20 >=20 >=20 >=20 > This message and any attachment are intended solely for the addressee = and may contain confidential information. If you have received this = message in error, please send it back to me, and immediately delete it. = Please do not use, copy or disclose the information contained in this = message or in any attachment. Any views or opinions expressed by the = author of this email do not necessarily reflect the views of the = University of Nottingham. >=20 > This message has been checked for viruses but the contents of an = attachment > may still contain software viruses which could damage your computer = system: > you are advised to perform your own checks. Email communications with = the > University of Nottingham may be monitored as permitted by UK = legislation. >=20 From Richard.Mortier@nottingham.ac.uk Tue May 14 21:48:19 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcM8x-0006qU-8T (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 21:48:19 +0100 X-Cam-SpamScore: s X-Cam-SpamDetails: score 1.1 from SpamAssassin-3.3.2-1481731 * 0.0 HTML_MESSAGE BODY: HTML included in message * 1.1 MIME_HTML_ONLY BODY: Message only has text/html MIME parts X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from engine06-20433-6.icritical.com ([195.62.217.150]:56115) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UcM8w-0004kC-EU (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 14 May 2013 21:48:19 +0100 Received: (qmail 15806 invoked from network); 14 May 2013 20:48:09 -0000 Received: from localhost (127.0.0.1) by engine06-20433-6.icritical.com with SMTP; 14 May 2013 20:48:09 -0000 Received: from engine06-20433-6.icritical.com ([127.0.0.1]) by localhost (engine06-20433-6.icritical.com [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 15363-04 for ; Tue, 14 May 2013 21:48:06 +0100 (BST) Received: (qmail 15793 invoked by uid 599); 14 May 2013 20:48:06 -0000 Received: from unknown (HELO smtp4.nottingham.ac.uk) (128.243.220.65) by engine06-20433-6.icritical.com (qpsmtpd/0.28) with ESMTP; Tue, 14 May 2013 21:48:06 +0100 Received: from uiwexhub01.ad.nottingham.ac.uk ([128.243.15.133]) by smtp4.nottingham.ac.uk with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1UcM8s-00022N-Ey; Tue, 14 May 2013 21:48:14 +0100 From: Richard Mortier To: "anil@recoil.org" Date: Tue, 14 May 2013 21:48:13 +0100 Subject: Re: Problems Installing OPAM Thread-Topic: Problems Installing OPAM Thread-Index: Ac5Q4JlNOmaQ0L/cQRiaKtGkGskpgwAA8Dfe Message-ID: <2sx5b5t8h77nqpu1t70ycw6s.1368564489162@email.android.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: by iCritical at engine06-20433-6.icritical.com Cc: "richard.mortier@nottingham.ac.uk" , "cl-mirage@lists.cam.ac.uk" , "hh360@cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 20:48:19 -0000 Content-Length: 5322 Lines: 140 Debian is, ironically, famously stable :)  (unlike far too many = other linux distros...)

Sent from Sams= ung Mobile


Anil Madhavapeddy <anil@recoil.org> wro= te:



=

=
Ah yes, that's right.  The s= olution is basically to get the Debian packages upstreamed properly so that= a source install is never required.  This has been blocked due to the= 8 month (!) upstream Debian release freeze, but this ended last week. = ; I've pinged the Debian issue on Github to understand the current state of= play.  It seems that getting a new package into Debian is quite an ep= ic year-long endeavour...

https://github.com/= OCamlPro/opam/issues/149

-anil


On 14 May 2013, at 18:40, Richard Mortier <Richard.Mortier@nottingham.ac= .uk> wrote:

> looked to me more like it had once been run from a local install in u-= l-b and then heidi had installed the .debs which put it in u-b
>
> On 14 May 2013, at 17:17, Anil Madhavapeddy wrote:
>
>> This is a new one on me.  I'm not sure that OPAM can do anyth= ing to rebuild the shell cache.  But it's very very weird that invokin= g OPAM from the shell would work, and then a subshell fails to find the bin= ary that launched it!
>>
>> So, hrm, not sure what we can do to improve this...
>>
>> -anil
>>
>> On 14 May 2013, at 16:51, Heidi Howard <hh360@cam.ac.uk> wro= te:
>>
>>> Fixed, using "hash -r"
>>>
>>> On 14 May 2013 16:21, Heidi Howard <hh360@cam.ac.uk> wro= te:
>>>> I'm trying to install OPAM on Ubuntu 12.04 64-bit using th= e package
>>>> manager, but I don't understand why its not working, any s= uggestions ?
>>>>
>>>>
>>>> $ echo "deb [arch=3Damd64] http://www.recoil.org/~avsm/ wheezy main" >><= br> >>>> /etc/apt/sources.list
>>>>
>>>> $ sudo apt-get install opam
>>>> Reading package lists... Done
>>>> Building dependency tree
>>>> Reading state information... Done
>>>> The following NEW packages will be installed
>>>> opam
>>>> 0 upgraded, 1 newly installed, 0 to remove and 6 not upgra= ded.
>>>> Need to get 2,598 kB of archives.
>>>> After this operation, 7,478 kB of additional disk space wi= ll be used.
>>>> WARNING: The following packages cannot be authenticated! >>>> opam
>>>> Install these packages without verification [y/N]? y
>>>> Get:1 http://www.= recoil.org/~avsm/ wheezy/main opam amd64 0.9.6 [2,598 kB]
>>>> Fetched 2,598 kB in 0s (9,820 kB/s)
>>>> Selecting previously unselected package opam.
>>>> (Reading database ... 313487 files and directories current= ly installed.)
>>>> Unpacking opam (from .../archives/opam_0.9.6_amd64.deb) ..= .
>>>> Processing triggers for man-db ...
>>>> Setting up opam (0.9.6) ...
>>>>
>>>> $ opam init
>>>> bash: /usr/local/bin/opam: No such file or directory
>>>>
>>>>
>>>>
>>>> --
>>>> Regards
>>>> Heidi
>>>
>>>
>>>
>>> --
>>> Regards
>>> Heidi
>>>
>>
>>
>
>
> --
> Cheers,
>
> R.
>
>
>
>
> This message and any attachment are intended solely for the addressee = and may contain confidential information. If you have received this message= in error, please send it back to me, and immediately delete it.  = ; Please do not use, copy or disclose the information contained in this mes= sage or in any attachment.  Any views or opinions expressed by the aut= hor of this email do not necessarily reflect the views of the University of= Nottingham.
>
> This message has been checked for viruses but the contents of an attac= hment
> may still contain software viruses which could damage your computer sy= stem:
> you are advised to perform your own checks. Email communications with = the
> University of Nottingham may be monitored as permitted by UK legislati= on.
>

From stedolan@stedolan.net Wed May 15 09:50:19 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcXPf-0006cg-4C (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 15 May 2013 09:50:19 +0100 X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1482254 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.215.43 listed in list.dnswl.dnsbl.ja.net] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-la0-f43.google.com ([209.85.215.43]:36866) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UcXPe-0003iT-2D (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 15 May 2013 09:50:19 +0100 Received: by mail-la0-f43.google.com with SMTP id ez20so305823lab.30 for ; Wed, 15 May 2013 01:50:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:sender:x-originating-ip:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:x-gm-message-state; bh=c5TfwT18wlCVjpkJKfwBGHUKsOaDzFXXyCGXV9LegKU=; b=oHlwV3mXAIyLMtnhGxWHrjRBLKe2BxyK+uIpfGZ4X7nrtgg9SBHFL66bGd8lHzniQ+ k1wlUF6Qsrcp7bflMQNLZ0J5jWa7931zHxg3BmckMfTJdDGO42mEhu0pH1Ub6Uop7eGZ 5BlPgHGYlCOYVmVw9jbTEJUhmhplEaQOqKvcl/Od07DVXH6hbFceBMZQO+DMCiDSj5rb Av5lKJkelWPDrgNzAoHhccRN6PM5YAh5UDYudXozXa56O944L8iB5PgnvcVt4Y/Eplqn r3cfapJuss1HQXP36HRkywvAU1RUqmD8RFGoCkhKo3aceG/gTLAJfbcdtgPt5HAmJRRc zPqA== MIME-Version: 1.0 X-Received: by 10.112.126.226 with SMTP id nb2mr17338596lbb.38.1368607818123; Wed, 15 May 2013 01:50:18 -0700 (PDT) Sender: stedolan@stedolan.net Received: by 10.114.160.136 with HTTP; Wed, 15 May 2013 01:50:18 -0700 (PDT) X-Originating-IP: [131.111.184.8] In-Reply-To: References: <0104D85B-66F7-4096-B578-EBAD6F7F4235@nottingham.ac.uk> Date: Wed, 15 May 2013 09:50:18 +0100 X-Google-Sender-Auth: M04h2iwLjzsLEPzlImg_DAGUBzo Message-ID: Subject: Re: Problems Installing OPAM From: Stephen Dolan To: Anil Madhavapeddy Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQmkoiZ2bzgRbzUvalCriG8dJxxYtracf5qWEQmLXZSLfk/jTd0lOgYhqnDd3SermS+wzIs8 Cc: Richard Mortier , "cl-mirage@lists.cam.ac.uk" , Heidi Howard X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2013 08:50:19 -0000 Content-Length: 645 Lines: 11 On Tue, May 14, 2013 at 9:21 PM, Anil Madhavapeddy wrote: > Ah yes, that's right. The solution is basically to get the Debian packages upstreamed properly so that a source install is never required. This has been blocked due to the 8 month (!) upstream Debian release freeze, but this ended last week. That doesn't block new package uploads, it just blocks them from getting into the Debian "stable" distribution. New packages arrive in "testing" or "unstable" where they hang around until the next stable release. So getting stuff into Debian is slow, but it's not like it's been completely stopped for 8 months. Stephen From anil@recoil.org Wed May 15 11:12:32 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcYhE-0004bk-EE (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 15 May 2013 11:12:32 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1482254 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:45808 helo=dark.recoil.org) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UcYhD-0003kK-2r (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 15 May 2013 11:12:32 +0100 Received: (qmail 12669 invoked by uid 634); 15 May 2013 10:12:31 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from no-dns-yet.demon.co.uk (HELO [192.168.14.242]) (62.49.66.12) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 15 May 2013 11:12:30 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Problems Installing OPAM From: Anil Madhavapeddy In-Reply-To: Date: Wed, 15 May 2013 11:12:29 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <0104D85B-66F7-4096-B578-EBAD6F7F4235@nottingham.ac.uk> To: Stephen Dolan X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Richard Mortier , "cl-mirage@lists.cam.ac.uk" , Heidi Howard X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2013 10:12:32 -0000 Content-Length: 950 Lines: 24 On 15 May 2013, at 09:50, Stephen Dolan = wrote: > On Tue, May 14, 2013 at 9:21 PM, Anil Madhavapeddy = wrote: >> Ah yes, that's right. The solution is basically to get the Debian = packages upstreamed properly so that a source install is never required. = This has been blocked due to the 8 month (!) upstream Debian release = freeze, but this ended last week. >=20 > That doesn't block new package uploads, it just blocks them from > getting into the Debian "stable" distribution. New packages arrive in > "testing" or "unstable" where they hang around until the next stable > release. So getting stuff into Debian is slow, but it's not like it's > been completely stopped for 8 months. There seems to be a lack of bandwidth for handling the NEW queue heading = up to a release: http://ftp-master.debian.org/stat.html ...but it appears that we're not even in the NEW queue yet. Woops... -anil= From heidihoward360@gmail.com Thu May 16 08:48:33 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcsvR-0005D7-2F (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 16 May 2013 08:48:33 +0100 X-Cam-SpamDetails: score -0.3 from SpamAssassin-3.3.2-1482727 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.216.181 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (heidihoward360[at]gmail.com) * 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in * digit (heidihoward360[at]gmail.com) * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-qc0-f181.google.com ([209.85.216.181]:60841) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UcsvQ-0002Ag-Es (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 16 May 2013 08:48:33 +0100 Received: by mail-qc0-f181.google.com with SMTP id s10so873548qcv.40 for ; Thu, 16 May 2013 00:48:32 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.49.104.7 with SMTP id ga7mr35662184qeb.27.1368690511934; Thu, 16 May 2013 00:48:31 -0700 (PDT) Sender: heidihoward360@gmail.com Received: by 10.49.107.36 with HTTP; Thu, 16 May 2013 00:48:31 -0700 (PDT) Date: Thu, 16 May 2013 08:48:31 +0100 X-Google-Sender-Auth: mYSv-igqrKlwhg57lkEj2_jsXik Message-ID: Subject: where to start with OPAM errors From: Heidi Howard To: cl-mirage@lists.cam.ac.uk Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 May 2013 07:48:33 -0000 Content-Length: 2024 Lines: 55 another day, another question :) $opam install exited erroneously with the following error. I have no idea really were to start to try to identify the issue, any suggestions ? Due to some errors while processing pcre-ocaml.7.0.2, the following actions will NOT be proceeded: - install core_extended.109.21.00 ==== ERROR [while installing pcre-ocaml.7.0.2] ==== # opam-version 1.0.0 # os linux # command make # path /auto/homes/hh360/.opam/4.00.1/build/pcre-ocaml.7.0.2 # exit-code 2 # env-file /auto/homes/hh360/.opam/4.00.1/build/pcre-ocaml.7.0.2/pcre-ocaml-865888.env # stdout-file /auto/homes/hh360/.opam/4.00.1/build/pcre-ocaml.7.0.2/pcre-ocaml-865888.out # stderr-file /auto/homes/hh360/.opam/4.00.1/build/pcre-ocaml.7.0.2/pcre-ocaml-865888.err ### stdout ### ...[truncated] with_pcre_config: .................................... pcre-config Strict compile-time checks: .......................... true Build examples: ...................................... true Create documentations: ............................... true Compile tests executable and library and run them: ... false ocamldoc: ............................................ /auto/homes/hh360/.opam/4.00.1/bin/ocamldoc ocaml setup.ml -build /auto/homes/hh360/.opam/4.00.1/bin/ocamlopt.opt -I /auto/homes/hh360/.opam/4.00.1/lib/ocaml/ocamlbuild unix.cmxa /auto/homes/hh360/.opam/4.00.1/lib/ocaml/ocamlbuild/ocamlbuildlib.cmxa myocamlbuild.ml /auto/homes/hh360/.opam/4.00.1/lib/ocaml/ocamlbuild/ocamlbuild.cmx -o myocamlbuild Exception End_of_file. ### stderr ### /bin/sh: 1: pcre-config: not found E: Failure("Command ''/auto/homes/hh360/.opam/4.00.1/bin/ocamlbuild' lib/libpcre_stubs.a lib/dllpcre_stubs.so lib/pcre.cma lib/pcre.cmxa lib/pcre.a lib/pcre.cmxs examples/cloc/cloc.native examples/count_hash/count_hash.native examples/pcregrep/pcregrep.native examples/subst/subst.native -tag debug' terminated with error code 100") make: *** [build] Error 1 Thank you so much for your help :) -- Regards Heidi From anil@recoil.org Thu May 16 08:54:52 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uct1Y-0005KI-HY (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 16 May 2013 08:54:52 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1482727 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:16011 helo=dark.recoil.org) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1Uct1Y-0004Aa-DF (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 16 May 2013 08:54:52 +0100 Received: (qmail 10986 invoked by uid 634); 16 May 2013 07:54:51 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.116]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 16 May 2013 08:54:51 +0100 References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <24EA297A-E135-469D-8854-4A8EFFDAFE87@recoil.org> X-Mailer: iPad Mail (10B329) From: Anil Madhavapeddy Subject: Re: where to start with OPAM errors Date: Thu, 16 May 2013 08:54:51 +0100 To: Heidi Howard X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 May 2013 07:54:52 -0000 Content-Length: 2363 Lines: 64 You need to install the PCRE C library; see the Real World OCaml installatio= n section. This is where "pcre-config" comes from. On 16 May 2013, at 08:48, Heidi Howard wrote: > another day, another question :) >=20 > $opam install exited erroneously with the following error. I have no > idea really were to start to try to identify the issue, any > suggestions ? >=20 > Due to some errors while processing pcre-ocaml.7.0.2, the following > actions will NOT be proceeded: > - install core_extended.109.21.00 >=20 > =3D=3D=3D=3D ERROR [while installing pcre-ocaml.7.0.2] =3D=3D=3D=3D > # opam-version 1.0.0 > # os linux > # command make > # path /auto/homes/hh360/.opam/4.00.1/build/pcre-ocaml.7.0.2 > # exit-code 2 > # env-file > /auto/homes/hh360/.opam/4.00.1/build/pcre-ocaml.7.0.2/pcre-ocaml-865888.en= v > # stdout-file > /auto/homes/hh360/.opam/4.00.1/build/pcre-ocaml.7.0.2/pcre-ocaml-865888.ou= t > # stderr-file > /auto/homes/hh360/.opam/4.00.1/build/pcre-ocaml.7.0.2/pcre-ocaml-865888.er= r > ### stdout ### > ...[truncated] > with_pcre_config: .................................... pcre-config > Strict compile-time checks: .......................... true > Build examples: ...................................... true > Create documentations: ............................... true > Compile tests executable and library and run them: ... false > ocamldoc: ............................................ > /auto/homes/hh360/.opam/4.00.1/bin/ocamldoc >=20 > ocaml setup.ml -build > /auto/homes/hh360/.opam/4.00.1/bin/ocamlopt.opt -I > /auto/homes/hh360/.opam/4.00.1/lib/ocaml/ocamlbuild unix.cmxa > /auto/homes/hh360/.opam/4.00.1/lib/ocaml/ocamlbuild/ocamlbuildlib.cmxa > myocamlbuild.ml > /auto/homes/hh360/.opam/4.00.1/lib/ocaml/ocamlbuild/ocamlbuild.cmx -o > myocamlbuild > Exception End_of_file. > ### stderr ### > /bin/sh: 1: pcre-config: not found > E: Failure("Command ''/auto/homes/hh360/.opam/4.00.1/bin/ocamlbuild' > lib/libpcre_stubs.a lib/dllpcre_stubs.so lib/pcre.cma lib/pcre.cmxa > lib/pcre.a lib/pcre.cmxs examples/cloc/cloc.native > examples/count_hash/count_hash.native > examples/pcregrep/pcregrep.native examples/subst/subst.native -tag > debug' terminated with error code 100") > make: *** [build] Error 1 >=20 > Thank you so much for your help :) > -- > Regards > Heidi >=20 From heidihoward360@gmail.com Thu May 16 08:59:31 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uct63-0005SD-QR (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 16 May 2013 08:59:31 +0100 X-Cam-SpamDetails: score -0.3 from SpamAssassin-3.3.2-1482727 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.216.175 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (heidihoward360[at]gmail.com) * 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in * digit (heidihoward360[at]gmail.com) * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-qc0-f175.google.com ([209.85.216.175]:50709) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1Uct63-00025K-17 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 16 May 2013 08:59:31 +0100 Received: by mail-qc0-f175.google.com with SMTP id a1so891846qcx.34 for ; Thu, 16 May 2013 00:59:30 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.229.105.222 with SMTP id u30mr13035802qco.142.1368691170650; Thu, 16 May 2013 00:59:30 -0700 (PDT) Sender: heidihoward360@gmail.com Received: by 10.49.107.36 with HTTP; Thu, 16 May 2013 00:59:30 -0700 (PDT) In-Reply-To: <24EA297A-E135-469D-8854-4A8EFFDAFE87@recoil.org> References: <24EA297A-E135-469D-8854-4A8EFFDAFE87@recoil.org> Date: Thu, 16 May 2013 08:59:30 +0100 X-Google-Sender-Auth: T_62aubp4dqWiHwlAsgBw83G6zk Message-ID: Subject: Re: where to start with OPAM errors From: Heidi Howard To: Anil Madhavapeddy Content-Type: text/plain; charset=ISO-8859-1 Cc: "cl-mirage@lists.cam.ac.uk" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 May 2013 07:59:31 -0000 Content-Length: 2490 Lines: 71 Success thanks anil On 16 May 2013 08:54, Anil Madhavapeddy wrote: > You need to install the PCRE C library; see the Real World OCaml installation section. This is where "pcre-config" comes from. > > On 16 May 2013, at 08:48, Heidi Howard wrote: > >> another day, another question :) >> >> $opam install exited erroneously with the following error. I have no >> idea really were to start to try to identify the issue, any >> suggestions ? >> >> Due to some errors while processing pcre-ocaml.7.0.2, the following >> actions will NOT be proceeded: >> - install core_extended.109.21.00 >> >> ==== ERROR [while installing pcre-ocaml.7.0.2] ==== >> # opam-version 1.0.0 >> # os linux >> # command make >> # path /auto/homes/hh360/.opam/4.00.1/build/pcre-ocaml.7.0.2 >> # exit-code 2 >> # env-file >> /auto/homes/hh360/.opam/4.00.1/build/pcre-ocaml.7.0.2/pcre-ocaml-865888.env >> # stdout-file >> /auto/homes/hh360/.opam/4.00.1/build/pcre-ocaml.7.0.2/pcre-ocaml-865888.out >> # stderr-file >> /auto/homes/hh360/.opam/4.00.1/build/pcre-ocaml.7.0.2/pcre-ocaml-865888.err >> ### stdout ### >> ...[truncated] >> with_pcre_config: .................................... pcre-config >> Strict compile-time checks: .......................... true >> Build examples: ...................................... true >> Create documentations: ............................... true >> Compile tests executable and library and run them: ... false >> ocamldoc: ............................................ >> /auto/homes/hh360/.opam/4.00.1/bin/ocamldoc >> >> ocaml setup.ml -build >> /auto/homes/hh360/.opam/4.00.1/bin/ocamlopt.opt -I >> /auto/homes/hh360/.opam/4.00.1/lib/ocaml/ocamlbuild unix.cmxa >> /auto/homes/hh360/.opam/4.00.1/lib/ocaml/ocamlbuild/ocamlbuildlib.cmxa >> myocamlbuild.ml >> /auto/homes/hh360/.opam/4.00.1/lib/ocaml/ocamlbuild/ocamlbuild.cmx -o >> myocamlbuild >> Exception End_of_file. >> ### stderr ### >> /bin/sh: 1: pcre-config: not found >> E: Failure("Command ''/auto/homes/hh360/.opam/4.00.1/bin/ocamlbuild' >> lib/libpcre_stubs.a lib/dllpcre_stubs.so lib/pcre.cma lib/pcre.cmxa >> lib/pcre.a lib/pcre.cmxs examples/cloc/cloc.native >> examples/count_hash/count_hash.native >> examples/pcregrep/pcregrep.native examples/subst/subst.native -tag >> debug' terminated with error code 100") >> make: *** [build] Error 1 >> >> Thank you so much for your help :) >> -- >> Regards >> Heidi >> -- Regards Heidi From anil@recoil.org Thu May 16 14:39:43 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UcyPH-0004sY-1P (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 16 May 2013 14:39:43 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1482727 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:1901 helo=dark.recoil.org) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UcyPG-00036Z-gA (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 16 May 2013 14:39:43 +0100 Received: (qmail 12696 invoked by uid 634); 16 May 2013 13:39:41 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from volstagg-0.srg.cl.cam.ac.uk (HELO [10.0.1.75]) (128.232.32.232) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Thu, 16 May 2013 14:39:41 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Mirage/OCaml IRC channel Message-Id: <7B76DA4E-F66C-428A-8F3D-0E853443D055@recoil.org> Date: Thu, 16 May 2013 14:39:39 +0100 To: "cl-ocamllabs-staff@lists.cam.ac.uk Labs" , "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 May 2013 13:39:43 -0000 Content-Length: 602 Lines: 17 Just a reminder that despite being geographically distributed in terms = of countries (and offices within Cambridge), there is usually a set of = people hanging out on IRC. This is a useful place to idle and catch the chatter and calls for = lunch/pub if you are in Cambridge (and be intensely jealous if you are = stranded somewhere like a beach in Nice). We use the Freenode IRC network, and details on connecting are available = here: http://freenode.net/using_the_network.shtml The channel is #mirage, and usually includes local OCaml chatter as well = as Mirage-specific conversation. -anil= From amc79@cam.ac.uk Fri May 17 15:40:43 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UdLpr-0005AI-2l (Exim 4.70) (return-path ); Fri, 17 May 2013 15:40:43 +0100 X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from dhcp-128-232-140-2.eduroam.lapwing.cam.ac.uk ([128.232.140.2]:50358) by ppsw-33.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.157]:587) with esmtpsa (PLAIN:amc79) (TLSv1:AES128-SHA:128) id 1UdLpr-0003HV-g8 (Exim 4.80_167-5a66dd3) (return-path ); Fri, 17 May 2013 15:40:43 +0100 From: Amir Chaudhry Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: OCaml Labs Meeting - Friday 31st May at 4pm in the Computer Lab Message-Id: <5A7CD1F1-5A82-4FDC-AAB9-9402E7A42C4F@cam.ac.uk> Date: Fri, 17 May 2013 15:40:43 +0100 To: "cl-ocamllabs@lists.cam.ac.uk" , "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 May 2013 14:40:43 -0000 Content-Length: 682 Lines: 33 Dear all, The next OCaml Labs meeting will take place on the 31st of April at 4pm = in the Lab. Please note the change of time with respect to previous meetings. This also means we can continue discussions in the pub after the meeting = :) A skeleton agenda is below and a more detailed one will follow in = advance of the meeting. =20 Please do let me know if you will be attending. -- Details -- OCaml Labs Meeting 31st May 2013 4pm =96 5pm Room FW26 (TBC) - Cambridge Computer Laboratory William Gates Building JJ Thomson Avenue Cambridge CB3 0FD -- Agenda -- OCL Updates - Platform projects - Systems projects - Compiler projects Open discussion Close Best wishes, Amir= From amc79@cam.ac.uk Fri May 17 15:53:10 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UdM1u-0005Uh-MO (Exim 4.70) (return-path ); Fri, 17 May 2013 15:53:10 +0100 X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from dhcp-128-232-140-2.eduroam.lapwing.cam.ac.uk ([128.232.140.2]:50521) by ppsw-33.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.157]:587) with esmtpsa (PLAIN:amc79) (TLSv1:AES128-SHA:128) id 1UdM1u-0000k6-i3 (Exim 4.80_167-5a66dd3) (return-path ); Fri, 17 May 2013 15:53:10 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: OCaml Labs Meeting - Friday 31st May at 4pm in the Computer Lab From: Amir Chaudhry In-Reply-To: <5A7CD1F1-5A82-4FDC-AAB9-9402E7A42C4F@cam.ac.uk> Date: Fri, 17 May 2013 15:53:10 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5A7CD1F1-5A82-4FDC-AAB9-9402E7A42C4F@cam.ac.uk> To: "cl-ocamllabs@lists.cam.ac.uk" , "cl-mirage@lists.cam.ac.uk List" X-Mailer: Apple Mail (2.1503) X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 May 2013 14:53:10 -0000 Content-Length: 951 Lines: 42 Thanks to those who pointed out the error in my previous message. Of course, I did mean 31st of MAY - not April :) Best wishes, Amir On 17 May 2013, at 15:40, Amir Chaudhry wrote: > Dear all, >=20 > The next OCaml Labs meeting will take place on the 31st of MAY at 4pm = in the Lab. > Please note the change of time with respect to previous meetings. > This also means we can continue discussions in the pub after the = meeting :) >=20 > A skeleton agenda is below and a more detailed one will follow in = advance of the meeting. =20 >=20 > Please do let me know if you will be attending. >=20 > -- Details -- > OCaml Labs Meeting > 31st May 2013 > 4pm =96 5pm > Room FW26 (TBC) - Cambridge Computer Laboratory > William Gates Building > JJ Thomson Avenue > Cambridge CB3 0FD >=20 > -- Agenda -- > OCL Updates > - Platform projects > - Systems projects > - Compiler projects > Open discussion > Close >=20 > Best wishes, > Amir From anil@recoil.org Sun May 19 10:26:44 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Udzt6-0002sZ-9i (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 19 May 2013 10:26:44 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1484071 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:37074 helo=dark.recoil.org) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1Udzt5-0001WN-iD (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 19 May 2013 10:26:44 +0100 Received: (qmail 31651 invoked by uid 634); 19 May 2013 09:26:43 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from host81-149-102-120.in-addr.btopenworld.com (HELO clink-4.home) (81.149.102.120) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sun, 19 May 2013 10:26:42 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: cohttp tidyups for 0.9.8 From: Anil Madhavapeddy In-Reply-To: Date: Sun, 19 May 2013 10:26:40 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <20929310-C65C-461E-838F-EC33F814F0FD@recoil.org> References: To: Yaron Minsky , Jason Hickey , "cl-mirage@lists.cam.ac.uk List" , Leo White , Jeremy Yallop X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 May 2013 09:26:44 -0000 Content-Length: 2141 Lines: 45 I spent yesterday completely tidying up cohttp. It's now much simpler = and module'd up to the eyeballs, with very little redundancy (see below = for Lwt_unix, Mirage and Async interfaces). https://github.com/avsm/ocaml-cohttp/blob/master/lwt/cohttp_lwt_unix.mli https://github.com/avsm/ocaml-cohttp/blob/master/lwt/cohttp_mirage.mli https://github.com/avsm/ocaml-cohttp/blob/master/async/cohttp_async.mli The nice thing per backend is that you can easily see what is extended = in an OS-specific way. For example, in the UNIX backend, the file = functions are all separate: https://github.com/avsm/ocaml-cohttp/blob/master/lwt/cohttp_lwt_unix.ml The "trick" I learnt from Jeremy and Leo is to avoid functor hell by = defining more module types to stop the functors propagating. This is = what all the 'module type S' signatures are, and then a functor just = satisfies that signature. Kind of obvious now that I see it, but we = don't use this pattern enough in our protocol libraries! I've queued up = a blog post to write about this more before the cohttp-1.0 release. Before I cut the next release in a few days, I'm going to: - add a simple ocurl client to satisfy the Client functor with Async - sync this version with the RWO Async examples (should be very minor = changes) - add a very high-level Http_client functor for Async and Lwt that can = be used by client libraries (e.g. the Github/Facebook/Google bindings) = that just want to fetch webpages simply. Dave, do you want to add your POSIX direct-fd version in a unix = subdirectory at all? It should be quite straightforward now if you = follow the Async template. Yaron, I did a (very) quick performance test with httperf of the Async = and Lwt_unix servers in the lib_test directory, and noticed a = significant (3-4x) response rate performance drop in the Async server. = Need to queue up some profiling here at some point. One major downside to this approach is that ocamldoc simply falls over. = Luckily, Leo has an upcoming and much improved ocamldoc that can resolve = complex module includes and have decent HTML output. More on that = later... -anil= From anil@recoil.org Sun May 19 10:30:54 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Udzx7-0002vD-Ud (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 19 May 2013 10:30:53 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1484071 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:22916 helo=dark.recoil.org) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1Udzx7-0002Xz-gX (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 19 May 2013 10:30:53 +0100 Received: (qmail 14514 invoked by uid 634); 19 May 2013 09:30:53 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from host81-149-102-120.in-addr.btopenworld.com (HELO clink-4.home) (81.149.102.120) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sun, 19 May 2013 10:30:52 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: cohttp tidyups for 0.9.8 From: Anil Madhavapeddy In-Reply-To: <20929310-C65C-461E-838F-EC33F814F0FD@recoil.org> Date: Sun, 19 May 2013 10:30:53 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20929310-C65C-461E-838F-EC33F814F0FD@recoil.org> To: Yaron Minsky , Jason Hickey , "cl-mirage@lists.cam.ac.uk List" , Leo White , Jeremy Yallop X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 May 2013 09:30:54 -0000 Content-Length: 519 Lines: 17 On 19 May 2013, at 10:26, Anil Madhavapeddy wrote: > I spent yesterday completely tidying up cohttp. It's now much simpler = and module'd up to the eyeballs, with very little redundancy (see below = for Lwt_unix, Mirage and Async interfaces). >=20 > = https://github.com/avsm/ocaml-cohttp/blob/master/lwt/cohttp_lwt_unix.mli > https://github.com/avsm/ocaml-cohttp/blob/master/lwt/cohttp_mirage.mli correction; = https://github.com/avsm/ocaml-cohttp/blob/master/lwt/cohttp_lwt_mirage.mli= -anil= From anil@recoil.org Sun May 19 22:06:15 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UeAo3-000333-25 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 19 May 2013 22:06:15 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1484071 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:38892 helo=dark.recoil.org) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UeAo2-00025e-1A (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 19 May 2013 22:06:15 +0100 Received: (qmail 23292 invoked by uid 634); 19 May 2013 21:06:14 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.48]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sun, 19 May 2013 22:06:13 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Reminder: Mirage weekly call - 14 May at 1600 BST From: Anil Madhavapeddy In-Reply-To: Date: Sun, 19 May 2013 22:06:13 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <49CADCC7-33A1-4E43-B51E-2DF266E50E0A@gmail.com> <9E08CA48-929E-4BB9-88C6-312B379A088A@recoil.org> <3547678E-4F38-4F89-99F3-2DBFB1D4D088@recoil.org> To: "cl-mirage@lists.cam.ac.uk List" , Jon Ludlam X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 May 2013 21:06:15 -0000 Content-Length: 3365 Lines: 103 Minutes for this are now on the website: http://www.openmirage.org/wiki/weekly-2013-05-14 -anil On 14 May 2013, at 09:06, Anil Madhavapeddy wrote: > Agenda: > - Ocamlot update and use in Mirage (Anil, David Sheets) > - Actors: Dave's been working on this with his message switch > - Performance testing (all!) > - Irminsule demo for next week (Thomas/Anil) > - Follow-up on various other items in last two discussions as needed >=20 > -- Go To Meeting details -- >=20 > You can either use the online GoToMeeting or dial in via the numbers = below: >=20 > 1. Please join my meeting. > https://www1.gotomeeting.com/join/211547328 >=20 > 2. Use your microphone and speakers (VoIP) - a headset is = recommended. Or, call in using your telephone. >=20 > United States: +1 (224) 649-0001 > Argentina (toll-free): 0 800 266 1382 > Australia (toll-free): 1 800 193 385 > Australia: +61 2 8355 1020 > Austria (toll-free): 0 800 202148 > Austria: +43 (0) 7 2088 1047 > Bahrain (toll-free): 800 81 111 > Belarus (toll-free): 8 820 0011 0214 > Belgium (toll-free): 0 800 26116 > Belgium: +32 (0) 28 08 4368 > Brazil (toll-free): 0 800 047 4906 > Canada (toll-free): 1 888 455 1389 > Canada: +1 (647) 497-9353 > China (toll-free): 4008 811084 > Czech Republic (toll-free): 800 500448 > Denmark (toll-free): 8090 1924 > Denmark: +45 (0) 69 91 89 28 > Finland (toll-free): 80094507 > Finland: +358 (0) 942 59 7850 > France (toll-free): 0 805 541 047 > France: +33 (0) 170 950 594 > Germany (toll-free): 0 800 723 5270 > Germany: +49 (0) 811 8899 6903 > Hong Kong (toll-free): 30713169 > Iceland (toll-free): 800 9869 > India (toll-free): 000 800 100 7855 > Indonesia (toll-free): 007 803 011 0395 > Ireland (toll-free): 1 800 946 538 > Ireland: +353 (0) 19 030 010 > Israel (toll-free): 1 809 212 875 > Italy (toll-free): 800 906959 > Italy: +39 0 247 92 13 01 > Japan (toll-free): 00 120 663 800 > Korea, Republic of (toll-free): 806150880 > Luxembourg (toll-free): 800 22104 > Malaysia (toll-free): 1 800 81 5373 > Mexico (toll-free): 01 800 925 0372 > Netherlands (toll-free): 0 800 265 8469 > Netherlands: +31 (0) 208 080 219 > New Zealand (toll-free): 0 800 45 2202 > New Zealand: +64 (0) 9 280 6302 > Norway (toll-free): 800 30 257 > Norway: +47 75 80 32 07 > Panama (toll-free): 00 800 226 8832 > Peru (toll-free): 0 800 54682 > Philippines (toll-free): 1 800 1651 0716 > Poland (toll-free): 00 800 1213979 > Portugal (toll-free): 800 784 461 > Russian Federation (toll-free): 810 800 29674011 > Saudi Arabia (toll-free): +9668008443633 > Singapore (toll-free): 800 120 5615 > South Africa (toll-free): 0 800 983 867 > Spain (toll-free): 0 800 900 582 > Spain: +34 911 82 9906 > Sweden (toll-free): 020 980 772 > Sweden: +46 (0) 852 500 186 > Switzerland (toll-free): 0 800 740 393 > Switzerland: +41 (0) 435 0167 13 > Taiwan (toll-free): 00 800 666 854 > Thailand (toll-free): 001 800 658 131 > Turkey (toll-free): +90800448823683 > Ukraine (toll-free): 0 800 50 0641 > United Arab Emirates (toll-free): +97180004440439 > United Kingdom (toll-free): 0 808 168 0229 > United Kingdom: +44 (0) 207 151 1853 > United States (toll-free): 1 877 309 2073 > Uruguay (toll-free): 000 413 598 4110 > Viet Nam (toll-free): 120 65 157 >=20 > Access Code: 211-547-328 > Audio PIN: Shown after joining the meeting >=20 > Meeting ID: 211-547-328 >=20 >=20 >=20 From pmundkur.ocaml@gmail.com Mon May 20 17:56:25 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UeTNp-0006YA-Kb (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 20 May 2013 17:56:25 +0100 X-Cam-SpamDetails: score 0.6 from SpamAssassin-3.3.2-1484242 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.160.53 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (pmundkur.ocaml[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 1.2 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-pb0-f53.google.com ([209.85.160.53]:46948) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UeTNo-00008q-9e (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 20 May 2013 17:56:25 +0100 Received: by mail-pb0-f53.google.com with SMTP id un4so1026193pbc.12 for ; Mon, 20 May 2013 09:56:24 -0700 (PDT) X-Received: by 10.68.171.36 with SMTP id ar4mr33545003pbc.195.1369068983897; Mon, 20 May 2013 09:56:23 -0700 (PDT) Received: from damage (visnet-156.csl.sri.com. [130.107.98.156]) by mx.google.com with ESMTPSA id zy5sm24778459pbb.43.2013.05.20.09.56.22 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 20 May 2013 09:56:22 -0700 (PDT) Date: Mon, 20 May 2013 09:56:20 -0700 From: Prashanth Mundkur To: Anil Madhavapeddy Subject: Re: https://polarssl.org/ Message-ID: <20130520165620.GA22270@damage> References: <5181591B.1070004@luminar.eu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: cl-mirage@lists.cam.ac.uk, Vincent Bernardoff X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 16:56:25 -0000 Content-Length: 770 Lines: 19 Just came across this BSD licensed library http://axtls.sourceforge.net/index.htm On 19:07 Wed 01 May, Anil Madhavapeddy wrote: > On 1 May 2013, at 19:04, Vincent Bernardoff wrote: > > > Another embeddded SSL lib, I just found out that mini-os is using it. Might be worth a look if not already done ? > > > > It's also dual commercial GPLv2 licensed, unfortunately, and so roughly the same as MatrixSSL. I've not looked in more depth though, as MatrixSSL has a really nice low-level API that is perfect for our binding needs... > > Jon, is your WIP on Github anywhere? I do think it would be easier to have something on UNIX ahead of the mirage-platform version, if only to run it through valgrind and bind to Async too... > > -anil > > From Dave.Scott@eu.citrix.com Tue May 21 10:49:58 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UejCg-0004zi-CS (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 21 May 2013 10:49:58 +0100 X-Cam-SpamDetails: score -1.1 from SpamAssassin-3.3.2-1484242 * -1.1 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from smtp.eu.citrix.com ([46.33.159.39]:47736) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UejCf-0000Mi-iK (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 21 May 2013 10:49:58 +0100 X-IronPort-AV: E=Sophos;i="4.87,713,1363132800"; d="scan'208";a="4797085" Received: from lonpex01cl02.citrite.net ([10.30.203.102]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/AES128-SHA; 21 May 2013 09:49:58 +0000 Received: from LONPEX01CL03.citrite.net ([169.254.3.241]) by LONPEX01CL02.citrite.net ([169.254.2.133]) with mapi id 14.02.0342.003; Tue, 21 May 2013 10:49:57 +0100 From: Dave Scott To: "cl-mirage@lists.cam.ac.uk" , Jonathan Ludlam Subject: Reminder: mirage weekly call Thread-Topic: Reminder: mirage weekly call Thread-Index: Ac5WCAmSjyMLhDZ7R/m+uPDZhUuyeQ== Date: Tue, 21 May 2013 09:49:56 +0000 Message-ID: <6FB4516F0E9B0F43B54F88D855ABB79010E814@LONPEX01CL03.citrite.net> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.30.203.1] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 May 2013 09:49:58 -0000 Content-Length: 2962 Lines: 92 Hi, Anil and Thomas are stuck in London so I'll run this weeks meeting. So far I have the following agenda items, feel free to suggest more: * quick mirari update? (Vincent) * should we use jenga? (Dave) -- what else? -- Go To Meeting details -- You can either use the online GoToMeeting or dial in via the numbers below: 1. Please join my meeting. https://www1.gotomeeting.com/join/211547328 2. Use your microphone and speakers (VoIP) - a headset is recommended. Or= , call in using your telephone. United States: +1 (224) 649-0001 Argentina (toll-free): 0 800 266 1382 Australia (toll-free): 1 800 193 385 Australia: +61 2 8355 1020 Austria (toll-free): 0 800 202148 Austria: +43 (0) 7 2088 1047 Bahrain (toll-free): 800 81 111 Belarus (toll-free): 8 820 0011 0214 Belgium (toll-free): 0 800 26116 Belgium: +32 (0) 28 08 4368 Brazil (toll-free): 0 800 047 4906 Canada (toll-free): 1 888 455 1389 Canada: +1 (647) 497-9353 China (toll-free): 4008 811084 Czech Republic (toll-free): 800 500448 Denmark (toll-free): 8090 1924 Denmark: +45 (0) 69 91 89 28 Finland (toll-free): 80094507 Finland: +358 (0) 942 59 7850 France (toll-free): 0 805 541 047 France: +33 (0) 170 950 594 Germany (toll-free): 0 800 723 5270 Germany: +49 (0) 811 8899 6903 Hong Kong (toll-free): 30713169 Iceland (toll-free): 800 9869 India (toll-free): 000 800 100 7855 Indonesia (toll-free): 007 803 011 0395 Ireland (toll-free): 1 800 946 538 Ireland: +353 (0) 19 030 010 Israel (toll-free): 1 809 212 875 Italy (toll-free): 800 906959 Italy: +39 0 247 92 13 01 Japan (toll-free): 00 120 663 800 Korea, Republic of (toll-free): 806150880 Luxembourg (toll-free): 800 22104= Malaysia (toll-free): 1 800 81 5373 Mexico (toll-free): 01 800 925 0372 Ne= therlands (toll-free): 0 800 265 8469 Netherlands: +31 (0) 208 080 219 New Zealand (toll-free): 0 800 45 2202 New Zealand: +64 (0) 9 280 6302 Norway (toll-free): 800 30 257 Norway: +47 75 80 32 07 Panama (toll-free): 00 800 226 8832 Peru (toll-free): 0 800 54682 Philippines (toll-free): 1 800 1651 0716 Poland (toll-free): 00 800 1213979= Portugal (toll-free): 800 784 461 Russian Federation (toll-free): 810 800 = 29674011 Saudi Arabia (toll-free): +9668008443633 Singapore (toll-free): 80= 0 120 5615 South Africa (toll-free): 0 800 983 867 Spain (toll-free): 0 800= 900 582 Spain: +34 911 82 9906 Sweden (toll-free): 020 980 772 Sweden: +46 (0) 852 500 186 Switzerland (toll-free): 0 800 740 393 Switzerland: +41 (0) 435 0167 13 Taiwan (toll-free): 00 800 666 854 Thailand (toll-free): 001 800 658 131 Turkey (toll-free): +90800448823683 Ukraine (toll-free): 0 800 50 0641 United Arab Emirates (toll-free): +97180004440439 United Kingdom (toll-free= ): 0 808 168 0229 United Kingdom: +44 (0) 207 151 1853 United States (toll-= free): 1 877 309 2073 Uruguay (toll-free): 000 413 598 4110 Viet Nam (toll-= free): 120 65 157 Access Code: 211-547-328 Audio PIN: Shown after joining the meeting Meeting ID: 211-547-328 From anil@recoil.org Tue May 21 11:16:27 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UejcJ-0006El-5N (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 21 May 2013 11:16:27 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1484242 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:5702 helo=dark.recoil.org) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1UejcI-0001vN-6o (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 21 May 2013 11:16:27 +0100 Received: (qmail 6923 invoked by uid 634); 21 May 2013 10:16:25 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from dhcp-98-47.external.eduroam.ucl.ac.uk (HELO dhcp-98-47.external.eduroam.ucl.ac.uk) (144.82.98.47) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 21 May 2013 11:16:25 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: nice paper from HotOS 99 on distributed systems Message-Id: Date: Tue, 21 May 2013 11:16:13 +0100 To: "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 May 2013 10:16:27 -0000 Content-Length: 159 Lines: 7 Ran across this nice set of thougts from Fox and Brewer on how to build = scalable systems http://lab.mscs.mu.edu/Dist2012/lectures/HarvestYield.pdf -anil= From crowcroft@gmail.com Tue May 21 11:59:04 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UekHY-0000RL-LX (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 21 May 2013 11:59:04 +0100 X-Cam-SpamDetails: score -0.6 from SpamAssassin-3.3.2-1484242 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [74.125.83.53 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (crowcroft[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ee0-f53.google.com ([74.125.83.53]:32930) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UekHX-0007Lb-2Z (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 21 May 2013 11:59:04 +0100 Received: by mail-ee0-f53.google.com with SMTP id c1so292622eek.26 for ; Tue, 21 May 2013 03:59:03 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.14.48.4 with SMTP id u4mr5313666eeb.10.1369133943563; Tue, 21 May 2013 03:59:03 -0700 (PDT) Sender: crowcroft@gmail.com Received: by 10.223.96.200 with HTTP; Tue, 21 May 2013 03:59:03 -0700 (PDT) In-Reply-To: References: Date: Tue, 21 May 2013 11:59:03 +0100 X-Google-Sender-Auth: bXus_tWF51eGe-SkYjBBz_1nAe0 Message-ID: Subject: Re: nice paper from HotOS 99 on distributed systems From: Jon Crowcroft To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=001a11c3fbd85b558204dd3856f3 Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 May 2013 10:59:04 -0000 Content-Length: 2062 Lines: 48 --001a11c3fbd85b558204dd3856f3 Content-Type: text/plain; charset=ISO-8859-1 so CAP is susceptible to our quantum hack by the way - if we keep entangled bits in every replica as part of a version vector, then when the net partitions, we can tell tha records that we have to lock from update (or read, if we want read consistency too) in minority partitions - this would be a variant of the idea of replicating the version vector in more places than the data, so that you can increase availability On Tue, May 21, 2013 at 11:16 AM, Anil Madhavapeddy wrote: > Ran across this nice set of thougts from Fox and Brewer on how to build > scalable systems > > http://lab.mscs.mu.edu/Dist2012/lectures/HarvestYield.pdf > > -anil > --001a11c3fbd85b558204dd3856f3 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
so CAP is susceptible to our quantum hack by the way - if = we keep entangled bits in every replica as part of a version vector, then w= hen the net partitions, we can tell tha records that we have to lock from u= pdate (or read, if we want read consistency too) in minority partitions - t= his would be a variant of the idea of replicating the version vector in mor= e places than the data, so that you can increase availability


On Tue, May 2= 1, 2013 at 11:16 AM, Anil Madhavapeddy <anil@recoil.org> wrote= :
Ran across this nice set of thougts from Fox and Brewer on how to build sca= lable systems

http://lab.mscs.mu.edu/Dist2012/lectures/HarvestYield.pdf<= br>
-anil

--001a11c3fbd85b558204dd3856f3-- From jjl25@cam.ac.uk Tue May 21 16:06:29 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Ueo8z-000572-7q (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 21 May 2013 16:06:29 +0100 X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from firewall.ctxuk.citrix.com ([46.33.159.2]:14117 helo=[10.80.118.245]) by ppsw-32.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.156]:587) with esmtpsa (PLAIN:jjl25) (TLSv1:AES128-SHA:128) id 1Ueo8y-00063Z-2g (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 21 May 2013 16:06:29 +0100 Subject: Mirage weekly call From: Jon Ludlam Content-Type: text/plain; charset=us-ascii X-Mailer: iPad Mail (10B329) Message-Id: <53F918E8-14A0-4E13-95B6-100F75CC994B@cam.ac.uk> Date: Tue, 21 May 2013 16:06:28 +0100 To: "cl-mirage@lists.cam.ac.uk" Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 May 2013 15:06:29 -0000 Content-Length: 135 Lines: 9 Hi all, Daves having trouble with gotomeeting - I'll send another invite now to repl= ace the existing one. Jon Sent from my iPad= From jjl25@cam.ac.uk Tue May 21 16:07:21 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Ueo9p-0005Ck-1e (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 21 May 2013 16:07:21 +0100 X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from firewall.ctxuk.citrix.com ([46.33.159.2]:14117 helo=[10.80.118.245]) by ppsw-32.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.156]:587) with esmtpsa (PLAIN:jjl25) (TLSv1:AES128-SHA:128) id 1Ueo9o-00063Z-33 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 21 May 2013 16:07:21 +0100 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: GoToMeeting Invitation - Mirage Weekly Call Backup From: Jon Ludlam Message-Id: <803358F9-D0B1-4FF8-BF4C-18435CBF6F1C@cam.ac.uk> Date: Tue, 21 May 2013 16:07:21 +0100 To: cl-mirage@lists.cam.ac.uk Mime-Version: 1.0 (1.0) X-Mailer: iPad Mail (10B329) X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 May 2013 15:07:21 -0000 Content-Length: 2694 Lines: 85 1. Please join my meeting, 21 May 2013 at 16:00 BST. https://www1.gotomeeting.com/join/942644576 2. Use your microphone and speakers (VoIP) - a headset is recommended. Or, c= all in using your telephone. United Arab Emirates (toll-free): 800 044 40439 Argentina (toll-free): 0 800 266 1382 Austria: +43 (0) 7 2088 1047 Austria (toll-free): 0 800 202148 Australia: +61 2 8355 1020 Australia (toll-free): 1 800 193 385 Belgium: +32 (0) 28 08 4368 Belgium (toll-free): 0 800 26116 Bahrain (toll-free): 800 81 111 Brazil (toll-free): 0 800 047 4906 Belarus (toll-free): 8 820 0011 0214 Canada: +1 (647) 497-9353 Canada (toll-free): 1 888 455 1389 Switzerland: +41 (0) 435 0167 13 Switzerland (toll-free): 0 800 740 393 China (toll-free): 4008 811084 Czech Republic (toll-free): 800 500448 Germany: +49 (0) 811 8899 6903 Germany (toll-free): 0 800 723 5270 Denmark: +45 (0) 69 91 89 28 Denmark (toll-free): 8090 1924 Spain: +34 911 82 9906 Spain (toll-free): 0 800 900 582 Finland: +358 (0) 942 59 7850 Finland (toll-free): 80094507 France: +33 (0) 170 950 594 France (toll-free): 0 805 541 047 United Kingdom: +44 (0) 207 151 1853 United Kingdom (toll-free): 0 808 168 0229 Hong Kong SAR China (toll-free): 30713169 Indonesia (toll-free): 007 803 011 0395 Ireland: +353 (0) 19 030 010 Ireland (toll-free): 1 800 946 538 Israel (toll-free): 1 809 212 875 India (toll-free): 000 800 100 7855 Iceland (toll-free): 800 9869 Italy: +39 0 247 92 13 01 Italy (toll-free): 800 906959 Japan (toll-free): 0 120 663 800 South Korea (toll-free): 806150880 Luxembourg (toll-free): 800 22104 Mexico (toll-free): 01 800 925 0372 Malaysia (toll-free): 1 800 81 5373 Netherlands: +31 (0) 208 080 219 Netherlands (toll-free): 0 800 265 8469 Norway: +47 75 80 32 07 Norway (toll-free): 800 30 257 New Zealand: +64 (0) 9 280 6302 New Zealand (toll-free): 0 800 45 2202 Panama (toll-free): 00 800 226 8832 Peru (toll-free): 0 800 54682 Philippines (toll-free): 1 800 1651 0716 Poland (toll-free): 00 800 1213979 Portugal (toll-free): 800 784 461 Russia (toll-free): 810 800 29674011 Saudi Arabia (toll-free): 800 844 3633 Sweden: +46 (0) 852 500 186 Sweden (toll-free): 020 980 772 Singapore (toll-free): 800 120 5615 Thailand (toll-free): 001 800 658 131 Turkey (toll-free): 00 800 4488 23683 Taiwan (toll-free): 00 800 666 854 Ukraine (toll-free): 0 800 50 0641 United States: +1 (224) 649-0001 United States (toll-free): 1 877 309 2073 Uruguay (toll-free): 000 413 598 4110 Vietnam (toll-free): 120 65 157 South Africa (toll-free): 0 800 983 867 Access Code: 942-644-576 Audio PIN: Shown after joining the meeting Meeting ID: 942-644-576 GoToMeeting=C2=AE Online Meetings Made Easy=C2=AE Sent from my iPad= From jjl25@cam.ac.uk Tue May 21 19:35:25 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UerPB-0003FW-48 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 21 May 2013 19:35:25 +0100 X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from cpc2-cmbg14-2-0-cust622.5-4.cable.virginmedia.com ([86.26.2.111]:55357 helo=[192.168.1.144]) by ppsw-42.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.159]:587) with esmtpsa (PLAIN:jjl25) (TLSv1:AES128-SHA:128) id 1UerPA-0002f9-9N (Exim 4.80_167-5a66dd3) (return-path ); Tue, 21 May 2013 19:35:24 +0100 References: <5181591B.1070004@luminar.eu.org> <20130520165620.GA22270@damage> Mime-Version: 1.0 (1.0) In-Reply-To: <20130520165620.GA22270@damage> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <7A7AC10A-3DF6-45C3-BBD0-FD0758EC8421@cam.ac.uk> X-Mailer: iPad Mail (10B329) From: Jon Ludlam Subject: Re: https://polarssl.org/ Date: Tue, 21 May 2013 19:35:25 +0100 To: Prashanth Mundkur Cc: Vincent Bernardoff , "cl-mirage@lists.cam.ac.uk" , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 May 2013 18:35:25 -0000 Content-Length: 1101 Lines: 37 Looks very nice :-) The only thing I see is that it requires unix file descriptors whereas matri= xssl doesn't - this might make mirage integration a bit more tricky. Jon Sent from my iPad On 20 May 2013, at 17:56, Prashanth Mundkur wrote= : > Just came across this BSD licensed library >=20 > http://axtls.sourceforge.net/index.htm >=20 >=20 > On 19:07 Wed 01 May, Anil Madhavapeddy wrote: >> On 1 May 2013, at 19:04, Vincent Bernardoff wrote: >>=20 >>> Another embeddded SSL lib, I just found out that mini-os is using it. Mi= ght be worth a look if not already done ? >>>=20 >>=20 >> It's also dual commercial GPLv2 licensed, unfortunately, and so roughly t= he same as MatrixSSL. I've not looked in more depth though, as MatrixSSL ha= s a really nice low-level API that is perfect for our binding needs... >>=20 >> Jon, is your WIP on Github anywhere? I do think it would be easier to ha= ve something on UNIX ahead of the mirage-platform version, if only to run it= through valgrind and bind to Async too... >>=20 >> -anil >>=20 >>=20 >=20 From anil@recoil.org Wed May 22 14:19:55 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uf8xP-00004Z-JY (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 14:19:55 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1484700 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:15663 helo=dark.recoil.org) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1Uf8xP-0001Ti-Cw (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 14:19:55 +0100 Received: (qmail 4795 invoked by uid 634); 22 May 2013 13:19:54 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from volstagg-0.srg.cl.cam.ac.uk (HELO [10.0.1.75]) (128.232.32.232) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 22 May 2013 14:19:54 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: GADT printer patch Message-Id: <9536B24C-D491-4719-92C5-CAEE3BDD636B@recoil.org> Date: Wed, 22 May 2013 14:19:54 +0100 To: "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 13:19:55 -0000 Content-Length: 282 Lines: 10 This is really neat: http://caml.inria.fr/mantis/view.php?id=3D6017 If I understand it right, it eliminates our need for the printf dietlibc = bits, which means we could simplify mirage-platform quite a bit! Should cook up an OPAM compiler patch to experiment with it... -anil= From stedolan@stedolan.net Wed May 22 14:56:03 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uf9WN-0001JG-Eq (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 14:56:03 +0100 X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1484700 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.217.173 listed in list.dnswl.dnsbl.ja.net] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-lb0-f173.google.com ([209.85.217.173]:50910) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1Uf9WM-0001eU-EN (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 14:56:03 +0100 Received: by mail-lb0-f173.google.com with SMTP id t10so2098924lbi.4 for ; Wed, 22 May 2013 06:56:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:sender:x-originating-ip:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :x-gm-message-state; bh=lskrrXeVR/rphDupYpqbFXgkDa0A4YMIfGLKg15dYaM=; b=a0f9rWes5ltGxWUuEtQMzO9eDJ8s8cstGfh9mgz0lfR46b2cqGM0ZsIur0FYVa30ad 6lVE7L90+mk3/MfOdvvP9/s4uFjPZtPyymPjjJcMss67+HYcTZHmH1AdEUQtA1Tr+jHa j0E8yc3eAqsE5vjfxY7vHosomNZBTv8yvvmISLbFaxUR43fygQEXjMBctn+7DIenY6Es 9s0WNZflnQDNSHi8+idWoKmu9NwOe1v+Mrpg34jlcmXLzLhEIKNvjoSwtbY9NRA8eR3m NcgSb/3Oogj3d4h6m5QoWPXgi2pLsm5Ww/DwIhF+xTzoXJ5jW8gFdziThClhBfXSou8O 8Ypg== MIME-Version: 1.0 X-Received: by 10.112.166.101 with SMTP id zf5mr4090307lbb.59.1369230961839; Wed, 22 May 2013 06:56:01 -0700 (PDT) Sender: stedolan@stedolan.net Received: by 10.114.186.41 with HTTP; Wed, 22 May 2013 06:56:01 -0700 (PDT) X-Originating-IP: [128.232.9.157] In-Reply-To: <9536B24C-D491-4719-92C5-CAEE3BDD636B@recoil.org> References: <9536B24C-D491-4719-92C5-CAEE3BDD636B@recoil.org> Date: Wed, 22 May 2013 14:56:01 +0100 X-Google-Sender-Auth: 4EVjSdO_iuvVLNxqvDgzo-XPqL4 Message-ID: Subject: Re: GADT printer patch From: Stephen Dolan To: Anil Madhavapeddy Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQmYZaLz322kSLeQXu3AminPlKD79P9yKxy65oSLbTYtE93J1RICOTnyXtSxshpyvriR8ewc Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 13:56:03 -0000 Content-Length: 562 Lines: 16 It still requires sprintf. In particular, it uses caml_format_float to do %f and friends, which uses sprintf internally. Correct float formatting is a surprisingly large amount of code. Stephen On Wed, May 22, 2013 at 2:19 PM, Anil Madhavapeddy wrote: > This is really neat: > http://caml.inria.fr/mantis/view.php?id=6017 > > If I understand it right, it eliminates our need for the printf dietlibc bits, which means we could simplify mirage-platform quite a bit! > > Should cook up an OPAM compiler patch to experiment with it... > > -anil From scott.dj@gmail.com Wed May 22 15:07:03 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uf9h0-0001df-Vn (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:07:03 +0100 X-Cam-SpamDetails: score 0.6 from SpamAssassin-3.3.2-1484700 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.160.47 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (scott.dj[at]gmail.com) * 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is * CUSTOM_MED * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid * 1.2 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing * list X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-pb0-f47.google.com ([209.85.160.47]:50322) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1Uf9gz-00079F-Eq (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:07:02 +0100 Received: by mail-pb0-f47.google.com with SMTP id rr4so1718982pbb.6 for ; Wed, 22 May 2013 07:07:00 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.66.164.195 with SMTP id ys3mr8485552pab.30.1369231620408; Wed, 22 May 2013 07:07:00 -0700 (PDT) Received: by 10.70.102.176 with HTTP; Wed, 22 May 2013 07:07:00 -0700 (PDT) In-Reply-To: References: <9536B24C-D491-4719-92C5-CAEE3BDD636B@recoil.org> Date: Wed, 22 May 2013 15:07:00 +0100 Message-ID: Subject: Re: GADT printer patch From: David Scott To: Stephen Dolan Content-Type: multipart/alternative; boundary=047d7b6d7c5a598caf04dd4f1445 Cc: "cl-mirage@lists.cam.ac.uk List" , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 14:07:03 -0000 Content-Length: 2449 Lines: 65 --047d7b6d7c5a598caf04dd4f1445 Content-Type: text/plain; charset=ISO-8859-1 Could we wean ourselves off correct float formatting and use something less ... correct? :) On Wed, May 22, 2013 at 2:56 PM, Stephen Dolan wrote: > It still requires sprintf. In particular, it uses caml_format_float to > do %f and friends, which uses sprintf internally. Correct float > formatting is a surprisingly large amount of code. > > Stephen > > On Wed, May 22, 2013 at 2:19 PM, Anil Madhavapeddy > wrote: > > This is really neat: > > http://caml.inria.fr/mantis/view.php?id=6017 > > > > If I understand it right, it eliminates our need for the printf dietlibc > bits, which means we could simplify mirage-platform quite a bit! > > > > Should cook up an OPAM compiler patch to experiment with it... > > > > -anil > > --047d7b6d7c5a598caf04dd4f1445 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Could we wean ourselves off correct float formatting and u= se something less ... correct? :)


On Wed, May 22, 2013 at 2:56 PM, Stephen Dolan <stephen.dolan@cl.cam.ac.uk> wrote:
It still requires sprintf. In particular, it= uses caml_format_float to
do %f and friends, which uses sprintf internally. Correct float
formatting is a surprisingly large amount of code.

Stephen

On Wed, May 22, 2013 at 2:19 PM, Anil Madhavapeddy <anil@recoil.org> wrote:
> This is really neat:
> http://caml.inria.fr/mantis/view.php?id=3D6017
>
> If I understand it right, it eliminates our need for the printf dietli= bc bits, which means we could simplify mirage-platform quite a bit!
>
> Should cook up an OPAM compiler patch to experiment with it...
>
> -anil




<= /div> --047d7b6d7c5a598caf04dd4f1445-- From anil@recoil.org Wed May 22 15:08:16 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uf9iC-0001fV-8X (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:08:16 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1484700 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:1277 helo=dark.recoil.org) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1Uf9iB-0007g3-EX (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:08:16 +0100 Received: (qmail 16124 invoked by uid 634); 22 May 2013 14:08:15 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from volstagg-0.srg.cl.cam.ac.uk (HELO [10.0.1.75]) (128.232.32.232) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 22 May 2013 15:08:15 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: GADT printer patch From: Anil Madhavapeddy In-Reply-To: Date: Wed, 22 May 2013 15:08:14 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <9536B24C-D491-4719-92C5-CAEE3BDD636B@recoil.org> To: Stephen Dolan X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 14:08:16 -0000 Content-Length: 1069 Lines: 38 Right, I want to separate float formatting anyway. Some libraries = (notably FreeBSD's libkern printf) don't do floats at all, so separating those = cleanly in the OCaml layer would be lovely. Currently, the whole format string gets shoved to vsnprintf(3). With = this patch, we just need an implementation of caml_format_float, but not all = the other bits of printf. -anil On 22 May 2013, at 14:56, Stephen Dolan = wrote: > It still requires sprintf. In particular, it uses caml_format_float to > do %f and friends, which uses sprintf internally. Correct float > formatting is a surprisingly large amount of code. >=20 > Stephen >=20 > On Wed, May 22, 2013 at 2:19 PM, Anil Madhavapeddy = wrote: >> This is really neat: >> http://caml.inria.fr/mantis/view.php?id=3D6017 >>=20 >> If I understand it right, it eliminates our need for the printf = dietlibc bits, which means we could simplify mirage-platform quite a = bit! >>=20 >> Should cook up an OPAM compiler patch to experiment with it... >>=20 >> -anil >=20 From stedolan@stedolan.net Wed May 22 15:09:45 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uf9jd-0001ib-3L (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:09:45 +0100 X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1484700 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.215.43 listed in list.dnswl.dnsbl.ja.net] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-la0-f43.google.com ([209.85.215.43]:37316) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1Uf9jb-0008Rl-G4 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:09:45 +0100 Received: by mail-la0-f43.google.com with SMTP id ez20so2006365lab.2 for ; Wed, 22 May 2013 07:09:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:sender:x-originating-ip:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :x-gm-message-state; bh=wagnt/X7nqETM1QyrYd5k8b51BcDilf+RguxANcbqms=; b=Dv1NIg0P18ExS357QM48j5nZRRGBcg0mw4sOPUceo74EZYSqGSMIHp4jh8MHklOEPg WgP8221w3+QT5VgCbK1ikYtGNF5ybbc9rLLHtioCdAUdXbmX6z+Wd9XHKXxNSzg9JxG3 1WeShVnMvJT5p9HfMeP8m/JmBIpZuPFxn6gGjqBOzRvT1wTqivhGSC5H8tmvG44BV8tx ygDci/gJDJVgo+Bt+2NMnAzoc3ulhLHxxxYtu3+grgnvS1t7m38SGcZz5L0uRFz5j9Ya WdbaUNfEgI/B8VlbCSvg9+SyoeCyTLIBhF/beBj1Myi21YMiCu2Ob5bhaG4T1rjtQksU esVQ== MIME-Version: 1.0 X-Received: by 10.112.72.163 with SMTP id e3mr4177440lbv.28.1369231783379; Wed, 22 May 2013 07:09:43 -0700 (PDT) Sender: stedolan@stedolan.net Received: by 10.114.186.41 with HTTP; Wed, 22 May 2013 07:09:43 -0700 (PDT) X-Originating-IP: [128.232.9.157] In-Reply-To: References: <9536B24C-D491-4719-92C5-CAEE3BDD636B@recoil.org> Date: Wed, 22 May 2013 15:09:43 +0100 X-Google-Sender-Auth: QVDUnYFzfL5O4F0L88iR98d30kI Message-ID: Subject: Re: GADT printer patch From: Stephen Dolan To: Anil Madhavapeddy Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQmZd1msj5u/ZFYB2qHcHw8+LV4yzf/frS9bguC1y6uqNLpvfYQ+VcVRk8xb75MQQr8IVKN1 Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 14:09:45 -0000 Content-Length: 1322 Lines: 36 Such an implementation would probably be some variant of http://www.netlib.org/fp/dtoa.c It turns out that formatting floats involves implementing a small bigint library (!). On Wed, May 22, 2013 at 3:08 PM, Anil Madhavapeddy wrote: > Right, I want to separate float formatting anyway. Some libraries (notably > FreeBSD's libkern printf) don't do floats at all, so separating those cleanly > in the OCaml layer would be lovely. > > Currently, the whole format string gets shoved to vsnprintf(3). With this > patch, we just need an implementation of caml_format_float, but not all the > other bits of printf. > > -anil > > On 22 May 2013, at 14:56, Stephen Dolan wrote: > >> It still requires sprintf. In particular, it uses caml_format_float to >> do %f and friends, which uses sprintf internally. Correct float >> formatting is a surprisingly large amount of code. >> >> Stephen >> >> On Wed, May 22, 2013 at 2:19 PM, Anil Madhavapeddy wrote: >>> This is really neat: >>> http://caml.inria.fr/mantis/view.php?id=6017 >>> >>> If I understand it right, it eliminates our need for the printf dietlibc bits, which means we could simplify mirage-platform quite a bit! >>> >>> Should cook up an OPAM compiler patch to experiment with it... >>> >>> -anil >> > From anil@recoil.org Wed May 22 15:12:16 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uf9m4-0001oe-77 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:12:16 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1484700 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:35777 helo=dark.recoil.org) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1Uf9m3-0001MH-Ef (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:12:16 +0100 Received: (qmail 12803 invoked by uid 634); 22 May 2013 14:12:15 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from volstagg-0.srg.cl.cam.ac.uk (HELO [10.0.1.75]) (128.232.32.232) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 22 May 2013 15:12:14 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: GADT printer patch From: Anil Madhavapeddy In-Reply-To: Date: Wed, 22 May 2013 15:12:14 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <1B9493DF-A4E1-47D8-97D5-28809CEC1F19@recoil.org> References: <9536B24C-D491-4719-92C5-CAEE3BDD636B@recoil.org> To: Stephen Dolan X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 14:12:16 -0000 Content-Length: 1783 Lines: 62 I wonder if we should explicitly ban %f in Mirage code and see how far = we get. We rarely need to render anything other than very limited precision = floats, and it's information that is only *required* in the Gc interface from = the standard library (unfortunately). -anil On 22 May 2013, at 15:09, Stephen Dolan = wrote: > Such an implementation would probably be some variant of > http://www.netlib.org/fp/dtoa.c > It turns out that formatting floats involves implementing a small > bigint library (!). >=20 > On Wed, May 22, 2013 at 3:08 PM, Anil Madhavapeddy = wrote: >> Right, I want to separate float formatting anyway. Some libraries = (notably >> FreeBSD's libkern printf) don't do floats at all, so separating those = cleanly >> in the OCaml layer would be lovely. >>=20 >> Currently, the whole format string gets shoved to vsnprintf(3). With = this >> patch, we just need an implementation of caml_format_float, but not = all the >> other bits of printf. >>=20 >> -anil >>=20 >> On 22 May 2013, at 14:56, Stephen Dolan = wrote: >>=20 >>> It still requires sprintf. In particular, it uses caml_format_float = to >>> do %f and friends, which uses sprintf internally. Correct float >>> formatting is a surprisingly large amount of code. >>>=20 >>> Stephen >>>=20 >>> On Wed, May 22, 2013 at 2:19 PM, Anil Madhavapeddy = wrote: >>>> This is really neat: >>>> http://caml.inria.fr/mantis/view.php?id=3D6017 >>>>=20 >>>> If I understand it right, it eliminates our need for the printf = dietlibc bits, which means we could simplify mirage-platform quite a = bit! >>>>=20 >>>> Should cook up an OPAM compiler patch to experiment with it... >>>>=20 >>>> -anil >>>=20 >>=20 >=20 From stedolan@stedolan.net Wed May 22 15:22:35 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uf9w3-0002G0-Bx (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:22:35 +0100 X-Cam-SpamDetails: score -0.7 from SpamAssassin-3.3.2-1484700 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.223.182 listed in list.dnswl.dnsbl.ja.net] X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-ie0-f182.google.com ([209.85.223.182]:59750) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1Uf9w2-0005cO-9Q (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:22:35 +0100 Received: by mail-ie0-f182.google.com with SMTP id a14so5231580iee.41 for ; Wed, 22 May 2013 07:22:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:sender:x-originating-ip:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :x-gm-message-state; bh=QSQDAjQ62a5jHsVApfGk6iu/3/TFZuFeN7DofJLTQ30=; b=L9IZqAwbaG5bhnfK2pIpv0lq0w3iZwbdTl6yRDiyIgERkn5JscgZc3lZ3BPtJSOyBT fFwsfYfWQTnPLDkTQynnLSDKcmOQ6nxawGZ7hzOEAxawEYYsD8jRbSIFItZMsYrLw6X1 T9uEBX0Tl1pSCZ2HZMISIwyuxUHTVQKfjgWHGJKATOVDY5KgM0Yn/20Ga1gVByQZ+LMD Gjb0igppWJnmxUK6uNtqLYBifSpRozvDhtG3BD08UFX3O0246WKvZVJK5pgZt53G0Jld swGml3WwgQyT4IO+Tg5VjygLTjSGqOb8JcX89VVqRGtncei5k0st7U8tj4PTILs3tBmb pVWA== MIME-Version: 1.0 X-Received: by 10.42.41.210 with SMTP id q18mr6086838ice.13.1369232554133; Wed, 22 May 2013 07:22:34 -0700 (PDT) Sender: stedolan@stedolan.net Received: by 10.43.15.194 with HTTP; Wed, 22 May 2013 07:22:33 -0700 (PDT) X-Originating-IP: [128.232.9.157] In-Reply-To: <1B9493DF-A4E1-47D8-97D5-28809CEC1F19@recoil.org> References: <9536B24C-D491-4719-92C5-CAEE3BDD636B@recoil.org> <1B9493DF-A4E1-47D8-97D5-28809CEC1F19@recoil.org> Date: Wed, 22 May 2013 15:22:33 +0100 X-Google-Sender-Auth: hFm6KAs7dCzUs_AS4k6mC-50RPY Message-ID: Subject: Re: GADT printer patch From: Stephen Dolan To: Anil Madhavapeddy Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQk5VtxSge8qsgPGP/CoCAPtdBbU0bGNPDjFgfolh6dW7nxJffqMohej4Jb0L5nE3xQj0SJL Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 14:22:35 -0000 Content-Length: 297 Lines: 7 > We rarely need to render anything other than very limited precision floats, > and it's information that is only *required* in the Gc interface from the > standard library (unfortunately). It is a little strange that those use floats. They should be int64, but this would be a breaking change. From daniel.buenzli@erratique.ch Wed May 22 15:25:54 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uf9zG-0002MF-EI (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:25:54 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1484700 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail6.webfaction.com ([74.55.86.74]:33917 helo=smtp.webfaction.com) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1Uf9zF-0007RT-F1 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:25:54 +0100 Received: from [10.80.118.32] (firewall.ctxuk.citrix.com [46.33.159.2]) by smtp.webfaction.com (Postfix) with ESMTP id 3C31920F2CF9; Wed, 22 May 2013 14:25:52 +0000 (UTC) Date: Wed, 22 May 2013 15:25:51 +0100 From: =?utf-8?Q?Daniel_B=C3=BCnzli?= To: Anil Madhavapeddy Message-ID: <9B5E2044D232471581B422BC881A81A2@erratique.ch> In-Reply-To: <1B9493DF-A4E1-47D8-97D5-28809CEC1F19@recoil.org> References: <9536B24C-D491-4719-92C5-CAEE3BDD636B@recoil.org> <1B9493DF-A4E1-47D8-97D5-28809CEC1F19@recoil.org> Subject: Re: GADT printer patch X-Mailer: sparrow 1.6.4 (build 1178) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Stephen Dolan , "=?utf-8?Q?cl-mirage=40lists.cam.ac.uk_List?=" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 14:25:54 -0000 Content-Length: 375 Lines: 16 Depends on why you need your "%f" but if that may be of some interest Gg has a (slow) lossless float pretty printer that doesn't use "%f": https://github.com/dbuenzli/gg/blob/master/src/gg.ml#L186 The actual string representation is documented here: http://erratique.ch/software/gg/doc/Gg.Float.html#printers But it's not really for human consumption. Best, Daniel From anil@recoil.org Wed May 22 15:27:57 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UfA1F-0002Pk-02 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:27:57 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1484700 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:46780 helo=dark.recoil.org) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with smtp id 1UfA1E-0008D8-En (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 15:27:56 +0100 Received: (qmail 20938 invoked by uid 634); 22 May 2013 14:27:56 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from volstagg-0.srg.cl.cam.ac.uk (HELO [10.0.1.75]) (128.232.32.232) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 22 May 2013 15:27:56 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: GADT printer patch From: Anil Madhavapeddy In-Reply-To: Date: Wed, 22 May 2013 15:27:56 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <9536B24C-D491-4719-92C5-CAEE3BDD636B@recoil.org> <1B9493DF-A4E1-47D8-97D5-28809CEC1F19@recoil.org> To: Stephen Dolan X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 14:27:57 -0000 Content-Length: 637 Lines: 21 On 22 May 2013, at 15:22, Stephen Dolan = wrote: >> We rarely need to render anything other than very limited precision = floats, >> and it's information that is only *required* in the Gc interface from = the >> standard library (unfortunately). >=20 > It is a little strange that those use floats. They should be int64, > but this would be a breaking change. Yeah. Primarily historical, but rather entrenched now. We're going to = change those in the Mirage OPAM switch to be Int64. It requires small tweaks = to the runtime also, which Gabor did last summer when porting Mirage to = kFreeBSD. -anil= From anil@recoil.org Wed May 22 21:44:20 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UfFtU-0002gM-1q (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 21:44:20 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1484700 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:23657 helo=dark.recoil.org) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UfFtT-0001kR-hL (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Wed, 22 May 2013 21:44:19 +0100 Received: (qmail 15153 invoked by uid 634); 22 May 2013 20:44:19 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.84]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Wed, 22 May 2013 21:44:18 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: ANN: mirage-platform, uri, cow Message-Id: Date: Wed, 22 May 2013 21:44:18 +0100 To: "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 20:44:20 -0000 Content-Length: 1556 Lines: 36 All on OPAM! Note that Mirage trunk was broken due to missing = constraints; I've fixed up the problems with older version too, but = please let the list know if you spot any more problems. mirage-platform-0.9.0 (22-May-2013): = https://github.com/mirage/mirage-platform * [xen] Add a simple 'resume hook' mechanism for xen devices * [xen] Update to new shared ring signature in 0.4.0 * [xen] Allow device providers to be added later * [xen] Move the blkif implementation to ocaml-xen-block-driver * [xen] Io_page.string_blit has now an interface similar to String.blit * [xen] Console.write_s has been renamed Console.write_all * [xen] Gnttab.unmap_exn now throws an exception on failure, like the xenctrl version. This makes it possible to run the same blkback code in kernelspace and userspace. cow-0.5.4 (21-May-2013): https://github.com/mirage/ocaml-cow * Add `json_of` and `of_json` type-conv providers (#6). * Add `Json.to_string_hum` which adds more newlines for human-readable = output (#7). * Document the `-cow-no-open` camlp4 flag (#9). * Be compatible with Core by using `List.concat` instead of = `List.flatten` (#2). * Fix Atom feed by adding a `link` field. * Install `.mli` files with the library binaries (#11). * New dependency on the `Uri` library. uri-1.3.8 (2013-05-19): https://github.com/mirage/ocaml-uri * Add `Uri.get_query_param` which selects a single value for a query = key. * Add `Uri.get_query_param'` which returns a list of values associated = with a query key. * Fix ocamldoc in `Uri` module to have a header. From anil@recoil.org Fri May 24 13:46:36 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UfrOG-0000oB-BJ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 24 May 2013 13:46:36 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485615 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:45857 helo=dark.recoil.org) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UfrOB-0002c7-1f (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 24 May 2013 13:46:36 +0100 Received: (qmail 9531 invoked by uid 634); 24 May 2013 12:46:31 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from volstagg-0.srg.cl.cam.ac.uk (HELO [10.0.1.75]) (128.232.32.232) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 24 May 2013 13:46:30 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: [Caml-list] French study on security and functional languages From: Anil Madhavapeddy In-Reply-To: <519F5FE5.5010504@ssi.gouv.fr> Date: Fri, 24 May 2013 13:46:30 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <912F36ED-B157-4DD6-BC68-FFCF99D05E7B@recoil.org> References: <519F5FE5.5010504@ssi.gouv.fr> To: Olivier Levillain X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "caml-list@inria.fr List" , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 May 2013 12:46:36 -0000 Content-Length: 1207 Lines: 33 On 24 May 2013, at 13:41, Olivier Levillain = wrote: > Hi everyone, >=20 >> For those reading French, ANSSI (French agency for information >> security) published a study on security and functional languages, = with >> a set of recommendations. OCaml is apparently well studied: >> = http://www.ssi.gouv.fr/fr/anssi/publications/publications-scientifiques/au= tres-publications/lafosec-securite-et-langages-fonctionnels.html >=20 > For information, some of the results have been presented last February > during the JFLA (Journ=E9es francophones des langages applicatifs). = The > slides presented are available on the conference web site > (http://jfla.inria.fr/2013/programme.html). >=20 =09 I was very glad to see the release of the Parsifal code onto Github too: https://github.com/ANSSI-FR/parsifal It looks like you have done a lot of the work required towards building a pure OCaml SSL and Kerberos stack, as well as DNS and SSH parsers in there too. We were just discussing the lack of a pure OCaml SSL library for MirageOS (which already has a full reimplementation of device = drivers and TCP/IP and HTTP, and is just missing the final SSL piece). best, Anil= From anil@recoil.org Fri May 24 14:02:21 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UfrdV-0001Jw-S0 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 24 May 2013 14:02:21 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485615 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:34452 helo=dark.recoil.org) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UfrdV-0001NU-0q (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 24 May 2013 14:02:21 +0100 Received: (qmail 14468 invoked by uid 634); 24 May 2013 13:02:20 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from volstagg-0.srg.cl.cam.ac.uk (HELO [10.0.1.75]) (128.232.32.232) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Fri, 24 May 2013 14:02:20 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: ANN: cohttp-0.9.8 Message-Id: <3C443DA1-55AA-4B90-9580-6D63C2C66CBD@recoil.org> Date: Fri, 24 May 2013 14:02:20 +0100 To: "cl-mirage@lists.cam.ac.uk List" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 May 2013 13:02:21 -0000 Content-Length: 832 Lines: 17 I've done a substantial rewrite of the internals of Cohttp to be much = cleaner, which results in breaking changes to older code. Thanks to the = magic of OPAM [1], the public package constraints are all correct (with = an upper bound of 0.9.6 for older ones). However, private forks will be = broken until they either fix their interfaces or add an upper bound. cohttp-0.9.8 (2013-05-24): * Lwt interface change: Rewrite Lwt backends to share code, and remove = duplicate function calls from Uri. * Depend on `Uri` 1.3.8+ as it exposes the parameter query functions now = removed from `Request`. * Do not depend on Cstruct in core library, as only Mirage needs it. * Remove `Cohttp_async.body` type alias and just use `string = Pipe.Reader.t` for more explicit types. [1] https://github.com/OCamlPro/opam-repository/pull/740= From vb@luminar.eu.org Fri May 24 17:26:34 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Ufup8-0007Vc-2B (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 24 May 2013 17:26:34 +0100 X-Cam-SpamDetails: score -1.1 from SpamAssassin-3.3.2-1485615 * -1.1 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:57709) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1Ufup6-0003XX-iT (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 24 May 2013 17:26:34 +0100 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id A6459BF679 for ; Fri, 24 May 2013 18:26:02 +0200 (CEST) Message-ID: <519F94B8.6000903@luminar.eu.org> Date: Fri, 24 May 2013 17:26:32 +0100 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130514 Thunderbird/17.0.6 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: New unix backend for mirage Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 May 2013 16:26:34 -0000 Content-Length: 2508 Lines: 63 Hi all, I have been working recently on refactoring the UNIX backend for Mirage. Mainly, the modification are that now the code that handles tun/tap creation has been extracted in a new library ocaml-tuntap (You car install it with OPAM, or git://github.com/vbmithr/ocaml-tuntap). I have refactored mirage-platform and mirage-net such that now, mirage-platform do not create tap interfaces directly in OS.Netif, but instead, there is an OS.Netif.add_vif function that takes a fd as parameter and that adds it as available interface for the unikernel. Now, mirari generates a Backend module that contains the code to create a tap interface, and this module calls the OS.Netif.add_vif (via the manager). This Backend module is actually the equivalent of XenStore for the UNIX backend, it implements a server that listens on a UNIX domain socket for commands from mirari. For now only ADD_VIF is implemented, but PAUSE, RESUME, SUSPEND, etc.. will be implemented in the future if there is a need for it. So basically, my big interrogation for now is "Why does this works perfectly on Linux and not on MacOS, given that the only difference is the tuntap creation code (but that is working, cleanly separated in ocaml-tuntap). If you want to test, try the following: Use mirage-platform, mirage-net and mirari from my repo (git://github.com/vbmithr/~), branch new-unix-backend2. (Typically, you opam pin these packages before installing). Then git clone git://github.com/mirage/mirage-skeleton cd mirage-skeleton/basic, then mirari configure --unix mirari build --unix sudo mirari run --unix This should typically "work" under UNIX: [vb@haramix ~/code/mirage-skeleton/basic]% sudo mirari run --unix [mirari] Using scanned config file hello.conf [mirari] Make the unikernel create the tap0 interface. Manager: create Manager: init done [mirari] Connecting to /tmp/mir-6224.sock... [mirari] Transmitted message ok. [backend]: Connection from mirari. [backend]: received command VIF_ADD [backend]: Creating tap0... plugging into tap0 with mac 92:f0:ab:83:1a:f6.. Netif: plug tap0 Manager: plug tap0 Manager: plug done, to listener Manager: Interface tap0 to DHCP DHCP: waiting for first offer DHCP: start discovery Sending DHCP broadcast len 552 But not work under MacOS: stuck at [backend]: Creating tap0... Thanks to anybody that can have a look at that (preferably people with macs). Thanks in advance, Vincent From vb@luminar.eu.org Fri May 24 19:23:45 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UfweX-0002ZU-Ca (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 24 May 2013 19:23:45 +0100 X-Cam-SpamDetails: score -1.1 from SpamAssassin-3.3.2-1485615 * -1.1 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:45550) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UfweX-0001h9-6r (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Fri, 24 May 2013 19:23:45 +0100 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id 9F2AABF679 for ; Fri, 24 May 2013 20:23:14 +0200 (CEST) Message-ID: <519FB030.80707@luminar.eu.org> Date: Fri, 24 May 2013 19:23:44 +0100 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130514 Thunderbird/17.0.6 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Current state of the UNIX backend Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 May 2013 18:23:45 -0000 Content-Length: 410 Lines: 20 Hi all, Apparently, the current state of mirage for UNIX-direct is broken for me. Could you test it for me, Mac users: cd mirage-skeleton/basic mirari configure --unix mirari build --unix sudo ./mir-hello On my test machine it fails with: Fatal error: exception Unix.Unix_error(3, "set_nonblock", "") Thanks in advance, I really think I have broken something with this tuntap extraction story. Vincent From anil@recoil.org Sat May 25 11:11:31 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UgBRj-0000Qj-RC (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 25 May 2013 11:11:31 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485974 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:44339 helo=dark.recoil.org) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UgBRj-0004F0-1f (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 25 May 2013 11:11:31 +0100 Received: (qmail 23798 invoked by uid 634); 25 May 2013 10:11:31 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from host86-148-227-23.range86-148.btcentralplus.com (HELO [10.10.42.129]) (86.148.227.23) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sat, 25 May 2013 11:11:30 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: one-byte TCP writes wedging Date: Sat, 25 May 2013 11:11:31 +0100 Message-Id: <27BA97B7-F2CC-414E-AEB4-268FBC5A62A7@recoil.org> To: Balraj Singh Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 May 2013 10:11:31 -0000 Content-Length: 826 Lines: 18 Balraj noticed that a stream of 1-byte writes on a TCP connection would = cause netfront/netback to wedge. This is obviously quite unrealistic, = but a good regression test. A quick chat with Steven Smith pointed out that some Linux netbacks had = a limit on the number of fragments allowed (based on the skbuff chain = limit size). So you might be ending up with a situation where the = backend drops the entire set of fragments, and the frontend is = retransmitting all the time. So if you modify our frontend to limit the fragment size to ~10 or so = for any given packet, that might help. On the other hand, if you're = doing writes with a TCP segment size of 1, but still only 3-4 fragments = (for the Ethernet/IP/TCP headers), then we have some other bug. What = does the Netif request look like, Balraj? -anil= From anil@recoil.org Sat May 25 12:54:32 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UgD3Q-0001Rz-4v (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 25 May 2013 12:54:32 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485974 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:16956 helo=dark.recoil.org) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UgD3P-0000bC-1l (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 25 May 2013 12:54:32 +0100 Received: (qmail 12830 invoked by uid 634); 25 May 2013 11:54:31 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from host86-148-227-23.range86-148.btcentralplus.com (HELO [10.10.42.129]) (86.148.227.23) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sat, 25 May 2013 12:54:30 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Current state of the UNIX backend From: Anil Madhavapeddy In-Reply-To: <519FB030.80707@luminar.eu.org> Date: Sat, 25 May 2013 12:54:31 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <2CE22A3B-FED0-42D7-BE72-81E283910846@recoil.org> References: <519FB030.80707@luminar.eu.org> To: Vincent Bernardoff X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 May 2013 11:54:32 -0000 Content-Length: 2377 Lines: 68 On 24 May 2013, at 19:23, Vincent Bernardoff wrote: > Hi all, >=20 > Apparently, the current state of mirage for UNIX-direct is broken for = me. >=20 > Could you test it for me, Mac users: >=20 > cd mirage-skeleton/basic > mirari configure --unix > mirari build --unix > sudo ./mir-hello >=20 One very quick thing I noticed is that the 4.01.0dev+mirage-unix = compiler is a bit out of date, as it used the old ocplib-endian branch = back when the function inlining was added by Pierre. I've modified the compiler descriptions to use ocaml-4.01.0-trunk now, = which has those changes and a lot more. However, it does point to the fact that we can separate the compiler = dependency from the package dependencies much more easily now, thanks to = OPAM.=20 We have these compilers: - 4.00.1 : stable release. - 4.01.0dev+trunk : latest trunk, currently being released. - pierre's new inline branch [1] OPAM can build custom package sets against a compiler without = reinstalling it, by doing "opam switch 4.00.1+custom --alias-of=3D4.00.1".= This is a fast clone, and then a package set can be installed in there = just like a normal compiler switch. So, we don't actually need 4.00.1+mirage-unix/xen as a compiler switch = any more. I propose simply adding these packages to mainline OPAM: - mirage-platform-unix (conficts: mirage-platform-xen) - mirage-platform-xen (conflicts: mirage-platform-unix) - mirage-net-socket (conflicts: mirage-net-direct) - mirage-net-direct (conflicts: mirage-net-socket) The mirage-platform-* packages can provide a mirage ocamlfind package as = normal. The only thing I think we need that is missing is a virtual = OPAM package that depends on either variant, so that upstream packages = like mirage-www can just depend on "mirage-platform" instead of a = specific variant. Thomas, I need your opinion on if this is sensible or not. It would = eliminate the need for the MIRAGE_NET/OS environment variables, and push = the remaining platform selection logic into OPAM, so that Mirari can = just install the right set of packages depending on what users want. [1] described in = http://www.ocamlpro.com/blog/2013/05/24/optimisations-you-shouldn-t-do.htm= l > On my test machine it fails with: >=20 > Fatal error: exception Unix.Unix_error(3, "set_nonblock", "") I'm recompiling now to try this out! -anil= From anil@recoil.org Sat May 25 14:16:12 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UgEKS-0002CV-99 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 25 May 2013 14:16:12 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485974 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:27685 helo=dark.recoil.org) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UgEKR-0000Tu-33 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 25 May 2013 14:16:12 +0100 Received: (qmail 8968 invoked by uid 634); 25 May 2013 13:16:11 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from host86-148-227-23.range86-148.btcentralplus.com (HELO [10.10.42.129]) (86.148.227.23) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sat, 25 May 2013 14:16:11 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Current state of the UNIX backend From: Anil Madhavapeddy In-Reply-To: <519FB030.80707@luminar.eu.org> Date: Sat, 25 May 2013 14:16:09 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <5E15C7C4-A19F-495E-BF65-EE876B248795@recoil.org> References: <519FB030.80707@luminar.eu.org> To: Vincent Bernardoff X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Jeremy Yallop , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 May 2013 13:16:12 -0000 Content-Length: 1921 Lines: 61 On 24 May 2013, at 19:23, Vincent Bernardoff wrote: > Hi all, >=20 > Apparently, the current state of mirage for UNIX-direct is broken for = me. >=20 > Could you test it for me, Mac users: >=20 > cd mirage-skeleton/basic > mirari configure --unix > mirari build --unix > sudo ./mir-hello >=20 > On my test machine it fails with: >=20 > Fatal error: exception Unix.Unix_error(3, "set_nonblock", "") I added a test_nonblock.ml to ocaml-tuntap to build a small repro case. $ sudo dtruss ./_build/test/test_nonblock.native 3321/0x46aed: open("/dev/tap0\0", 0x2, 0x4) =3D 5 0 3321/0x46aed: fcntl(0xB, 0x3, 0x0) =3D -1 Err#9 Notice that the fd we opened was '5' and the fd actually passed to fcntl = is 0xB. Which is a bit off, and looking at the tuntap_stubs.c shows: diff --git a/lib/tuntap_stubs.c b/lib/tuntap_stubs.c index cb5e11d..703d477 100644 --- a/lib/tuntap_stubs.c +++ b/lib/tuntap_stubs.c @@ -121,7 +121,7 @@ tun_alloc(char *dev, int kind, int pi, int persist, = int user, int group) caml_failwith("Unable to open the TUN or TAP interface"); } =20 - return Val_int(fd); + return fd; } =20 CAMLprim value Your sendmsg was likely also failing due to this problem -- if you get = an EBADF error, it normally means the file descriptor integer is somehow = wrong. I'll send you a pull request with the fixes and tests... Btw, Jeremy Yallop will be showing off a new type-safe FFI library he's = been building at the next OCaml Labs meeting on Friday, which will make = bugs like this a thing of the past... The UNIX backend isn't quite working yet on MacOS X after this. I get: Manager: plug tap0 [netif-input] error : Unix.Unix_error(Unix.EIO, "read", "") [netif-input] error : Unix.Unix_error(Unix.EIO, "read", "") [netif-input] error : Unix.Unix_error(Unix.EIO, "read", "") ...which probably means that the interface isn't up yet. -anil= From vb@luminar.eu.org Sat May 25 20:03:54 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UgJkw-0005Q3-Ho (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 25 May 2013 20:03:54 +0100 X-Cam-SpamDetails: score -1.1 from SpamAssassin-3.3.2-1485974 * -1.1 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:42535) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with esmtp id 1UgJkv-0004Aj-15 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 25 May 2013 20:03:54 +0100 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id B6DC7BFDC1 for ; Sat, 25 May 2013 21:03:19 +0200 (CEST) Message-ID: <51A10B18.80305@luminar.eu.org> Date: Sat, 25 May 2013 20:03:52 +0100 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130514 Thunderbird/17.0.6 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Current state of the UNIX backend References: <519FB030.80707@luminar.eu.org> <5E15C7C4-A19F-495E-BF65-EE876B248795@recoil.org> In-Reply-To: <5E15C7C4-A19F-495E-BF65-EE876B248795@recoil.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 May 2013 19:03:54 -0000 Content-Length: 1562 Lines: 54 On 25/05/2013 14:16, Anil Madhavapeddy wrote: > On 24 May 2013, at 19:23, Vincent Bernardoff wrote: > >> Hi all, >> >> Apparently, the current state of mirage for UNIX-direct is broken for me. >> >> Could you test it for me, Mac users: >> >> cd mirage-skeleton/basic >> mirari configure --unix >> mirari build --unix >> sudo ./mir-hello >> So, thanks to Anil we now have a fixed version of tuntap for Mac (for some reason, the code was already working on Linux). The next problem I have is with the new backend on Mac (please test for those who can): * Use my repo (git://github.com/vbmithr/, mirage-platform, mirage-net and mirari pinned to branch new-unix-backend2). Pin those packages and install them. * go to mirage-skeleton/basic * mirari configure --unix * Edit the generated file backend.ml to add a Tuntap.set_ipv4… line in order to assign an IPv4 on the host side for the unikernel (mandatory on mac). * mirari run --unix You will notice that the unikernel will block (Just after Manager: plug tap0). Debugging with printf, I was able to spot that the problem seems to be that in mirage-net/ Ethif.create, the listen function never returns, and the code blocks here. It happens only on Mac, so I am a bit confused here. The usage of dtruss did not help me much :( So Anil or anybody else, if you could have a look at that and understand the problem it would be awesome, and it would enable me to merge the new unix backend, that makes IP configuration on the interface much more flexible. Cheers, Vincent From balraj885@gmail.com Sat May 25 22:25:25 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UgLxt-0006LH-U5 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 25 May 2013 22:25:25 +0100 X-Cam-SpamDetails: score -0.3 from SpamAssassin-3.3.2-1485974 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.128.42 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (balraj885[at]gmail.com) * 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in * digit (balraj885[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-qe0-f42.google.com ([209.85.128.42]:39302) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UgLxt-0002Mj-hI (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sat, 25 May 2013 22:25:25 +0100 Received: by mail-qe0-f42.google.com with SMTP id cz11so3238395qeb.1 for ; Sat, 25 May 2013 14:25:24 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.49.94.231 with SMTP id df7mr25021974qeb.33.1369517124715; Sat, 25 May 2013 14:25:24 -0700 (PDT) Sender: balraj885@gmail.com Received: by 10.49.76.10 with HTTP; Sat, 25 May 2013 14:25:24 -0700 (PDT) In-Reply-To: <27BA97B7-F2CC-414E-AEB4-268FBC5A62A7@recoil.org> References: <27BA97B7-F2CC-414E-AEB4-268FBC5A62A7@recoil.org> Date: Sat, 25 May 2013 22:25:24 +0100 X-Google-Sender-Auth: bQTDHE_hnpBTJmpT1DX_r--aJuk Message-ID: Subject: Re: one-byte TCP writes wedging From: Balraj Singh To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=047d7b6d9c92bb892104dd918d0c Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 May 2013 21:25:26 -0000 Content-Length: 4764 Lines: 94 --047d7b6d9c92bb892104dd918d0c Content-Type: text/plain; charset=ISO-8859-1 In the particular test I am using I write 36 bytes of payload and use the Mirage equivalent of TCP_NODELAY. This works for a bit but then suffers some packet loss (why? TBD) and triggers a rexmit. The retransmitted packet is 1400+ bytes and is made up of a long chain of 36 byte io_pages. I thought that it may be that the ring did not have enough slots to take all the chunks of the pkt. Making the retransmitted pkt be the size of the original write improved it very significantly but it would still fail in the same way, tho less frequently. I'm working on it - I see available txring slots vary, but I havent yet found a case where the slots are fully depleted or down to fewer than chunks that need to be written. I'm still narrowing it down. This test originally was with 1-byte writes, but that seemed to wedge even before the 1st data packet made it to the wire. This may be because of the limitation Steven mentioned. I think I'm getting close on the 36 byte write test, once this is figured out I'll try it with 1 byte writes again. Balraj On Sat, May 25, 2013 at 11:11 AM, Anil Madhavapeddy wrote: > Balraj noticed that a stream of 1-byte writes on a TCP connection would > cause netfront/netback to wedge. This is obviously quite unrealistic, but > a good regression test. > > A quick chat with Steven Smith pointed out that some Linux netbacks had a > limit on the number of fragments allowed (based on the skbuff chain limit > size). So you might be ending up with a situation where the backend drops > the entire set of fragments, and the frontend is retransmitting all the > time. > > So if you modify our frontend to limit the fragment size to ~10 or so for > any given packet, that might help. On the other hand, if you're doing > writes with a TCP segment size of 1, but still only 3-4 fragments (for the > Ethernet/IP/TCP headers), then we have some other bug. What does the Netif > request look like, Balraj? > > -anil > --047d7b6d9c92bb892104dd918d0c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
In the particular test I am using I write 36 by= tes of payload and use the Mirage equivalent of TCP_NODELAY. =A0This works = for a bit but then suffers some packet loss (why? TBD) and triggers a rexmi= t. =A0The retransmitted packet is 1400+ bytes and is made up of a long chai= n of 36 byte io_pages. =A0I thought that it may be that the ring did not ha= ve enough slots to take all the chunks of the pkt. =A0Making the retransmit= ted pkt be the size of the original write improved it very significantly bu= t it would still=A0fail in the same way, tho less frequently. =A0I'm wo= rking on it - I see available txring slots vary, but I havent yet found a c= ase where the slots are fully depleted or down to fewer than chunks that ne= ed to be written. =A0I'm still narrowing it down.

This test originally was with 1-byte writes= , but that seemed to wedge even before the 1st data packet made it to the w= ire. =A0This may be because of the limitation Steven mentioned. =A0I think = I'm getting close on the 36 byte write test, once this is figured out I= 'll try it with 1 byte writes again.

Balraj



On Sat, May 25, 2013 at= 11:11 AM, Anil Madhavapeddy <anil@recoil.org> wrote:
Balraj noticed that a stream of 1-byte write= s on a TCP connection would cause netfront/netback to wedge. =A0This is obv= iously quite unrealistic, but a good regression test.

A quick chat with Steven Smith pointed out that some Linux netbacks had a l= imit on the number of fragments allowed (based on the skbuff chain limit si= ze). =A0So you might be ending up with a situation where the backend drops = the entire set of fragments, and the frontend is retransmitting all the tim= e.

So if you modify our frontend to limit the fragment size to ~10 or so for a= ny given packet, that might help. =A0On the other hand, if you're doing= writes with a TCP segment size of 1, but still only 3-4 fragments (for the= Ethernet/IP/TCP headers), then we have some other bug. =A0What does the Ne= tif request look like, Balraj?

-anil

--047d7b6d9c92bb892104dd918d0c-- From anil@recoil.org Sun May 26 10:51:17 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UgXbh-0005zj-PS (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 26 May 2013 10:51:17 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485974 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:20669 helo=dark.recoil.org) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UgXbg-0004vQ-hn (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 26 May 2013 10:51:17 +0100 Received: (qmail 22352 invoked by uid 634); 26 May 2013 09:51:16 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from host81-149-102-120.in-addr.btopenworld.com (HELO clink-4.home) (81.149.102.120) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sun, 26 May 2013 10:51:15 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Current state of the UNIX backend From: Anil Madhavapeddy In-Reply-To: <51A10B18.80305@luminar.eu.org> Date: Sun, 26 May 2013 10:51:10 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <6DCA3150-9B2A-4C0F-8A11-FD6119F2B73C@recoil.org> References: <519FB030.80707@luminar.eu.org> <5E15C7C4-A19F-495E-BF65-EE876B248795@recoil.org> <51A10B18.80305@luminar.eu.org> To: Vincent Bernardoff X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 May 2013 09:51:17 -0000 Content-Length: 2340 Lines: 68 On 25 May 2013, at 20:03, Vincent Bernardoff wrote: > On 25/05/2013 14:16, Anil Madhavapeddy wrote: >> On 24 May 2013, at 19:23, Vincent Bernardoff = wrote: >>=20 >>> Hi all, >>>=20 >>> Apparently, the current state of mirage for UNIX-direct is broken = for me. >>>=20 >>> Could you test it for me, Mac users: >>>=20 >>> cd mirage-skeleton/basic >>> mirari configure --unix >>> mirari build --unix >>> sudo ./mir-hello >>>=20 >=20 >=20 > So, thanks to Anil we now have a fixed version of tuntap for Mac (for = some reason, the code was already working on Linux). It's because the Val_int fix was only in the __APPLE__ ifdef, so your = other Linux code was just fine. > The next problem I have is with the new backend on Mac (please test = for those who can): >=20 > * Use my repo (git://github.com/vbmithr/, mirage-platform, mirage-net = and mirari pinned to branch new-unix-backend2). Pin those packages and = install them. >=20 > * go to mirage-skeleton/basic > * mirari configure --unix > * Edit the generated file backend.ml to add a Tuntap.set_ipv4=85 line = in order to assign an IPv4 on the host side for the unikernel (mandatory = on mac). > * mirari run --unix So a much easier way to debug this sort of issue is to build a unit test = that expresses the desired behaviour first. I put a small packet dumper = for tuntap in: https://github.com/avsm/ocaml-tuntap/blob/master/test/nonblock_read.ml You can build this without any other dependencies by: $ make $ ocamlbuild _build/test/nonblock_read.native $ sudo ./_build/test/nonblock_read.native and if you ping the new interface, you should see hexdump packets on the = console. I do see these on both MacOS X and Linux successfully. If = Mort does too, I'll be convinced it actually works :-) And now the question is why the Mirage libraries are different. I'm = guessing it's to do with setting the non-blocking flag, which is done = via a stub at the moment. If you make the behaviour of Ethif match that = of my test case, then it should spot the problem. PS: I also noticed that the netmask on the tap0 when you ifconfig is = "0xff000000", which isn't the same as the "0xffffff00" I was trying to = set it to. It didn't matter for this test, but should be fixed too or = else risk network routing heisenbugs... -anil= From anil@recoil.org Sun May 26 10:53:31 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UgXdr-00061E-5X (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 26 May 2013 10:53:31 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485974 * 0.0 HTML_MESSAGE BODY: HTML included in message X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:27759 helo=dark.recoil.org) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with smtp id 1UgXdq-0005Gu-hz (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Sun, 26 May 2013 10:53:31 +0100 Received: (qmail 7831 invoked by uid 634); 26 May 2013 09:53:30 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED,HTML_MESSAGE X-Spam-Check-By: dark.recoil.org Received: from host81-149-102-120.in-addr.btopenworld.com (HELO clink-4.home) (81.149.102.120) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Sun, 26 May 2013 10:53:29 +0100 Content-Type: multipart/alternative; boundary="Apple-Mail=_0316E164-DA5E-4F4C-A24C-1052DDC9C305" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: one-byte TCP writes wedging From: Anil Madhavapeddy In-Reply-To: Date: Sun, 26 May 2013 10:53:29 +0100 Message-Id: References: <27BA97B7-F2CC-414E-AEB4-268FBC5A62A7@recoil.org> To: Balraj Singh X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 May 2013 09:53:31 -0000 Content-Length: 6724 Lines: 135 --Apple-Mail=_0316E164-DA5E-4F4C-A24C-1052DDC9C305 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 The long chain of 36-byte frags is consistent with the backend dropping = it. Does it work better if you restrict the total fragment chains size = to just 10 or 11? The first unexplained packet loss is a real alarm bell though. The = entire TCP retransmit code on our stack is just a canary that spots = latent bugs elsewhere in the device stack :-) -anil On 25 May 2013, at 22:25, Balraj Singh = wrote: > In the particular test I am using I write 36 bytes of payload and use = the Mirage equivalent of TCP_NODELAY. This works for a bit but then = suffers some packet loss (why? TBD) and triggers a rexmit. The = retransmitted packet is 1400+ bytes and is made up of a long chain of 36 = byte io_pages. I thought that it may be that the ring did not have = enough slots to take all the chunks of the pkt. Making the = retransmitted pkt be the size of the original write improved it very = significantly but it would still fail in the same way, tho less = frequently. I'm working on it - I see available txring slots vary, but = I havent yet found a case where the slots are fully depleted or down to = fewer than chunks that need to be written. I'm still narrowing it down. >=20 > This test originally was with 1-byte writes, but that seemed to wedge = even before the 1st data packet made it to the wire. This may be = because of the limitation Steven mentioned. I think I'm getting close = on the 36 byte write test, once this is figured out I'll try it with 1 = byte writes again. >=20 > Balraj >=20 >=20 >=20 > On Sat, May 25, 2013 at 11:11 AM, Anil Madhavapeddy = wrote: > Balraj noticed that a stream of 1-byte writes on a TCP connection = would cause netfront/netback to wedge. This is obviously quite = unrealistic, but a good regression test. >=20 > A quick chat with Steven Smith pointed out that some Linux netbacks = had a limit on the number of fragments allowed (based on the skbuff = chain limit size). So you might be ending up with a situation where the = backend drops the entire set of fragments, and the frontend is = retransmitting all the time. >=20 > So if you modify our frontend to limit the fragment size to ~10 or so = for any given packet, that might help. On the other hand, if you're = doing writes with a TCP segment size of 1, but still only 3-4 fragments = (for the Ethernet/IP/TCP headers), then we have some other bug. What = does the Netif request look like, Balraj? >=20 > -anil >=20 --Apple-Mail=_0316E164-DA5E-4F4C-A24C-1052DDC9C305 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 The long chain of 36-byte frags is consistent with the = backend dropping it.  Does it work better if you restrict the total = fragment chains size to just 10 or 11?

The first unexplained packet loss is a = real alarm bell though.  The entire TCP retransmit code on our = stack is just a canary that spots latent bugs elsewhere in the device = stack :-)

-anil

On 25 May 2013, at 22:25, Balraj Singh <balraj.singh@cl.cam.ac.uk>= ; wrote:

In = the particular test I am using I write 36 bytes of payload and use the = Mirage equivalent of TCP_NODELAY.  This works for a bit but then = suffers some packet loss (why? TBD) and triggers a rexmit.  The = retransmitted packet is 1400+ bytes and is made up of a long chain of 36 = byte io_pages.  I thought that it may be that the ring did not have = enough slots to take all the chunks of the pkt.  Making the = retransmitted pkt be the size of the original write improved it very = significantly but it would still fail in the same way, tho less = frequently.  I'm working on it - I see available txring slots vary, = but I havent yet found a case where the slots are fully depleted or down = to fewer than chunks that need to be written.  I'm still narrowing = it down.

This test originally was with 1-byte writes, but that seemed = to wedge even before the 1st data packet made it to the wire.  This = may be because of the limitation Steven mentioned.  I think I'm = getting close on the 36 byte write test, once this is figured out I'll = try it with 1 byte writes again.

Balraj



On Sat, May 25, 2013 at 11:11 AM, = Anil Madhavapeddy <anil@recoil.org> wrote:
Balraj noticed that a = stream of 1-byte writes on a TCP connection would cause netfront/netback = to wedge.  This is obviously quite unrealistic, but a good = regression test.

A quick chat with Steven Smith pointed out that some Linux netbacks had = a limit on the number of fragments allowed (based on the skbuff chain = limit size).  So you might be ending up with a situation where the = backend drops the entire set of fragments, and the frontend is = retransmitting all the time.

So if you modify our frontend to limit the fragment size to ~10 or so = for any given packet, that might help.  On the other hand, if = you're doing writes with a TCP segment size of 1, but still only 3-4 = fragments (for the Ethernet/IP/TCP headers), then we have some other = bug.  What does the Netif request look like, Balraj?

-anil


= --Apple-Mail=_0316E164-DA5E-4F4C-A24C-1052DDC9C305-- From vb@vb.fdn.fr Mon May 27 12:30:34 2013 Received: from ppsw-33.csi.cam.ac.uk ([131.111.8.133]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UgvdK-0002Ds-M5 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 12:30:34 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485974 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:35238) by ppsw-33.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) with esmtp id 1UgvdI-000204-fy (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 12:30:34 +0100 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id C05DBBF7C3 for ; Mon, 27 May 2013 13:29:52 +0200 (CEST) Message-ID: <51A343D7.7000408@vb.fdn.fr> Date: Mon, 27 May 2013 12:30:31 +0100 From: "Vincent B." User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130403 Thunderbird/17.0.5 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Current state of the UNIX backend References: <519FB030.80707@luminar.eu.org> <5E15C7C4-A19F-495E-BF65-EE876B248795@recoil.org> <51A10B18.80305@luminar.eu.org> <6DCA3150-9B2A-4C0F-8A11-FD6119F2B73C@recoil.org> In-Reply-To: <6DCA3150-9B2A-4C0F-8A11-FD6119F2B73C@recoil.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 May 2013 11:30:34 -0000 Content-Length: 1528 Lines: 27 On 26/05/2013 10:51, Anil Madhavapeddy wrote: > So a much easier way to debug this sort of issue is to build a unit test that expresses the desired behaviour first. I put a small packet dumper for tuntap in: > > https://github.com/avsm/ocaml-tuntap/blob/master/test/nonblock_read.ml > > You can build this without any other dependencies by: > > $ make > $ ocamlbuild _build/test/nonblock_read.native > $ sudo ./_build/test/nonblock_read.native The second command does not work for me, I’m going to try anyway. Note that in your test you always use opentun instead of opentap, normally this would create a level3 interface instead of a lever2 one. > and if you ping the new interface, you should see hexdump packets on the console. I do see these on both MacOS X and Linux successfully. If Mort does too, I'll be convinced it actually works :-) > > And now the question is why the Mirage libraries are different. I'm guessing it's to do with setting the non-blocking flag, which is done via a stub at the moment. If you make the behaviour of Ethif match that of my test case, then it should spot the problem. > > PS: I also noticed that the netmask on the tap0 when you ifconfig is "0xff000000", which isn't the same as the "0xffffff00" I was trying to set it to. It didn't matter for this test, but should be fixed too or else risk network routing heisenbugs... I’m going to investigate more. Thanks for the tip, I’m going to try to isolate more problems instead of trying to fix them in the code, indeed. Vincent From vb@vb.fdn.fr Mon May 27 12:46:00 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UgvsG-0002T2-T6 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 12:46:00 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485974 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:48676) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UgvsG-0004IV-DB (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 12:46:00 +0100 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id 06F26BF66F for ; Mon, 27 May 2013 13:45:20 +0200 (CEST) Message-ID: <51A34777.5000100@vb.fdn.fr> Date: Mon, 27 May 2013 12:45:59 +0100 From: "Vincent B." User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130403 Thunderbird/17.0.5 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: Re: Current state of the UNIX backend References: <519FB030.80707@luminar.eu.org> <5E15C7C4-A19F-495E-BF65-EE876B248795@recoil.org> <51A10B18.80305@luminar.eu.org> <6DCA3150-9B2A-4C0F-8A11-FD6119F2B73C@recoil.org> <51A343D7.7000408@vb.fdn.fr> In-Reply-To: <51A343D7.7000408@vb.fdn.fr> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 May 2013 11:46:01 -0000 Content-Length: 1903 Lines: 57 On 27/05/2013 12:30, Vincent B. wrote: > On 26/05/2013 10:51, Anil Madhavapeddy wrote: >> So a much easier way to debug this sort of issue is to build a unit >> test that expresses the desired behaviour first. I put a small packet >> dumper for tuntap in: >> >> https://github.com/avsm/ocaml-tuntap/blob/master/test/nonblock_read.ml >> >> You can build this without any other dependencies by: >> >> $ make >> $ ocamlbuild _build/test/nonblock_read.native >> $ sudo ./_build/test/nonblock_read.native > > The second command does not work for me, I’m going to try anyway. > Note that in your test you always use opentun instead of opentap, > normally this would create a level3 interface instead of a lever2 one. The command was $ ocamlbuild test/nonblock_read.native of course. Apart from that, >> and if you ping the new interface, you should see hexdump packets on >> the console. I do see these on both MacOS X and Linux successfully. >> If Mort does too, I'll be convinced it actually works :-) I don't see anything in the console, strangely. >> And now the question is why the Mirage libraries are different. I'm >> guessing it's to do with setting the non-blocking flag, which is done >> via a stub at the moment. If you make the behaviour of Ethif match >> that of my test case, then it should spot the problem. >> >> PS: I also noticed that the netmask on the tap0 when you ifconfig is >> "0xff000000", which isn't the same as the "0xffffff00" I was trying to >> set it to. It didn't matter for this test, but should be fixed too or >> else risk network routing heisenbugs... Strangely again, for me the mask is set correctly (Linux): 5: tap0: mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500 link/none inet 10.11.12.1/24 scope global tap0 I can ping it correctly, but I don't see anything in the console. Cheers, Vincent From anil@recoil.org Mon May 27 13:58:52 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Ugx0m-0004WQ-Sr (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 13:58:52 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485974 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:8906 helo=dark.recoil.org) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1Ugx0k-0001i9-0n (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 13:58:52 +0100 Received: (qmail 23668 invoked by uid 634); 27 May 2013 12:58:50 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from host81-149-102-120.in-addr.btopenworld.com (HELO clink-4.home) (81.149.102.120) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Mon, 27 May 2013 13:58:49 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Current state of the UNIX backend From: Anil Madhavapeddy In-Reply-To: <51A34777.5000100@vb.fdn.fr> Date: Mon, 27 May 2013 13:58:47 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <519FB030.80707@luminar.eu.org> <5E15C7C4-A19F-495E-BF65-EE876B248795@recoil.org> <51A10B18.80305@luminar.eu.org> <6DCA3150-9B2A-4C0F-8A11-FD6119F2B73C@recoil.org> <51A343D7.7000408@vb.fdn.fr> <51A34777.5000100@vb.fdn.fr> To: "Vincent B." X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: cl-mirage@lists.cam.ac.uk X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 May 2013 12:58:53 -0000 Content-Length: 1555 Lines: 46 On 27 May 2013, at 12:45, "Vincent B." wrote: > On 27/05/2013 12:30, Vincent B. wrote: >> On 26/05/2013 10:51, Anil Madhavapeddy wrote: >>> So a much easier way to debug this sort of issue is to build a unit >>> test that expresses the desired behaviour first. I put a small = packet >>> dumper for tuntap in: >>>=20 >>> = https://github.com/avsm/ocaml-tuntap/blob/master/test/nonblock_read.ml >>>=20 >>> You can build this without any other dependencies by: >>>=20 >>> $ make >>> $ ocamlbuild _build/test/nonblock_read.native >>> $ sudo ./_build/test/nonblock_read.native >>=20 >> The second command does not work for me, I=92m going to try anyway. >> Note that in your test you always use opentun instead of opentap, >> normally this would create a level3 interface instead of a lever2 = one. >=20 > The command was > $ ocamlbuild test/nonblock_read.native >=20 > of course. Apart from that, >=20 >>> and if you ping the new interface, you should see hexdump packets on >>> the console. I do see these on both MacOS X and Linux successfully. >>> If Mort does too, I'll be convinced it actually works :-) >=20 > I don't see anything in the console, strangely. Are you actually generating traffic? MacOS X sees multicast traffic by default, but Linux probably doesn't. Make sure you ping it, and tcpdump the tap to ensure that there is traffic on there that it should be seeing at the application, but isn't for some reason. The ktrace/dtruss/strace should also reveal the exact return from read calls, which is important. -anil From anil@recoil.org Mon May 27 14:35:10 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UgxZu-0005Ba-Pz (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 14:35:10 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485974 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:43695 helo=dark.recoil.org) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UgxZu-0002zv-1K (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 14:35:10 +0100 Received: (qmail 12648 invoked by uid 634); 27 May 2013 13:35:10 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from host81-149-102-120.in-addr.btopenworld.com (HELO clink-4.home) (81.149.102.120) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Mon, 27 May 2013 14:35:10 +0100 From: Anil Madhavapeddy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: xmlm input function with blocking Date: Mon, 27 May 2013 14:35:08 +0100 Message-Id: <151B1D02-EF6E-4BB0-BEC6-63C49BE6FA83@recoil.org> To: =?iso-8859-1?Q?Daniel_B=FCnzli?= Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 May 2013 13:35:10 -0000 Content-Length: 670 Lines: 23 Hi Daniel, I was just looking into using xmlm to do incremental parsing of a large = XML document coming in from the network. xmlm lets custom input functions be defined: type source =3D [ | `Channel of Pervasives.in_channel | `Fun of unit -> int | `String of int * string ] =20 However, both Async and Lwt (and presumably Fut) need to signal that Fun = might block if data is temporarily unavailable. Do you have any thoughts on how to support this in xmlm streaming API? = A Would_block exception would be sufficient to signal a temporary = failure, but then there also needs to be a function to restart the Xmlm = parser from within the I/O loop. -anil= From daniel.buenzli@erratique.ch Mon May 27 15:08:59 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Ugy6d-0005uW-SM (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 15:08:59 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485974 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail6.webfaction.com ([74.55.86.74]:60353 helo=smtp.webfaction.com) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1Ugy6c-0000Yz-G6 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 15:08:59 +0100 Received: from [192.168.1.126] (cpc3-cmbg9-0-0-cust324.5-4.cable.virginmedia.com [81.103.21.69]) by smtp.webfaction.com (Postfix) with ESMTP id 5AF782110868; Mon, 27 May 2013 14:08:57 +0000 (UTC) Date: Mon, 27 May 2013 15:08:55 +0100 From: =?utf-8?Q?Daniel_B=C3=BCnzli?= To: Anil Madhavapeddy Message-ID: <4445063DDE30432E8746B09734D83D19@erratique.ch> In-Reply-To: <151B1D02-EF6E-4BB0-BEC6-63C49BE6FA83@recoil.org> References: <151B1D02-EF6E-4BB0-BEC6-63C49BE6FA83@recoil.org> Subject: Re: xmlm input function with blocking X-Mailer: sparrow 1.6.4 (build 1178) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Cc: "=?utf-8?Q?cl-mirage=40lists.cam.ac.uk_List?=" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 May 2013 14:08:59 -0000 Content-Length: 1590 Lines: 44 Le lundi, 27 mai 2013 =C3=A0 14:35, Anil Madhavapeddy a =C3=A9crit : > xmlm lets custom input functions be defined: This interface is useless crap, it doesn't solve any problem. =20 Xmlm as it stands has a streaming but *blocking* interface and you cannot= do anything about it. =20 > However, both Async and Lwt (and presumably =46ut) need to signal that = =46un might block if data is temporarily unavailable. Yes. =20 =20 > Do you have any thoughts on how to support this in xmlm streaming API=3F= A Would=5Fblock exception would be sufficient to signal a temporary fail= ure, but then there also needs to be a function to restart the Xmlm parse= r from within the I/O loop. Sure, that's exactly what non-blocking codecs solve =5B1=5D. If you use a= =60Manual source you provide the buffer to read from and the computation= suspends and asks for more if it cannot proceed (Xmlm.input returns =60A= wait). =20 On my side, at the moment, only uutf =5B2=5D and jsonm =5B3=5D support th= at (and Vg's =22stored=22 -- file formats -- renderer). This is the reaso= n why I eventually need to make an incompatible Xmlm 2.0.0 rewritten on t= op of uutf. A simple example showing usage of =60Manual sources in uutf with Unix.rea= d can be found here =5B4=5D (the second function, lines=5Ffd). It should = convince you that this is what you want. Best, Daniel =5B1=5D https://github.com/dbuenzli/nbcodec/blob/master/RATIONALE =5B2=5D http://erratique.ch/software/uutf =5B3=5D http://erratique.ch/software/jsonm =5B4=5D http://erratique.ch/software/uutf/doc/Uutf=23readlines =20 From anil@recoil.org Mon May 27 15:30:13 2013 Received: from ppsw-32.csi.cam.ac.uk ([131.111.8.132]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UgyRB-0006Cf-SJ (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 15:30:13 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485974 X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:45624 helo=dark.recoil.org) by ppsw-32.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.146]:25) with smtp id 1UgyRB-0002yW-1e (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 15:30:13 +0100 Received: (qmail 2543 invoked by uid 634); 27 May 2013 14:30:13 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.48]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Mon, 27 May 2013 15:30:13 +0100 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: xmlm input function with blocking From: Anil Madhavapeddy In-Reply-To: <4445063DDE30432E8746B09734D83D19@erratique.ch> Date: Mon, 27 May 2013 15:30:11 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <151B1D02-EF6E-4BB0-BEC6-63C49BE6FA83@recoil.org> <4445063DDE30432E8746B09734D83D19@erratique.ch> To: =?iso-8859-1?Q?Daniel_B=FCnzli?= X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 May 2013 14:30:13 -0000 Content-Length: 1214 Lines: 30 On 27 May 2013, at 15:08, Daniel B=FCnzli = wrote: >=20 >> Do you have any thoughts on how to support this in xmlm streaming = API? A Would_block exception would be sufficient to signal a temporary = failure, but then there also needs to be a function to restart the Xmlm = parser from within the I/O loop. >=20 > Sure, that's exactly what non-blocking codecs solve [1]. If you use a = `Manual source you provide the buffer to read from and the computation = suspends and asks for more if it cannot proceed (Xmlm.input returns = `Await). =20 >=20 > On my side, at the moment, only uutf [2] and jsonm [3] support that = (and Vg's "stored" -- file formats -- renderer). This is the reason why = I eventually need to make an incompatible Xmlm 2.0.0 rewritten on top of = uutf. >=20 > A simple example showing usage of `Manual sources in uutf with = Unix.read can be found here [4] (the second function, lines_fd). It = should convince you that this is what you want. Got it. I'm happy to go with jsonm actually. I just need an example to = illustrate interfacing parsing long running connections with Async = Pipes, and JSON works just as well as XML for that purpose. cheers, Anil From balraj885@gmail.com Mon May 27 23:33:24 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Uh5ym-0004g2-N2 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 23:33:24 +0100 X-Cam-SpamDetails: score -0.3 from SpamAssassin-3.3.2-1485974 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.216.43 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (balraj885[at]gmail.com) * 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in * digit (balraj885[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-qa0-f43.google.com ([209.85.216.43]:37388) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1Uh5ym-0002Z2-6Y (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Mon, 27 May 2013 23:33:24 +0100 Received: by mail-qa0-f43.google.com with SMTP id j11so1190553qag.16 for ; Mon, 27 May 2013 15:33:23 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.49.104.114 with SMTP id gd18mr34141477qeb.13.1369694003330; Mon, 27 May 2013 15:33:23 -0700 (PDT) Sender: balraj885@gmail.com Received: by 10.49.29.232 with HTTP; Mon, 27 May 2013 15:33:23 -0700 (PDT) In-Reply-To: References: <27BA97B7-F2CC-414E-AEB4-268FBC5A62A7@recoil.org> Date: Mon, 27 May 2013 23:33:23 +0100 X-Google-Sender-Auth: 0vTGxW1zGoA8Ud5B1xPqBcFA7mY Message-ID: Subject: Re: one-byte TCP writes wedging From: Balraj Singh To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=047d7b6782fc8507ea04ddbabc1b Cc: "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 May 2013 22:33:24 -0000 Content-Length: 9497 Lines: 183 --047d7b6782fc8507ea04ddbabc1b Content-Type: text/plain; charset=ISO-8859-1 It turned out that both the suspected problems were real problems and the interference betw the two was confusing the debugging. It looks like the max skbuff frags is 18 (65536/page_size + 2) and indeed if the chain of packet fragmets is longer than 19 (the logic probably allows for one extra) it locks up the ring permanently. The other problem was that the ring does indeed get depleted down to the point where the available slots are fewer than the number needed for the current chain of frags. Unfortunately in this case the write is still permitted which overwrites/corrupts freely and things immediately or pretty soon thereafter go kaplooey. To confirm that there is nothing else, I implemented a quick workaround - the chain of frags is never allowed to be longer than 19 and if there aren't enough free slots then the whole chain is dropped. With these two changes all tests always completed and completed correctly. However, just dropping when not enough slots causes excessive pkt loss so slows things randomly and a lot - it should either block or the write should fail with an ENOBUFS flavoured exception. The good news though is that it still works and a lot of the other tricky machinery also works correctly. Balraj On Sun, May 26, 2013 at 10:53 AM, Anil Madhavapeddy wrote: > The long chain of 36-byte frags is consistent with the backend dropping > it. Does it work better if you restrict the total fragment chains size to > just 10 or 11? > > The first unexplained packet loss is a real alarm bell though. The entire > TCP retransmit code on our stack is just a canary that spots latent bugs > elsewhere in the device stack :-) > > -anil > > On 25 May 2013, at 22:25, Balraj Singh wrote: > > In the particular test I am using I write 36 bytes of payload and use the > Mirage equivalent of TCP_NODELAY. This works for a bit but then suffers > some packet loss (why? TBD) and triggers a rexmit. The retransmitted > packet is 1400+ bytes and is made up of a long chain of 36 byte io_pages. > I thought that it may be that the ring did not have enough slots to take > all the chunks of the pkt. Making the retransmitted pkt be the size of the > original write improved it very significantly but it would still fail in > the same way, tho less frequently. I'm working on it - I see available > txring slots vary, but I havent yet found a case where the slots are fully > depleted or down to fewer than chunks that need to be written. I'm still > narrowing it down. > > This test originally was with 1-byte writes, but that seemed to wedge even > before the 1st data packet made it to the wire. This may be because of the > limitation Steven mentioned. I think I'm getting close on the 36 byte > write test, once this is figured out I'll try it with 1 byte writes again. > > Balraj > > > > On Sat, May 25, 2013 at 11:11 AM, Anil Madhavapeddy wrote: > >> Balraj noticed that a stream of 1-byte writes on a TCP connection would >> cause netfront/netback to wedge. This is obviously quite unrealistic, but >> a good regression test. >> >> A quick chat with Steven Smith pointed out that some Linux netbacks had a >> limit on the number of fragments allowed (based on the skbuff chain limit >> size). So you might be ending up with a situation where the backend drops >> the entire set of fragments, and the frontend is retransmitting all the >> time. >> >> So if you modify our frontend to limit the fragment size to ~10 or so for >> any given packet, that might help. On the other hand, if you're doing >> writes with a TCP segment size of 1, but still only 3-4 fragments (for the >> Ethernet/IP/TCP headers), then we have some other bug. What does the Netif >> request look like, Balraj? >> >> -anil >> > > > --047d7b6782fc8507ea04ddbabc1b Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
It turned out that both the suspected problems were real p= roblems and the interference betw the two was confusing the debugging. =A0I= t looks like the max skbuff frags is 18 (65536/page_size + 2) and = indeed if the chain of packet fragmets is longer than 19 (the logic probabl= y allows for one extra) it locks up the ring permanently. =A0The other prob= lem was that the ring does indeed get depleted down to the point where the = available slots are fewer than the number needed for the current chain of f= rags. =A0Unfortunately in this case the write is still permitted which over= writes/corrupts freely and things immediately or pretty soon thereafter go = kaplooey. =A0To confirm that there is nothing else, I implemented a quick w= orkaround - the chain of frags is never allowed to be longer than 19 and if= there aren't enough free slots then the whole chain is dropped. =A0Wit= h these two changes all tests always completed and completed correctly. =A0= However, just dropping when not enough slots causes excessive pkt loss so s= lows things randomly and a lot - it should either block or the write should= fail with an ENOBUFS flavoured exception. =A0The good news though is that = it still works and a lot of the other tricky machinery also works correctly= .

Balraj



<= div class=3D"gmail_quote">On Sun, May 26, 2013 at 10:53 AM, Anil Madhavaped= dy <anil@recoil.org> wrote:
The long= chain of 36-byte frags is consistent with the backend dropping it. =A0Does= it work better if you restrict the total fragment chains size to just 10 o= r 11?

The first unexplained packet loss is a real alarm bell thoug= h. =A0The entire TCP retransmit code on our stack is just a canary that spo= ts latent bugs elsewhere in the device stack :-)

-anil

On 25 May 2013, at 22:25, Balraj Singh <balraj.singh@cl.cam.ac.uk> wro= te:

In the particular test = I am using I write 36 bytes of payload and use the Mirage equivalent of TCP= _NODELAY. =A0This works for a bit but then suffers some packet loss (why? T= BD) and triggers a rexmit. =A0The retransmitted packet is 1400+ bytes and i= s made up of a long chain of 36 byte io_pages. =A0I thought that it may be = that the ring did not have enough slots to take all the chunks of the pkt. = =A0Making the retransmitted pkt be the size of the original write improved = it very significantly but it would still=A0fail in the same way, tho less f= requently. =A0I'm working on it - I see available txring slots vary, bu= t I havent yet found a case where the slots are fully depleted or down to f= ewer than chunks that need to be written. =A0I'm still narrowing it dow= n.

This test originally was with 1-byte writes, but that s= eemed to wedge even before the 1st data packet made it to the wire. =A0This= may be because of the limitation Steven mentioned. =A0I think I'm gett= ing close on the 36 byte write test, once this is figured out I'll try = it with 1 byte writes again.

Balraj



On Sat, May 25, 2013 at 11:11 AM, A= nil Madhavapeddy <anil@recoil.org> wrote:
Balraj noticed that a stream of 1-byte write= s on a TCP connection would cause netfront/netback to wedge. =A0This is obv= iously quite unrealistic, but a good regression test.

A quick chat with Steven Smith pointed out that some Linux netbacks had a l= imit on the number of fragments allowed (based on the skbuff chain limit si= ze). =A0So you might be ending up with a situation where the backend drops = the entire set of fragments, and the frontend is retransmitting all the tim= e.

So if you modify our frontend to limit the fragment size to ~10 or so for a= ny given packet, that might help. =A0On the other hand, if you're doing= writes with a TCP segment size of 1, but still only 3-4 fragments (for the= Ethernet/IP/TCP headers), then we have some other bug. =A0What does the Ne= tif request look like, Balraj?

-anil



--047d7b6782fc8507ea04ddbabc1b-- From Dave.Scott@eu.citrix.com Tue May 28 08:46:49 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UhEcL-0007cQ-Gg (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 28 May 2013 08:46:49 +0100 X-Cam-SpamDetails: score -1.1 from SpamAssassin-3.3.2-1485974 * -1.1 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain * -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no * trust * [46.33.159.39 listed in list.dnswl.dnsbl.ja.net] * 0.0 HTML_MESSAGE BODY: HTML included in message X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from smtp.eu.citrix.com ([46.33.159.39]:4969) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UhEcK-0007Lp-8U (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 28 May 2013 08:46:49 +0100 X-IronPort-AV: E=Sophos;i="4.87,756,1363132800"; d="scan'208,217";a="5031325" Received: from lonpex01cl01.citrite.net ([10.30.203.101]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/AES128-SHA; 28 May 2013 07:46:49 +0000 Received: from LONPEX01CL03.citrite.net ([169.254.3.241]) by LONPEX01CL01.citrite.net ([169.254.1.104]) with mapi id 14.02.0342.003; Tue, 28 May 2013 08:46:47 +0100 From: Dave Scott To: Balraj Singh Subject: Re: one-byte TCP writes wedging Thread-Topic: one-byte TCP writes wedging Thread-Index: AQHOWTA9JMZ2sTnG1UKHlBhrKDV4K5kWWZgAgADRA4CAAmalgIAAq2IP Date: Tue, 28 May 2013 07:46:47 +0000 Message-ID: References: <27BA97B7-F2CC-414E-AEB4-268FBC5A62A7@recoil.org> , In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_F36DF6154D264AC28B22B824FF674B0Eeucitrixcom_" MIME-Version: 1.0 Cc: "cl-mirage@lists.cam.ac.uk List" , Anil Madhavapeddy X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 May 2013 07:46:49 -0000 Content-Length: 11659 Lines: 281 --_000_F36DF6154D264AC28B22B824FF674B0Eeucitrixcom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Balraj, Very interesting discoveries! Regarding the skbuff frag limit, should this be considered as part of the p= rotocol even though it was originally a Linux implementation issue leaking = through? Do you know if it has been stable over time? It might be worth ask= ing on xen-devel. Regarding there not being enough slots: there's already a "wait for a free = slot" mechanism so we could add a "wait for n free slots". Do you have para= llel threads transmitting at once? We should probably take care that a "wai= t for n" doesn't end up constantly getting gazumped by lots of "wait for 1"= s Cheers, -- Dave Scott On May 27, 2013, at 11:33 PM, "Balraj Singh" > wrote: It turned out that both the suspected problems were real problems and the i= nterference betw the two was confusing the debugging. It looks like the ma= x skbuff frags is 18 (65536/page_size + 2) and indeed if the chain of packe= t fragmets is longer than 19 (the logic probably allows for one extra) it l= ocks up the ring permanently. The other problem was that the ring does ind= eed get depleted down to the point where the available slots are fewer than= the number needed for the current chain of frags. Unfortunately in this c= ase the write is still permitted which overwrites/corrupts freely and thing= s immediately or pretty soon thereafter go kaplooey. To confirm that there= is nothing else, I implemented a quick workaround - the chain of frags is = never allowed to be longer than 19 and if there aren't enough free slots th= en the whole chain is dropped. With these two changes all tests always com= pleted and completed correctly. However, just dropping when not enough slo= ts causes excessive pkt loss so slows things randomly and a lot - it should= either block or the write should fail with an ENOBUFS flavoured exception.= The good news though is that it still works and a lot of the other tricky= machinery also works correctly. Balraj On Sun, May 26, 2013 at 10:53 AM, Anil Madhavapeddy > wrote: The long chain of 36-byte frags is consistent with the backend dropping it.= Does it work better if you restrict the total fragment chains size to jus= t 10 or 11? The first unexplained packet loss is a real alarm bell though. The entire = TCP retransmit code on our stack is just a canary that spots latent bugs el= sewhere in the device stack :-) -anil On 25 May 2013, at 22:25, Balraj Singh > wrote: In the particular test I am using I write 36 bytes of payload and use the M= irage equivalent of TCP_NODELAY. This works for a bit but then suffers som= e packet loss (why? TBD) and triggers a rexmit. The retransmitted packet i= s 1400+ bytes and is made up of a long chain of 36 byte io_pages. I though= t that it may be that the ring did not have enough slots to take all the ch= unks of the pkt. Making the retransmitted pkt be the size of the original = write improved it very significantly but it would still fail in the same wa= y, tho less frequently. I'm working on it - I see available txring slots v= ary, but I havent yet found a case where the slots are fully depleted or do= wn to fewer than chunks that need to be written. I'm still narrowing it do= wn. This test originally was with 1-byte writes, but that seemed to wedge even = before the 1st data packet made it to the wire. This may be because of the= limitation Steven mentioned. I think I'm getting close on the 36 byte wri= te test, once this is figured out I'll try it with 1 byte writes again. Balraj On Sat, May 25, 2013 at 11:11 AM, Anil Madhavapeddy > wrote: Balraj noticed that a stream of 1-byte writes on a TCP connection would cau= se netfront/netback to wedge. This is obviously quite unrealistic, but a g= ood regression test. A quick chat with Steven Smith pointed out that some Linux netbacks had a l= imit on the number of fragments allowed (based on the skbuff chain limit si= ze). So you might be ending up with a situation where the backend drops th= e entire set of fragments, and the frontend is retransmitting all the time. So if you modify our frontend to limit the fragment size to ~10 or so for a= ny given packet, that might help. On the other hand, if you're doing write= s with a TCP segment size of 1, but still only 3-4 fragments (for the Ether= net/IP/TCP headers), then we have some other bug. What does the Netif requ= est look like, Balraj? -anil --_000_F36DF6154D264AC28B22B824FF674B0Eeucitrixcom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
Hi Balraj,

Very interesting discoveries!

Regarding the skbuff frag limit, should this be considered as part of = the protocol even though it was originally a Linux implementation issue lea= king through? Do you know if it has been stable over time? It might be wort= h asking on xen-devel.

Regarding there not being enough slots: there's already a "wait f= or a free slot" mechanism so we could add a "wait for n free slot= s". Do you have parallel threads transmitting at once? We should proba= bly take care that a "wait for n" doesn't end up constantly getting gazumped by lots of "wait for 1"s

Cheers,

-- 
Dave Scott

On May 27, 2013, at 11:33 PM, "Balraj Singh" <balraj.singh@cl.cam.ac.uk> wrote:

It turned out that both the suspected problems were real p= roblems and the interference betw the two was confusing the debugging. &nbs= p;It looks like the max skbuff frags is 18 (65536/page_size + 2) and indeed if the chain of packet = fragmets is longer than 19 (the logic probably allows for one extra) it loc= ks up the ring permanently.  The other problem was that the ring does indeed get depleted down to the point where the ava= ilable slots are fewer than the number needed for the current chain of frag= s.  Unfortunately in this case the write is still permitted which over= writes/corrupts freely and things immediately or pretty soon thereafter go kaplooey.  To confirm that there is noth= ing else, I implemented a quick workaround - the chain of frags is never al= lowed to be longer than 19 and if there aren't enough free slots then the w= hole chain is dropped.  With these two changes all tests always completed and completed correctly.  However,= just dropping when not enough slots causes excessive pkt loss so slows thi= ngs randomly and a lot - it should either block or the write should fail wi= th an ENOBUFS flavoured exception.  The good news though is that it still works and a lot of the other tricky mach= inery also works correctly.

Balraj



On Sun, May 26, 2013 at 10:53 AM, Anil Madhavape= ddy <anil@recoil.org> wrote:
The long chain of 36-byte frags is cons= istent with the backend dropping it.  Does it work better if you restr= ict the total fragment chains size to just 10 or 11?

The first unexplained packet loss is a real alarm bell though.  T= he entire TCP retransmit code on our stack is just a canary that spots late= nt bugs elsewhere in the device stack :-)

-anil


In the particular test I am using I write 36 bytes of payload and use = the Mirage equivalent of TCP_NODELAY.  This works for a bit but then s= uffers some packet loss (why? TBD) and triggers a rexmit.  The retrans= mitted packet is 1400+ bytes and is made up of a long chain of 36 byte io_pages.  I thought that it may be that t= he ring did not have enough slots to take all the chunks of the pkt.  = Making the retransmitted pkt be the size of the original write improved it = very significantly but it would still fail in the same way, tho less frequently.  I'm working on it - I see avai= lable txring slots vary, but I havent yet found a case where the slots are = fully depleted or down to fewer than chunks that need to be written.  = I'm still narrowing it down.

This test originally was with 1-byte writes, but that seemed to wedge = even before the 1st data packet made it to the wire.  This may be beca= use of the limitation Steven mentioned.  I think I'm getting close on = the 36 byte write test, once this is figured out I'll try it with 1 byte writes again.

Balraj



On Sat, May 25, 2013 at 11:11 AM, Anil Madhavape= ddy <anil@recoil.org> wrote:
Balraj noticed that a stream of 1-byte writes on a TCP connection would cau= se netfront/netback to wedge.  This is obviously quite unrealistic, bu= t a good regression test.

A quick chat with Steven Smith pointed out that some Linux netbacks had a l= imit on the number of fragments allowed (based on the skbuff chain limit si= ze).  So you might be ending up with a situation where the backend dro= ps the entire set of fragments, and the frontend is retransmitting all the time.

So if you modify our frontend to limit the fragment size to ~10 or so for a= ny given packet, that might help.  On the other hand, if you're doing = writes with a TCP segment size of 1, but still only 3-4 fragments (for the = Ethernet/IP/TCP headers), then we have some other bug.  What does the Netif request look like, Balraj?

-anil



--_000_F36DF6154D264AC28B22B824FF674B0Eeucitrixcom_-- From anil@recoil.org Tue May 28 10:15:15 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UhFzv-0005Ja-IA (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 28 May 2013 10:15:15 +0100 X-Cam-SpamDetails: score 0.0 from SpamAssassin-3.3.2-1485974 * 0.0 HTML_MESSAGE BODY: HTML included in message X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from recoil.dh.bytemark.co.uk ([89.16.177.154]:33446 helo=dark.recoil.org) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with smtp id 1UhFzu-0004ax-98 (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 28 May 2013 10:15:15 +0100 Received: (qmail 20218 invoked by uid 634); 28 May 2013 09:15:14 -0000 X-Spam-Level: * X-Spam-Status: No, hits=-1.0 required=5.0 tests=ALL_TRUSTED,HTML_MESSAGE X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginmedia.com (HELO [192.168.1.48]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Tue, 28 May 2013 10:15:13 +0100 Content-Type: multipart/alternative; boundary="Apple-Mail=_34039232-F57F-4FDC-88D4-6F15326D76FF" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: one-byte TCP writes wedging From: Anil Madhavapeddy In-Reply-To: Date: Tue, 28 May 2013 10:15:11 +0100 Message-Id: References: <27BA97B7-F2CC-414E-AEB4-268FBC5A62A7@recoil.org> , To: Dave Scott X-Mailer: Apple Mail (2.1503) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Cc: Balraj Singh , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 May 2013 09:15:15 -0000 Content-Length: 13549 Lines: 323 --Apple-Mail=_34039232-F57F-4FDC-88D4-6F15326D76FF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii I think it's far safer to serialise a single fragment batch on the ring = (in an ordered Lwt_sequence) rather than have fragments interleaved = across multiple packet write requests. Any other path will have packets being transmitted out-of-order, which = will severely mess up performance. This includes the case where there = are multiple outstanding write requests with different numbers of = fragments -- these should be delivered in the order they are = transmitted, and so a large packet could indeed block a series of small = ones. It also makes hardware offload easier if the fragments aren't = scattered over the ring. -anil On 28 May 2013, at 08:46, Dave Scott wrote: > Hi Balraj, >=20 > Very interesting discoveries! >=20 > Regarding the skbuff frag limit, should this be considered as part of = the protocol even though it was originally a Linux implementation issue = leaking through? Do you know if it has been stable over time? It might = be worth asking on xen-devel. >=20 > Regarding there not being enough slots: there's already a "wait for a = free slot" mechanism so we could add a "wait for n free slots". Do you = have parallel threads transmitting at once? We should probably take care = that a "wait for n" doesn't end up constantly getting gazumped by lots = of "wait for 1"s >=20 > Cheers, >=20 > --=20 > Dave Scott >=20 > On May 27, 2013, at 11:33 PM, "Balraj Singh" = wrote: >=20 >> It turned out that both the suspected problems were real problems and = the interference betw the two was confusing the debugging. It looks = like the max skbuff frags is 18 (65536/page_size + 2) and indeed if the = chain of packet fragmets is longer than 19 (the logic probably allows = for one extra) it locks up the ring permanently. The other problem was = that the ring does indeed get depleted down to the point where the = available slots are fewer than the number needed for the current chain = of frags. Unfortunately in this case the write is still permitted which = overwrites/corrupts freely and things immediately or pretty soon = thereafter go kaplooey. To confirm that there is nothing else, I = implemented a quick workaround - the chain of frags is never allowed to = be longer than 19 and if there aren't enough free slots then the whole = chain is dropped. With these two changes all tests always completed and = completed correctly. However, just dropping when not enough slots = causes excessive pkt loss so slows things randomly and a lot - it should = either block or the write should fail with an ENOBUFS flavoured = exception. The good news though is that it still works and a lot of the = other tricky machinery also works correctly. >>=20 >> Balraj >>=20 >>=20 >>=20 >> On Sun, May 26, 2013 at 10:53 AM, Anil Madhavapeddy = wrote: >> The long chain of 36-byte frags is consistent with the backend = dropping it. Does it work better if you restrict the total fragment = chains size to just 10 or 11? >>=20 >> The first unexplained packet loss is a real alarm bell though. The = entire TCP retransmit code on our stack is just a canary that spots = latent bugs elsewhere in the device stack :-) >>=20 >> -anil >>=20 >> On 25 May 2013, at 22:25, Balraj Singh = wrote: >>=20 >>> In the particular test I am using I write 36 bytes of payload and = use the Mirage equivalent of TCP_NODELAY. This works for a bit but then = suffers some packet loss (why? TBD) and triggers a rexmit. The = retransmitted packet is 1400+ bytes and is made up of a long chain of 36 = byte io_pages. I thought that it may be that the ring did not have = enough slots to take all the chunks of the pkt. Making the = retransmitted pkt be the size of the original write improved it very = significantly but it would still fail in the same way, tho less = frequently. I'm working on it - I see available txring slots vary, but = I havent yet found a case where the slots are fully depleted or down to = fewer than chunks that need to be written. I'm still narrowing it down. >>>=20 >>> This test originally was with 1-byte writes, but that seemed to = wedge even before the 1st data packet made it to the wire. This may be = because of the limitation Steven mentioned. I think I'm getting close = on the 36 byte write test, once this is figured out I'll try it with 1 = byte writes again. >>>=20 >>> Balraj >>>=20 >>>=20 >>>=20 >>> On Sat, May 25, 2013 at 11:11 AM, Anil Madhavapeddy = wrote: >>> Balraj noticed that a stream of 1-byte writes on a TCP connection = would cause netfront/netback to wedge. This is obviously quite = unrealistic, but a good regression test. >>>=20 >>> A quick chat with Steven Smith pointed out that some Linux netbacks = had a limit on the number of fragments allowed (based on the skbuff = chain limit size). So you might be ending up with a situation where the = backend drops the entire set of fragments, and the frontend is = retransmitting all the time. >>>=20 >>> So if you modify our frontend to limit the fragment size to ~10 or = so for any given packet, that might help. On the other hand, if you're = doing writes with a TCP segment size of 1, but still only 3-4 fragments = (for the Ethernet/IP/TCP headers), then we have some other bug. What = does the Netif request look like, Balraj? >>>=20 >>> -anil >>>=20 >>=20 >>=20 --Apple-Mail=_34039232-F57F-4FDC-88D4-6F15326D76FF Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii I = think it's far safer to serialise a single fragment batch on the ring = (in an ordered Lwt_sequence) rather than have fragments interleaved = across multiple packet write requests.

Any other path = will have packets being transmitted out-of-order, which will severely = mess up performance.  This includes the case where there are = multiple outstanding write requests with different numbers of fragments = -- these should be delivered in the order they are transmitted, and so a = large packet could indeed block a series of small ones.  It also = makes hardware offload easier if the fragments aren't scattered over the = ring.

-anil


Hi Balraj,

Very interesting discoveries!

Regarding the skbuff frag limit, should this be considered as part = of the protocol even though it was originally a Linux implementation = issue leaking through? Do you know if it has been stable over time? It = might be worth asking on xen-devel.

Regarding there not being enough slots: there's already a "wait for = a free slot" mechanism so we could add a "wait for n free slots". Do you = have parallel threads transmitting at once? We should probably take care = that a "wait for n" doesn't end up constantly getting gazumped by lots of "wait for 1"s

Cheers,

-- 
Dave Scott

On May 27, 2013, at 11:33 PM, "Balraj Singh" <balraj.singh@cl.cam.ac.uk>= ; wrote:

It turned out that both the suspected problems were = real problems and the interference betw the two was confusing the = debugging.  It looks like the max skbuff frags is 18 (65536/page_size + 2) and indeed if the = chain of packet fragmets is longer than 19 (the logic probably allows = for one extra) it locks up the ring permanently.  The other problem was that the ring does indeed get depleted down to the point where the = available slots are fewer than the number needed for the current chain = of frags.  Unfortunately in this case the write is still permitted = which overwrites/corrupts freely and things immediately or pretty soon thereafter go kaplooey.  To confirm that there is = nothing else, I implemented a quick workaround - the chain of frags is = never allowed to be longer than 19 and if there aren't enough free slots = then the whole chain is dropped.  With these two changes all tests always completed and completed correctly. =  However, just dropping when not enough slots causes excessive pkt = loss so slows things randomly and a lot - it should either block or the = write should fail with an ENOBUFS flavoured exception.  The good news though is that it still works and a lot of the other tricky = machinery also works correctly.

Balraj



On Sun, May 26, 2013 at 10:53 AM, Anil = Madhavapeddy <anil@recoil.org> wrote:
The long chain of 36-byte frags is = consistent with the backend dropping it.  Does it work better if = you restrict the total fragment chains size to just 10 or 11?

The first unexplained packet loss is a real alarm bell though. =  The entire TCP retransmit code on our stack is just a canary that = spots latent bugs elsewhere in the device stack :-)

-anil

On 25 May 2013, at 22:25, Balraj Singh <balraj.singh@cl.cam.ac.uk> wrote:

In the particular test I am using I write 36 bytes of payload and = use the Mirage equivalent of TCP_NODELAY.  This works for a bit but = then suffers some packet loss (why? TBD) and triggers a rexmit. =  The retransmitted packet is 1400+ bytes and is made up of a long chain of 36 byte io_pages.  I thought that it may be = that the ring did not have enough slots to take all the chunks of the = pkt.  Making the retransmitted pkt be the size of the original = write improved it very significantly but it would still fail in the same way, tho less frequently.  I'm working on it - I see = available txring slots vary, but I havent yet found a case where the = slots are fully depleted or down to fewer than chunks that need to be = written.  I'm still narrowing it down.

This test originally was with 1-byte writes, but that seemed to = wedge even before the 1st data packet made it to the wire.  This = may be because of the limitation Steven mentioned.  I think I'm = getting close on the 36 byte write test, once this is figured out I'll try it with 1 byte writes again.

Balraj



On Sat, May 25, 2013 at 11:11 AM, Anil = Madhavapeddy <anil@recoil.org> wrote:
Balraj noticed that a stream of 1-byte writes on a TCP connection would = cause netfront/netback to wedge.  This is obviously quite = unrealistic, but a good regression test.

A quick chat with Steven Smith pointed out that some Linux netbacks had = a limit on the number of fragments allowed (based on the skbuff chain = limit size).  So you might be ending up with a situation where the = backend drops the entire set of fragments, and the frontend is retransmitting all the time.

So if you modify our frontend to limit the fragment size to ~10 or so = for any given packet, that might help.  On the other hand, if = you're doing writes with a TCP segment size of 1, but still only 3-4 = fragments (for the Ethernet/IP/TCP headers), then we have some other bug.  What does the Netif request look like, = Balraj?

-anil




= --Apple-Mail=_34039232-F57F-4FDC-88D4-6F15326D76FF-- From Dave.Scott@eu.citrix.com Tue May 28 11:11:26 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UhGsI-0000WG-HN (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 28 May 2013 11:11:26 +0100 X-Cam-SpamDetails: score -1.1 from SpamAssassin-3.3.2-1485974 * -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no * trust * [46.33.159.39 listed in list.dnswl.dnsbl.ja.net] * -1.1 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from smtp.eu.citrix.com ([46.33.159.39]:32396) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1UhGsD-0005Mt-7h (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 28 May 2013 11:11:26 +0100 X-IronPort-AV: E=Sophos;i="4.87,757,1363132800"; d="scan'208";a="5039229" Received: from lonpex01cl01.citrite.net ([10.30.203.101]) by LONPIPO01.EU.CITRIX.COM with ESMTP/TLS/AES128-SHA; 28 May 2013 10:11:21 +0000 Received: from LONPEX01CL03.citrite.net ([169.254.3.241]) by LONPEX01CL01.citrite.net ([169.254.1.104]) with mapi id 14.02.0342.003; Tue, 28 May 2013 11:11:21 +0100 From: Dave Scott To: "cl-mirage@lists.cam.ac.uk" Subject: new meeting ID for today's Mirage weekly call (1600-1700 BST) Thread-Topic: new meeting ID for today's Mirage weekly call (1600-1700 BST) Thread-Index: Ac5bi6dTd2RhbuRqTXmfraUOQIsX4Q== Date: Tue, 28 May 2013 10:11:20 +0000 Message-ID: <6FB4516F0E9B0F43B54F88D855ABB7903D59B5@LONPEX01CL03.citrite.net> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.30.203.1] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 May 2013 10:11:26 -0000 Content-Length: 2735 Lines: 89 Hi, Here's a new meeting invite since the old one has been GCed. It's at 1600-1= 700 BST. 1. Please join my meeting. https://www1.gotomeeting.com/join/591890401 2. Use your microphone and speakers (VoIP) - a headset is recommended. Or= , call in using your telephone. United States: +1 (626) 521-0017 Argentina (toll-free): 0 800 266 1385 Australia (toll-free): 1 800 191 358 Australia: +61 2 8355 1038 Austria (toll-free): 0 800 080061 Austria: +43 (0) 7 2088 0716 Bahrain (toll-free): 800 81 305 Belarus (toll-free): 8 820 0011 0331 Belgium (toll-free): 0 800 81388 Belgium: +32 (0) 28 08 4372 Brazil (toll-free): 0 800 047 4909 Canada (toll-free): 1 877 777 3281 Canada: +1 (647) 497-9380 China (toll-free): 4008 866154 Czech Republic (toll-free): 800 500453 Denmark (toll-free): 8025 0919 Denmark: +45 (0) 69 91 84 58 Finland (toll-free): 80094473 Finland: +358 (0) 931 58 1773 France (toll-free): 0 805 541 052 France: +33 (0) 170 950 590 Germany (toll-free): 0 800 723 5274 Germany: +49 (0) 811 8899 6934 Hong Kong (toll-free): 30774812 Iceland (toll-free): 800 9993 India (toll-free): 000 800 100 8227 Indonesia (toll-free): 001 803 020 2563 Ireland (toll-free): 1 800 818 263 Ireland: +353 (0) 15 133 006 Israel (toll-free): 1 809 388 020 Italy (toll-free): 800 906962 Italy: +39 0 699 26 68 65 Japan (toll-free): 0 120 242 200 Korea, Republic of (toll-free): 806180880 Luxembourg (toll-free): 800 81016 Malaysia (toll-free): 1 800 81 6504 Mexico (toll-free): 01 800 123 8367 Netherlands (toll-free): 0 800 020 0178 Netherlands: +31 (0) 208 080 759 New Zealand (toll-free): 0 800 44 9375 New Zealand: +64 (0) 9 974 9579 Norway (toll-free): 800 33 083 Norway: +47 21 04 30 59 Panama (toll-free): 18005072789 Peru (toll-free): 0 800 55253 Philippines (toll-free): 1 800 1110 1565 Poland (toll-free): 00 800 3211434 Portugal (toll-free): 800 180 139 Russian Federation (toll-free): 8 800 100 6914 Saudi Arabia (toll-free): 800 844 3636 Singapore (toll-free): 800 321 1143 South Africa (toll-free): 0 800 988 836 Spain (toll-free): 0 800 900 593 Spain: +34 931 76 1534 Sweden (toll-free): 020 794 545 Sweden: +46 (0) 852 500 691 Switzerland (toll-free): 0 800 000 452 Switzerland: +41 (0) 435 0026 89 Taiwan (toll-free): 00 800 666 846 Thailand (toll-free): 001 800 852 2442 Turkey (toll-free): 00 800 4488 29001 Ukraine (toll-free): 0 800 50 4691 United Arab Emirates (toll-free): 800 044 40444 United Kingdom (toll-free): 0 808 168 0209 United Kingdom: +44 (0) 207 151 1817 United States (toll-free): 1 877 309 2070 Uruguay (toll-free): 4054459 Viet Nam (toll-free): 180 06 627 Access Code: 591-890-401 Audio PIN: Shown after joining the meeting Meeting ID: 591-890-401 GoToMeeting(r)=20 Online Meetings Made Easy(r) From balraj885@gmail.com Tue May 28 15:11:23 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UhKcV-0003u0-6a (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 28 May 2013 15:11:23 +0100 X-Cam-SpamDetails: score -0.3 from SpamAssassin-3.3.2-1485974 * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.216.47 listed in list.dnswl.dnsbl.ja.net] * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (balraj885[at]gmail.com) * 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in * digit (balraj885[at]gmail.com) * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from mail-qa0-f47.google.com ([209.85.216.47]:62713) by ppsw-52.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.148]:25) with esmtp id 1UhKcT-0000AM-FQ (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Tue, 28 May 2013 15:11:23 +0100 Received: by mail-qa0-f47.google.com with SMTP id o13so1521722qaj.20 for ; Tue, 28 May 2013 07:11:21 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.224.180.206 with SMTP id bv14mr32163885qab.56.1369750281036; Tue, 28 May 2013 07:11:21 -0700 (PDT) Sender: balraj885@gmail.com Received: by 10.49.29.232 with HTTP; Tue, 28 May 2013 07:11:20 -0700 (PDT) In-Reply-To: References: <27BA97B7-F2CC-414E-AEB4-268FBC5A62A7@recoil.org> Date: Tue, 28 May 2013 15:11:20 +0100 X-Google-Sender-Auth: cbzFWX4KacBJUpVaA8bHtX9MuVs Message-ID: Subject: Re: one-byte TCP writes wedging From: Balraj Singh To: Anil Madhavapeddy Content-Type: multipart/alternative; boundary=20cf30684a71eea90e04ddc7d6b7 Cc: Dave Scott , "cl-mirage@lists.cam.ac.uk List" X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 May 2013 14:11:23 -0000 Content-Length: 15639 Lines: 363 --20cf30684a71eea90e04ddc7d6b7 Content-Type: text/plain; charset=ISO-8859-1 Hi Dave, Anil, "Wait for n free slots" is exactly what is needed. And at this low level, even if there are multiple writers, the writes should all be serialized. It'll be best if the order of the pkts on the wire is always the same as order of the successful writes from the application. Also in addition to the blocking write, we should also have a non-blocking write that fails if there aren't enough free slots available. For the fragments, I think we should have an internal threshold (say 8 fragments) and if the number of fragments in the write is great than the threshold then it triggers a compaction or repacking before sticking it on the ring. As long as the threshold is higher than most use cases it should have no impact at all. In any case the repacking work has to be done at some point so it shouldn't affect performance even if it is triggered often. Balraj On Tue, May 28, 2013 at 10:15 AM, Anil Madhavapeddy wrote: > I think it's far safer to serialise a single fragment batch on the ring > (in an ordered Lwt_sequence) rather than have fragments interleaved across > multiple packet write requests. > > Any other path will have packets being transmitted out-of-order, which > will severely mess up performance. This includes the case where there are > multiple outstanding write requests with different numbers of fragments -- > these should be delivered in the order they are transmitted, and so a large > packet could indeed block a series of small ones. It also makes hardware > offload easier if the fragments aren't scattered over the ring. > > -anil > > On 28 May 2013, at 08:46, Dave Scott wrote: > > Hi Balraj, > > Very interesting discoveries! > > Regarding the skbuff frag limit, should this be considered as part of > the protocol even though it was originally a Linux implementation issue > leaking through? Do you know if it has been stable over time? It might be > worth asking on xen-devel. > > Regarding there not being enough slots: there's already a "wait for a > free slot" mechanism so we could add a "wait for n free slots". Do you have > parallel threads transmitting at once? We should probably take care that a > "wait for n" doesn't end up constantly getting gazumped by lots of "wait > for 1"s > > Cheers, > > -- > Dave Scott > > On May 27, 2013, at 11:33 PM, "Balraj Singh" > wrote: > > It turned out that both the suspected problems were real problems and > the interference betw the two was confusing the debugging. It looks like > the max skbuff frags is 18 (65536/page_size + 2) and indeed if the chain > of packet fragmets is longer than 19 (the logic probably allows for one > extra) it locks up the ring permanently. The other problem was that the > ring does indeed get depleted down to the point where the available slots > are fewer than the number needed for the current chain of frags. > Unfortunately in this case the write is still permitted which > overwrites/corrupts freely and things immediately or pretty soon thereafter > go kaplooey. To confirm that there is nothing else, I implemented a quick > workaround - the chain of frags is never allowed to be longer than 19 and > if there aren't enough free slots then the whole chain is dropped. With > these two changes all tests always completed and completed correctly. > However, just dropping when not enough slots causes excessive pkt loss so > slows things randomly and a lot - it should either block or the write > should fail with an ENOBUFS flavoured exception. The good news though is > that it still works and a lot of the other tricky machinery also works > correctly. > > Balraj > > > > On Sun, May 26, 2013 at 10:53 AM, Anil Madhavapeddy wrote: > >> The long chain of 36-byte frags is consistent with the backend dropping >> it. Does it work better if you restrict the total fragment chains size to >> just 10 or 11? >> >> The first unexplained packet loss is a real alarm bell though. The >> entire TCP retransmit code on our stack is just a canary that spots latent >> bugs elsewhere in the device stack :-) >> >> -anil >> >> On 25 May 2013, at 22:25, Balraj Singh >> wrote: >> >> In the particular test I am using I write 36 bytes of payload and use >> the Mirage equivalent of TCP_NODELAY. This works for a bit but then >> suffers some packet loss (why? TBD) and triggers a rexmit. The >> retransmitted packet is 1400+ bytes and is made up of a long chain of 36 >> byte io_pages. I thought that it may be that the ring did not have enough >> slots to take all the chunks of the pkt. Making the retransmitted pkt be >> the size of the original write improved it very significantly but it would >> still fail in the same way, tho less frequently. I'm working on it - I see >> available txring slots vary, but I havent yet found a case where the slots >> are fully depleted or down to fewer than chunks that need to be written. >> I'm still narrowing it down. >> >> This test originally was with 1-byte writes, but that seemed to wedge >> even before the 1st data packet made it to the wire. This may be because >> of the limitation Steven mentioned. I think I'm getting close on the 36 >> byte write test, once this is figured out I'll try it with 1 byte writes >> again. >> >> Balraj >> >> >> >> On Sat, May 25, 2013 at 11:11 AM, Anil Madhavapeddy wrote: >> >>> Balraj noticed that a stream of 1-byte writes on a TCP connection would >>> cause netfront/netback to wedge. This is obviously quite unrealistic, but >>> a good regression test. >>> >>> A quick chat with Steven Smith pointed out that some Linux netbacks had >>> a limit on the number of fragments allowed (based on the skbuff chain limit >>> size). So you might be ending up with a situation where the backend drops >>> the entire set of fragments, and the frontend is retransmitting all the >>> time. >>> >>> So if you modify our frontend to limit the fragment size to ~10 or so >>> for any given packet, that might help. On the other hand, if you're doing >>> writes with a TCP segment size of 1, but still only 3-4 fragments (for the >>> Ethernet/IP/TCP headers), then we have some other bug. What does the Netif >>> request look like, Balraj? >>> >>> -anil >>> >> >> >> > > --20cf30684a71eea90e04ddc7d6b7 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi Dave, Anil,

"Wait for n free sl= ots" is exactly what is needed. =A0And at this low level, even if ther= e are multiple writers, the writes should all be serialized. =A0It'll b= e best if the order of the pkts on the wire is always the same as order of = the successful writes from the application. =A0Also in addition to the bloc= king write, we should also have a non-blocking write that fails if there ar= en't enough free slots available.

For the fragments, I think we should have an internal = threshold (say 8 fragments) and if the number of fragments in the write is = great than the threshold then it triggers a compaction or repacking before = sticking it on the ring. =A0As long as the threshold is higher than most us= e cases it should have no impact at all. =A0In any case the repacking work = has to be done at some point so it shouldn't affect performance even if= it is triggered often.

Balraj



On Tue, May= 28, 2013 at 10:15 AM, Anil Madhavapeddy <anil@recoil.org> wro= te:
I think = it's far safer to serialise a single fragment batch on the ring (in an = ordered Lwt_sequence) rather than have fragments interleaved across multipl= e packet write requests.

Any other path will have packets being transmitted out-of-or= der, which will severely mess up performance. =A0This includes the case whe= re there are multiple outstanding write requests with different numbers of = fragments -- these should be delivered in the order they are transmitted, a= nd so a large packet could indeed block a series of small ones. =A0It also = makes hardware offload easier if the fragments aren't scattered over th= e ring.

-anil

On 28 May 2013, = at 08:46, Dave Scott <Dave.Scott@eu.citrix.com> wrote:

Hi Balraj,

Very interesting discoveries!

Regarding the skbuff frag limit, should this be considered as part of = the protocol even though it was originally a Linux implementation issue lea= king through? Do you know if it has been stable over time? It might be wort= h asking on xen-devel.

Regarding there not being enough slots: there's already a "wa= it for a free slot" mechanism so we could add a "wait for n free = slots". Do you have parallel threads transmitting at once? We should p= robably take care that a "wait for n" doesn't end up constant= ly getting gazumped by lots of "wait for 1"s

Cheers,

--=A0
Dave Scott

On May 27, 2013, at 11:33 PM, "Balraj Singh" <balraj.singh@cl.cam.ac.uk> wrote:

It turned out that both the suspected problems were real p= roblems and the interference betw the two was confusing the debugging. =A0I= t looks like the max skbuff frags is 18 (65536/page_size + 2) and indeed if the chain of packet frag= mets is longer than 19 (the logic probably allows for one extra) it locks u= p the ring permanently. =A0The other problem was that the ring does indeed get depleted down to the point where the ava= ilable slots are fewer than the number needed for the current chain of frag= s. =A0Unfortunately in this case the write is still permitted which overwri= tes/corrupts freely and things immediately or pretty soon thereafter go kaplooey. =A0To confirm that there is nothing= else, I implemented a quick workaround - the chain of frags is never allow= ed to be longer than 19 and if there aren't enough free slots then the = whole chain is dropped. =A0With these two changes all tests always completed and completed correctly. =A0However, ju= st dropping when not enough slots causes excessive pkt loss so slows things= randomly and a lot - it should either block or the write should fail with = an ENOBUFS flavoured exception. =A0The good news though is that it still works and a lot of the other tricky mach= inery also works correctly.

Balraj



On Sun, May 26, 2013 at 10:53 AM, Anil Madhavape= ddy <anil@recoil.org> wrote:
The long chain of 36-byte frags is cons= istent with the backend dropping it. =A0Does it work better if you restrict= the total fragment chains size to just 10 or 11?

The first unexplained packet loss is a real alarm bell though. =A0The = entire TCP retransmit code on our stack is just a canary that spots latent = bugs elsewhere in the device stack :-)

-anil


In the particular test I am using I write 36 bytes of payload and use = the Mirage equivalent of TCP_NODELAY. =A0This works for a bit but then suff= ers some packet loss (why? TBD) and triggers a rexmit. =A0The retransmitted= packet is 1400+ bytes and is made up of a long chain of 36 byte io_pages. =A0I thought that it may be that the = ring did not have enough slots to take all the chunks of the pkt. =A0Making= the retransmitted pkt be the size of the original write improved it very s= ignificantly but it would still=A0fail in the same way, tho less frequently. =A0I'm working on it - I see ava= ilable txring slots vary, but I havent yet found a case where the slots are= fully depleted or down to fewer than chunks that need to be written. =A0I&= #39;m still narrowing it down.

This test originally was with 1-byte writes, but that seemed to wedge = even before the 1st data packet made it to the wire. =A0This may be because= of the limitation Steven mentioned. =A0I think I'm getting close on th= e 36 byte write test, once this is figured out I'll try it with 1 byte writes again.

Balraj



On Sat, May 25, 2013 at 11:11 AM, Anil Madhavape= ddy <anil@recoil.org> wrote:
Balraj noticed that a stream of 1-byte writes on a TCP connection would cau= se netfront/netback to wedge. =A0This is obviously quite unrealistic, but a= good regression test.

A quick chat with Steven Smith pointed out that some Linux netbacks had a l= imit on the number of fragments allowed (based on the skbuff chain limit si= ze). =A0So you might be ending up with a situation where the backend drops = the entire set of fragments, and the frontend is retransmitting all the time.

So if you modify our frontend to limit the fragment size to ~10 or so for a= ny given packet, that might help. =A0On the other hand, if you're doing= writes with a TCP segment size of 1, but still only 3-4 fragments (for the= Ethernet/IP/TCP headers), then we have some other bug. =A0What does the Netif request look like, Balraj?

-anil





--20cf30684a71eea90e04ddc7d6b7-- From amc79@cam.ac.uk Wed May 29 18:42:27 2013 Received: from ppsw-52.csi.cam.ac.uk ([131.111.8.152]) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1UhkOI-0002Wr-Vl (Exim 4.70) (return-path ); Wed, 29 May 2013 18:42:26 +0100 X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from dhcp-172-17-157-185.eduroam.lapwing.private.cam.ac.uk ([172.17.157.185]:52086) by ppsw-52.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.158]:587) with esmtpsa (PLAIN:amc79) (TLSv1:AES128-SHA:128) id 1UhkOI-0004lc-Fo (Exim 4.80_167-5a66dd3) (return-path ); Wed, 29 May 2013 18:42:26 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: OCaml Labs Meeting - Friday 31st May at 4pm in the Computer Lab From: Amir Chaudhry In-Reply-To: <5A7CD1F1-5A82-4FDC-AAB9-9402E7A42C4F@cam.ac.uk> Date: Wed, 29 May 2013 18:42:26 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5A7CD1F1-5A82-4FDC-AAB9-9402E7A42C4F@cam.ac.uk> To: "cl-ocamllabs@lists.cam.ac.uk" , "cl-mirage@lists.cam.ac.uk List" X-Mailer: Apple Mail (2.1503) X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 May 2013 17:42:27 -0000 Content-Length: 1932 Lines: 79 Dear all, The next meeting will take place on Friday at 4pm in FW26 in the = Computer Lab and the Agenda is below. We'll have a lot of demos and I've added links to relevant repos and = sites so you can look over things beforehand. Overview - 6 Months of OCL (Anil ~10mins). Yup, OCaml Labs has been going for 6 months now and Anil will give a = brief round up. You can catchup on all our monthly updates via the = website:=20 http://www.cl.cam.ac.uk/projects/ocamllabs/news/ Platform and Systems - OCamlot Demo (David Sheets ~10mins). repo: https://github.com/ocamllabs/ocamlot - Vector Graphics demo (Daniel Buenzli ~5mins) repo: http://erratique.ch/repos/vg - OCaml-Java demo (Xavier Clerc ~10mins) site: http://ocamljava.x9c.fr/preview/ Compiler - OPAM-Doc (Vincent Botbol/Leo White ~5mins) repo: https://github.com/vincent-botbol/opam-doc repo: https://github.com/vincent-botbol/bin-doc - ocaml-ctypes demo (Jeremy Yallop ~10mins) repo (pending release): https://github.com/ocamllabs/ocaml-ctypes - The OCaml Bug (Jeremy Yallop ~5mins) ref: http://caml.inria.fr/mantis/view.php?id=3D5985 AoB - Lab Happy hour! (from 5pm) Amir Chaudhry wrote: > Dear all, >=20 > The next OCaml Labs meeting will take place on the 31st of May at 4pm = in the Lab. > Please note the change of time with respect to previous meetings. > This also means we can continue discussions in the pub after the = meeting :) >=20 > A skeleton agenda is below and a more detailed one will follow in = advance of the meeting. =20 >=20 > Please do let me know if you will be attending. >=20 > -- Details -- > OCaml Labs Meeting > 31st May 2013 > 4pm =96 5pm > Room FW26 (TBC) - Cambridge Computer Laboratory > William Gates Building > JJ Thomson Avenue > Cambridge CB3 0FD >=20 > -- Agenda -- > OCL Updates > - Platform projects > - Systems projects > - Compiler projects > Open discussion > Close >=20 > Best wishes, > Amir From vb@luminar.eu.org Thu May 30 12:58:38 2013 Received: from ppsw-mx-f.csi.cam.ac.uk ([131.111.8.149] helo=ppsw-42.csi.cam.ac.uk) by lists-2.csi.cam.ac.uk (lists.cam.ac.uk [131.111.8.15]:25) with esmtp id 1Ui1V8-0002OF-59 (Exim 4.70) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 30 May 2013 12:58:38 +0100 X-Cam-SpamDetails: score -1.1 from SpamAssassin-3.3.2-1485974 * -1.1 RP_MATCHES_RCVD Envelope sender domain matches handover relay * domain X-Cam-ScannerInfo: http://www.ucs.cam.ac.uk/email/scanner/ Received: from luminar.eu.org ([94.23.24.152]:54283) by ppsw-42.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.149]:25) with esmtp id 1Ui1V7-0001Rk-8F (Exim 4.80_167-5a66dd3) for cl-mirage@lists.cam.ac.uk (return-path ); Thu, 30 May 2013 12:58:38 +0100 Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) by luminar.eu.org (Postfix) with ESMTP id B1CA4BF6FD for ; Thu, 30 May 2013 13:57:48 +0200 (CEST) Message-ID: <51A73EEC.5020706@luminar.eu.org> Date: Thu, 30 May 2013 12:58:36 +0100 From: Vincent Bernardoff User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130514 Thunderbird/17.0.6 MIME-Version: 1.0 To: cl-mirage@lists.cam.ac.uk Subject: New UNIX backend, mirage-platform/mirage-net Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: cl-mirage@lists.cam.ac.uk X-Mailman-Version: 2.1.8 Precedence: list List-Id: MirageOS development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 May 2013 11:58:38 -0000 Content-Length: 1439 Lines: 36 Hi all, I would like any potential person interested in the UNIX-direct backend for Mirage to have a look at the changes I am going to introduce: * In mirage-platform: https://github.com/mirage/mirage-platform/pull/43 * In mirage-net: https://github.com/mirage/mirage-net/pull/28 Basically, the changes remove the creation of virtual interfaces (tap) from mirage-platform, instead, mirage-platform will listen to a Lwt_stream in order to receive "vif infos", the stream itself being pushable via a function in OS.Netif (OS.Netif.add_vif). The changes in mirage-net reflect those changes in mirage-platform, the Manager.create function now only takes a callback argument (the function to be called when new interfaces pop up), instead of giving optional arguments for OS.Netif.create to create something that has been requested by the manager. This is more consistent with how a unikernel under Xen gets its network interfaces via the XenStore. It is now up to mirari (or code generated by mirari) to call OS.Netif.add_vif with a fd of a created/opened interface. Haris, if you always work on openvswitch stuff on Mirage, you are probably the person the most concerned by those changes, I might have broken a bit your PCAP stuff, please tell me if you have any problem I would work with you to solve them. Cheers, Vincent PS: This new backend comes with a new version of mirari whose release will follow very soon.