From mirageos-devel-bounces@lists.xenproject.org Tue Oct 01 15:02:28 2024 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 01 Oct 2024 15:02:28 +0000 Received: from list by lists.xenproject.org with outflank-mailman.808213.1220045 (Exim 4.92) (envelope-from ) id 1sveOG-0002Wv-F9; Tue, 01 Oct 2024 15:02:16 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808213.1220045; Tue, 01 Oct 2024 15:02:16 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sveOG-0002Wo-CC; Tue, 01 Oct 2024 15:02:16 +0000 Received: by outflank-mailman (input) for mailman id 808213; Tue, 01 Oct 2024 15:02:15 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sveOE-0002Wi-EY for mirageos-devel@lists.xenproject.org; Tue, 01 Oct 2024 15:02:15 +0000 Received: from mail.w13.tutanota.de (mail.w13.tutanota.de [185.205.69.213]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 21f831bc-8006-11ef-a0ba-8be0dac302b0; Tue, 01 Oct 2024 17:02:12 +0200 (CEST) Received: from tutadb.w10.tutanota.de (w10.api.tuta.com [IPv6:fd:ac::d:10]) by mail.w13.tutanota.de (Postfix) with ESMTP id 37C3F27B50D1 for ; Tue, 1 Oct 2024 17:02:09 +0200 (CEST) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: 21f831bc-8006-11ef-a0ba-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1727794929; s=s1; d=tuta.io; h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Content-Transfer-Encoding:Cc:Date:Date:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:Sender; bh=oYEWHuGvothWX5Y2va3+9tk1fgWnVvpkzi09I7mnyzg=; b=lIkAHFbLkH/2GH0R3BZgnDdVdfhAmKop75j5wqTVaqixwERFQKI5T3yupDKwE2IQ dU6/WrUGd0S6rHVSnfE7X9AvNgGM3CW2yvs0Tt5Wulf6/H2wxgzrroLIXqnagRVtktz s1Dgtfj9F6cK/GvZJ9ObM9rA28+QHBZYj8xuj6KKk01qhZEgQoD659uXIX5JhyMRVVL jFfX18CmK7zwBrjvKsAA6AphweZNdHyKmt8pz5qbOUW0fAmfKuk0kmiQZYHmuMUZZPp lJ3qlbIXwahg1TtxI0UOcRgFjoT6tNdH8Fa/p1HHU4ZvfirZfkrgUWkXhMMrkP6GBfp VaFk1nHI1A== Date: Tue, 1 Oct 2024 17:02:09 +0200 (CEST) From: pierre.alain@tuta.io To: Mirageos Devel Message-ID: Subject: Remove Cstruct: one step forward? MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Dear all, Over the past few months, a large amount of work has been carried out to reduce the use of Cstruct.t. This was mainly driven by performances, as there were a lot of expensive allocations for small buffers (from 0 to a couple of bytes). Indeed, there were big performance improvements with mirage-crypto (which weren't limited to extracting Cstruct, but also algorithm improvements and a deeper thinking about allocations, like allocating a buffer once, and writing to it rather than allocating multiple buffers and concatenating them). Now I'm wondering if this could be a time to extend this work of Cstruct removal with another API breakage. You can see at https://github.com/mirage/mirage-net/pull/25 a proposal for the new API. It's still conservative in term of "where each buffers would be allocated". Of course, if it is merged, there will be the need for lot of PR to make dependent libraries compliant with the new interface. That's why I'm happy to ask your opinion on this possible step forward :) Best, Pierre -- P. From mirageos-devel-bounces@lists.xenproject.org Tue Oct 01 15:26:42 2024 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 01 Oct 2024 15:26:42 +0000 Received: from list by lists.xenproject.org with outflank-mailman.808266.1220160 (Exim 4.92) (envelope-from ) id 1sveln-0003Pw-NC; Tue, 01 Oct 2024 15:26:35 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808266.1220160; Tue, 01 Oct 2024 15:26:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sveln-0003Pp-Jq; Tue, 01 Oct 2024 15:26:35 +0000 Received: by outflank-mailman (input) for mailman id 808266; Tue, 01 Oct 2024 15:26:34 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svell-0003Mx-Oy for mirageos-devel@lists.xenproject.org; Tue, 01 Oct 2024 15:26:34 +0000 Received: from mta3.cl.cam.ac.uk (mta3.cl.cam.ac.uk [2001:630:212:200::25:3]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 899d32a6-8009-11ef-a0ba-8be0dac302b0; Tue, 01 Oct 2024 17:26:32 +0200 (CEST) Received: from fauth-a2-smtp.messagingengine.com ([103.168.172.201]:33007) (dnseec=no) by mta3.cl.cam.ac.uk:587 [128.232.25.23] with esmtpsa auth as avsm2+fm (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) id 1svelk-009ZI3-6a (envelope-from ); Tue, 01 Oct 2024 16:26:32 +0100 Received: from phl-compute-05.internal (phl-compute-05.phl.internal [10.202.2.45]) by mailfauth.phl.internal (Postfix) with ESMTP id 47ED11200043; Tue, 1 Oct 2024 11:26:27 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-05.internal (MEProxy); Tue, 01 Oct 2024 11:26:27 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 1 Oct 2024 11:26:26 -0400 (EDT) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: 899d32a6-8009-11ef-a0ba-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=cl.cam.ac.uk; s=mta3; h=References:To:Cc:In-Reply-To:Date:Subject: Mime-Version:Content-Type:Message-Id:From:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=a1yoBgW5sTlXqhk0yYBrtzkc6jjVmSwfyjqOxOeCNvU=; t=1727796392; x=1728660392; b=TPvm0Fb4gSb9sE4DB44wbWB8shwHLEavfMzBUUR+fc7JVsF5ZruXHexwjGMftidE0133BfpX1Kb Sw7W3XEy2+2/uHVTUyz8Y5XHcyS+wu9L2/VQHNNscZO9k7Dhf6lNS9noINSIsU3a8DSSmIUPv8sHW 2FHUJn8wAqoPC5Z7RWaQnousiAvvYnpJ6OcomrLBgoQPRMzFSBQgpsivcHrIAD5Wx1PU+Ru/um/ia a1mHdSS4VSQq9prNc5U7//WdGBmm6z0UtGH3r1Oqh6MzDCrUJOLoRibMAGnweUUBxleAakF/o1NdL nC64CF2aovCBSlUjdQ9BR3ldMre45imwhPIw==; X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrvddujedgkeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhephffktgggufffjgevvfhfofesrgdtmherhhdtjeen ucfhrhhomheptehnihhlucforgguhhgrvhgrphgvugguhicuoegrvhhsmhdvsegtlhdrtg grmhdrrggtrdhukheqnecuggftrfgrthhtvghrnhepvdfhudegheevffethedthfduvdek hefhgedutdejkeeijefhteeuvdfgteekvefgnecuffhomhgrihhnpehgihhthhhusgdrtg homhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegr vhhsmhdvodhmvghsmhhtphgruhhthhhpvghrshhonhgrlhhithihqdduvdehvdekfeduje efqddvkeduleejgeeiledqrghvshhmvdeppegtlhdrtggrmhdrrggtrdhukhesfhhmrdgt lhdrtggrmhdrrggtrdhukhdpnhgspghrtghpthhtohepvddpmhhouggvpehsmhhtphhouh htpdhrtghpthhtohepphhivghrrhgvrdgrlhgrihhnsehtuhhtrgdrihhopdhrtghpthht ohepmhhirhgrghgvohhsqdguvghvvghlsehlihhsthhsrdigvghnphhrohhjvggtthdroh hrgh X-ME-Proxy: Feedback-ID: i9ebc46d9:Fastmail From: Anil Madhavapeddy Message-Id: <22E46BE9-4101-4EBB-8D9B-C17736BBC5B4@cl.cam.ac.uk> Content-Type: multipart/alternative; boundary="Apple-Mail=_66C01EFA-A71F-4877-B842-2E8BC98C8D3A" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.200.84.1.2\)) Subject: Re: Remove Cstruct: one step forward? Date: Tue, 1 Oct 2024 16:26:14 +0100 In-Reply-To: Cc: Mirageos Devel To: pierre.alain@tuta.io References: X-Mailer: Apple Mail (2.3826.200.84.1.2) X-cl-SRS: mirageos-devel@lists.xenproject.org vs mirageos-devel@lists.xenproject.org --Apple-Mail=_66C01EFA-A71F-4877-B842-2E8BC98C8D3A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Dear Pierre, While this is a good step forward, it is worth considering how to deal = with the requirements of some backends for non-moving IO pages to pass = through to the kernel/unikernel. This interface will force a copy, but = perhaps that's ok. It should be possible to make OCaml 5 bytes = non-moving with some consideration for the garbage collector, but I = haven't had a chance to think through the details. best, Anil -- Anil Madhavapeddy, Professor of Planetary Computing Computer Laboratory, University of Cambridge (cst.cam.ac.uk) > On Oct 1, 2024, at 4:02=E2=80=AFPM, pierre.alain@tuta.io wrote: >=20 > Dear all, >=20 > Over the past few months, a large amount of work has been carried out = to reduce the use of Cstruct.t. This was mainly driven by performances, = as there were a lot of expensive allocations for small buffers (from 0 = to a couple of bytes). Indeed, there were big performance improvements = with mirage-crypto (which weren't limited to extracting Cstruct, but = also algorithm improvements and a deeper thinking about allocations, = like allocating a buffer once, and writing to it rather than allocating = multiple buffers and concatenating them). >=20 > Now I'm wondering if this could be a time to extend this work of = Cstruct removal with another API breakage. You can see at = https://github.com/mirage/mirage-net/pull/25 a proposal for the new API. = It's still conservative in term of "where each buffers would be = allocated". Of course, if it is merged, there will be the need for lot = of PR to make dependent libraries compliant with the new interface. >=20 > That's why I'm happy to ask your opinion on this possible step forward = :) > Best, > Pierre >=20 > -- > P. >=20 --Apple-Mail=_66C01EFA-A71F-4877-B842-2E8BC98C8D3A Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Dear = Pierre,

While this is a good step forward, it is = worth considering how to deal with the requirements of some backends for = non-moving IO pages to pass through to the kernel/unikernel. This = interface will force a copy, but perhaps that's ok. It should be = possible to make OCaml 5 bytes non-moving with some consideration for = the garbage collector, but I haven't had a chance to think through the = details.

best,
Anil

--
Anil Madhavapeddy, Professor = of Planetary Computing
Computer Laboratory, University = of Cambridge (cst.cam.ac.uk)

On Oct 1, 2024, at 4:02=E2=80=AFPM= , pierre.alain@tuta.io wrote:

Dear all,

Over the = past few months, a large amount of work has been carried out to reduce = the use of Cstruct.t. This was mainly driven by performances, as there = were a lot of expensive allocations for small buffers (from 0 to a = couple of bytes). Indeed, there were big performance improvements with = mirage-crypto (which weren't limited to extracting Cstruct, but also = algorithm improvements and a deeper thinking about allocations, like = allocating a buffer once, and writing to it rather than allocating = multiple buffers and concatenating them).

Now I'm wondering if = this could be a time to extend this work of Cstruct removal with another = API breakage. You can see at = https://github.com/mirage/mirage-net/pull/25 a proposal for the new API. = It's still conservative in term of "where each buffers would be = allocated". Of course, if it is merged, there will be the need for lot = of PR to make dependent libraries compliant with the new = interface.

That's why I'm happy to ask your opinion on this = possible step forward = :)
Best,
Pierre

--
P.


= --Apple-Mail=_66C01EFA-A71F-4877-B842-2E8BC98C8D3A-- From mirageos-devel-bounces@lists.xenproject.org Wed Oct 02 08:55:22 2024 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 02 Oct 2024 08:55:22 +0000 Received: from list by lists.xenproject.org with outflank-mailman.808647.1220609 (Exim 4.92) (envelope-from ) id 1svv8a-0002ls-0A; Wed, 02 Oct 2024 08:55:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808647.1220609; Wed, 02 Oct 2024 08:55:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svv8Z-0002ll-St; Wed, 02 Oct 2024 08:55:11 +0000 Received: by outflank-mailman (input) for mailman id 808647; Wed, 02 Oct 2024 08:55:10 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svv8Y-0002lf-19 for mirageos-devel@lists.xenproject.org; Wed, 02 Oct 2024 08:55:10 +0000 Received: from mail.mehnert.org (mail.mehnert.org [213.73.89.200]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 05e7a142-809c-11ef-99a2-01e77a169b0f; Wed, 02 Oct 2024 10:55:07 +0200 (CEST) Received: from [192.168.42.80] (i5C74C0C3.versanet.de [92.116.192.195]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "hannes@mehnert.org", Issuer "mehnert root CA" (not verified)) by mail.mehnert.org (Postfix) with ESMTPS id 8ECEAF3C8 for ; Wed, 2 Oct 2024 10:55:06 +0200 (CEST) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: 05e7a142-809c-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mehnert.org; s=mail; t=1727859306; bh=AjJdDuHNF54NkvG1oZmnsNgBgL1ZpsBrmkvySUrp1oM=; h=Date:Subject:To:References:From:In-Reply-To; b=MozPTZXUIsLX8ZvR4/hlfFvXOhdqQBY2aYup6Lxdo7GEdg0JQ2ckr8MzrCi5gxtyg WoNbCGtwjNBDOFekg3t9auyY2vSNSWgyAire2eS0kCeuGmK/uXbpChYrz1t2nbjXt7 7+FanarNlUEf8vJg/I8X6kKPInx1shYP7WgLUkqVaN1iA71V419puVJ9kfvArC/lzl NhX4ZGj14YfIMBtUuJIpqFnXRmS3vp+VFBcyaUl6Ykrl2hMnTNpdHGqO8LrjcohC3X RCjVp6OhOcKxiy6+k/2jRdtf8Z5lIN80RnraMeA0/Iq2EaaendjT5PkWYrA+6NFaDD dhDWSj/ce0iiw== Message-ID: <3b89122d-03e1-4ad3-96c5-6f8091a30ae1@mehnert.org> Date: Wed, 2 Oct 2024 10:55:05 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Remove Cstruct: one step forward? To: mirageos-devel@lists.xenproject.org References: <22E46BE9-4101-4EBB-8D9B-C17736BBC5B4@cl.cam.ac.uk> Content-Language: en-US From: Hannes Mehnert Autocrypt: addr=hannes@mehnert.org; keydata= xsFLBEIw1AoBEADAtXwEV8F1DBpE9lnBTbHDNeZwDVp84MhxxIT5GUexGgbOWGSEWHhC3rYe FfGRUxF4M9P4fwxpxCS5YCvxoijWHeEf8nG5IkztVv5cw63E443XWHcCMc80YAwglZ2cSP4U GTNeKb9rqVPckk/PL348BYRawhzvZK+Bc+bUvbtPCfUXT1BWIxAR1dzsfpAQVNZ4bA06xOoP QJYVNgl/lWOmQgnSgb0dE2zsgddKTOj05ru7Q7LobB7WAUTRJVkZcXnrvI1SOt/WbPTyqF8l RBh94xCqFhv4SlqZVOTXxo9gw3LpDv/cYXRl/m7+/7Wljl3ziQ9cawA6O1mbw8nm7Sfa+TZl qo+5lXEenXG+MCbH0XnnL2I4BO6HSGDtKX6htTG2xs6w4r9mVxTGJuJcGrC0dxuz5j4jylt/ KOVn9IaRKzhj8ga7kWffMp+JYdrn43732weoFFJxm78mD2ij4UbJtNkQIIcTv8IBJajHy2P3 h1NuBIwwb7RmBav4oo0CKWoasIHFwjMSBpCzJ8QOHeO/F3TY3DZp7FTwViUgSXVJoewO9yFG ctX7MC27/F1IonU9/SJW0j+F3Vz32SfxUBrDnLYpO7/vwA8w+xmWLnl0iJN/8injz5+CigsP e7O66t4MtC9BVCuLu7a/ikH5nW0q6RyTW8of9eZIsuEyqF1ZPwAGKc0jSGFubmVzIE1laG5l cnQgPGhhbm5lc0BtZWhuZXJ0Lm9yZz7CwXQEEwECAB4FAkIw1A0CGwMGCwkIBwMCAxUCAwMW AgECHgECF4AACgkQvIlliN98KO5HYg//UD6gk4sFcNop/EQivcnpfPnHrrUddsBl9bovQSXb zIh5HY/8xhO5i87n5Aox9jYLcZwa6HJ3ElHMOa+n9AY4/+H8bd+BiHWTgEhEzcZqcYwyP2S2 0X/e/m/+1XYs5tldKNZb7ruYRv6rNyUAF1H8EtYNaJpmGtXYurkMhWhEgeP9YB7svmkUN+JO og91tNhN1Wd10/JfKIytNcpXmW6zij0f3MJw/kdwIsmfSUMPaiEli+eB7nU0uLZWf4C3MWTT NmwNznEya5K9McH1Wc/lO9+oB+zRXFBUM/v9YaiyPZo0JcwSRdVYKvKteyqnL/lnx7vtkOnA EC/bcmMvlWLI+Q4Vw2cr2FKcIpJVwswZ5snFqgDr4O5JB88aEAzPFzyWWeBlVqXc0DbDu8jD YmG3yp/xn5UJQSRy6eUcXICNjJyIwekUCznRmhtGwkGFCFEZH/s2fQ7nETxZcuiE4meRnVQE 9lOafI5D+dlsG3SlyN1x0YvrPismep7PwA6FX3cDyz2iUUj4xICLvRLU6kq892KuFmv75pop VAZjJMQqc8BG3oN2YkDcO4NEuOT9/r9muk/WH5Mqcs2BJEG6+yiQ13uMS5TxXiPFp3vKRlq0 MFnm7YRZr5aK6B/WGLOHnRRb2OdAzUgsj4Qiyqvh8Ab+x9wjLwGePxlA1akrF2hQItfOwUsE QjDUdAEQAOHG4vdGxU3eH5hYDLYRsQP6ofoU36pV8iFEtZRJ833L5p9GP2xFUGVDH8yTdkdf QR1prsCJXA7sE/gYBf3k9lGicJQmYNo3uW9Ngz787BhiQJyW/JXcutyTt9b/AZmfJaDo1p0C 8IEtoG7wt4+giFwAJ1brTJtyxlKOGcjWiKh1/dTh13muXSOPcCmhNs4Zm0YNjrhW9nIn1iik lpMRJCCxY1RNcU2VZXfTqq63UTaIrZ1lgYXWilnTdpXt5UEDYBw8Ee6tpPfQflC02e8hbDeD JEP9MTM9pmmPOwZQXP36hTryakKt1Kpw3hgC+Yx9q4wwaZ4XIiWUgopT5mlI+LhnzCgO05YN NcPrbsr6Js34gC3odNicD+C1jSdOXCqAPZZNiVx0PBjRv+LbBZhUkjQJxidvXmrp55pLm+Ua IVl3E/HpFY8kTaJBHP7jvLp+W4J9tP64Ijk5Y9F0z93JwMspG671xuomFsRxUtyO6vldd7qH 1yVzDX7Dd0fAzMDOPQJW6zLiixCmA0McaZdeBXapMJDDoZAPY4pCbRyJJXe0tfv9ufzJrM8Z JHylONdBiIKWw0JldXkUvIGafl1JDOHjP1XoDWrSDO8yFhBR3uWxJy9u1s7aKvonQb5IcYU1 nPu1Olg3doPugXyC0V05MIa68iKw+Kv8KtDDWyibndoTAAYpwsFfBBgBAgAJBQJCMNR1AhsM AAoJELyJZYjffCjuelUP/jlCsxLzu3fZpuORY2LsOQMd4nFHSZLUjauLxDUn8jE//32IIJ0v QV9ab4k7JCLOuYJTTd9aYD6rkITZIVhAcsR/FQZNgVOvGTj6tAmNyn385vMz0p4bLOOy5T0C KMLKzzS4Rt4XgtzvH2xDXSHfPsqS/t/5WFkO+aLgcPALldWGQPgRu5DNoCLr989gCGu5vmd4 XwMRBt/LmJGI0v0EypL3eRmlGaUw5k6N1hStu4EETzdikAzXP5KTuloEXq/caYeUs/SIb5zi XVC1ISW0CIwj5ATbMh8DMG4splXCsajtnJjsKJATBZIWV4XoNqtgV+pQn1ShmW36nUfVGqzX AQ+9i/M+CCkxBrb85Bk8I1CA1nBHNk5SQqER40VRp6vcmuxvIBGi6t8dDWsDQ2q3kd4RjjDZ kYjSie7176bb9t5MfUGjA9WckHuyi+vjy3+sC/nRzByhXf+8iZsO2no3xWZkGUWI8F2hhpzW VsXqvC27LZvJk53fJbpuSueN8a7JKfbKPDqoDSsRaEtcM7ig475tqA/ZCzv6mdqhEV5buoLu cpW7UgYzjNQQXeYZygGWc7FTV3dqLmF1MY2+RlydQbUDjcj1CJ+UmKyxgoLyf7ru0sznr7Tp K4WDnVeJdWX1mqoSupF/u5LON1vpzh3OIl5NNAuV68Hb5On/ALC+DwFX In-Reply-To: <22E46BE9-4101-4EBB-8D9B-C17736BBC5B4@cl.cam.ac.uk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Dear Anil, dear Pierre, I appreciate to move forward to use bytes/string where possible - mainly guided by: - nicer tooling support (statmemprof) [lots of tooling doesn't deal with externally allocated memory well (doesn't take it into account, GC doesn't nicely work with us when it comes to high memory pressure, have a look at the hacks for qubes-mirage-firewall, now https://github.com/mirage/mirage-qubes/blob/v1.0.0/lib/misc.ml#L2) - finally we've time to think about "immutability" of data, and rethink the allocation disciplines (which leads to increased performance) -- as Sudha said in her FunOCaml (OCaml 5 GC) talk, the best thing to do is to allocate less - Cstruct was a pretty nice abstraction, but has two layers of bounds checks, is an external library that is not too easy to comprehend (I've heard several people complaining about the mirage-crypto interface before) - OCaml standard library moved to have get_uint8, set_uint16_be etc. accessors since 4.08, so we can just use them \o/ - Bigarray allocation is expensive (calls to malloc, and esp. in MirageOS we then use dlmalloc which is pretty slow) Now, there are certainly difficulties when porting code: - Cstruct.shift and Cstruct.sub were cheap, now a String.sub is expensive (allocates the whole data thing) - In e.g. ethernet, we have `val write: t -> ?src:Macaddr.t -> Macaddr.t -> Packet.proto -> ?size:int -> (Cstruct.t -> int) -> (unit, error) result Lwt.t` And here, the "fill" function (Cstruct.t -> int) is around to allow the lowest layer to allocate the buffer, and then pass the buffer to-be-send onto the upper layer -- but ethernet shifts the start by 14 bytes to have space for the ethernet header. --> Replacing this with "bytes -> int" won't pan out, since we'd need some sort of "bytes -> off:int -> int" and ensure (by code review?) that "off" is always used. A similar argument applies for the receiving side. ===> I'm still undecided whether we should give up on that defensive coding practise, or have a fresh library that encapsulates the offset and the underlying data (then again, shift would be cheap). What Anil says is for sure true, we have Xen supported (with the amazing qubes-mirage-firewall, I guess still a very widely used unikernel), which requires non-moving page-aligned data. All other backends don't need page-alignment, neither non-moving since they copy the data anyways. And of course we want to provide an IP stack that allows zero-copying on the receive and send path. Now, a quick idea: - what is the difference between bytes and bigarray, in terms of memory representation (of the underlying data)? I welcome any insight you may have. - if it is equal, can't we come up with a library that unifies the access to the data? would something as simple (and scary) as "let data : bytes = Obj.magic cstruct.underlying_bigarray.underlying_memory" work? - can we have a compile-time choice selecting the right type for the underlying data (and thus the right functions are selected)? It is also related to bounds checks (and how to avoid them using types or other optimizations), take a look into https://discuss.ocaml.org/t/bounds-checks-for-string-and-bytes-when-retrieving-or-setting-subparts-thereof/ if you're interested. Best, Hannes On 01/10/2024 17:26, Anil Madhavapeddy wrote: > Dear Pierre, > > While this is a good step forward, it is worth considering how to deal with the requirements of some backends for non-moving IO pages to pass through to the kernel/unikernel. This interface will force a copy, but perhaps that's ok. It should be possible to make OCaml 5 bytes non-moving with some consideration for the garbage collector, but I haven't had a chance to think through the details. > > best, > Anil > > -- > Anil Madhavapeddy, Professor of Planetary Computing > Computer Laboratory, University of Cambridge (cst.cam.ac.uk) > >> On Oct 1, 2024, at 4:02 PM, pierre.alain@tuta.io wrote: >> >> Dear all, >> >> Over the past few months, a large amount of work has been carried out to reduce the use of Cstruct.t. This was mainly driven by performances, as there were a lot of expensive allocations for small buffers (from 0 to a couple of bytes). Indeed, there were big performance improvements with mirage-crypto (which weren't limited to extracting Cstruct, but also algorithm improvements and a deeper thinking about allocations, like allocating a buffer once, and writing to it rather than allocating multiple buffers and concatenating them). >> >> Now I'm wondering if this could be a time to extend this work of Cstruct removal with another API breakage. You can see at https://github.com/mirage/mirage-net/pull/25 a proposal for the new API. It's still conservative in term of "where each buffers would be allocated". Of course, if it is merged, there will be the need for lot of PR to make dependent libraries compliant with the new interface. >> >> That's why I'm happy to ask your opinion on this possible step forward :) >> Best, >> Pierre >> >> -- >> P. >> > > From mirageos-devel-bounces@lists.xenproject.org Wed Oct 09 15:18:47 2024 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 09 Oct 2024 15:18:47 +0000 Received: from list by lists.xenproject.org with outflank-mailman.814804.1228482 (Exim 4.92) (envelope-from ) id 1syYSR-0003mr-Cb; Wed, 09 Oct 2024 15:18:35 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 814804.1228482; Wed, 09 Oct 2024 15:18:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1syYSR-0003mk-9p; Wed, 09 Oct 2024 15:18:35 +0000 Received: by outflank-mailman (input) for mailman id 814804; Wed, 09 Oct 2024 15:18:33 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1syYSP-0003lO-GS for mirageos-devel@lists.xenproject.org; Wed, 09 Oct 2024 15:18:33 +0000 Received: from mail.w13.tutanota.de (mail.w13.tutanota.de [185.205.69.213]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id bdb7df24-8651-11ef-99a2-01e77a169b0f; Wed, 09 Oct 2024 17:18:31 +0200 (CEST) Received: from tutadb.w10.tutanota.de (w10.api.tuta.com [IPv6:fd:ac::d:10]) by mail.w13.tutanota.de (Postfix) with ESMTP id AA0F22AFC6B9; Wed, 9 Oct 2024 17:17:59 +0200 (CEST) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: bdb7df24-8651-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1728487079; s=s1; d=tuta.io; h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Content-Transfer-Encoding:Cc:Cc:Date:Date:In-Reply-To:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:References:Sender; bh=KR2MHt5bJdqXX946ziPQRfk2akY7069doXWrBCIS3Bg=; b=a3e7RzYJ0HFzgcTUBzUqSgxKyqgpTd0qxzJPXkiJDzFKCZ6Mf6KwSFD7H7z0X85x yeWuM1PDOI9eJXlQOHeowEOjAgw1frDjv6L65ZSjzUwdCJ/vI+cd/BBPdGio5sPrXen CXcmmlTlQhKEMO5DOYs4iKcW8BStdpUUefMkbh4vPp+SHr1RJtMXvFqFbAVu3/GUTbe JpGEViIoflf0kZd1g5emy1i67RA/Lgc2E3AaYCUDPcEnjzALT4UwKN/VshhjS1sbjX0 ddldPFSOBbOPhGkV6fYzG84shK5+PRrJ2mVTFno1s4q2U9WT9L3atwdgO6qUP3IDEM5 Wcg12uYZBA== Date: Wed, 9 Oct 2024 17:17:59 +0200 (CEST) From: pierre.alain@tuta.io To: Hannes Mehnert Cc: Mirageos Devel Message-ID: In-Reply-To: <3b89122d-03e1-4ad3-96c5-6f8091a30ae1@mehnert.org> References: <22E46BE9-4101-4EBB-8D9B-C17736BBC5B4@cl.cam.ac.uk> <3b89122d-03e1-4ad3-96c5-6f8091a30ae1@mehnert.org> Subject: Re: Remove Cstruct: one step forward? MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Dear Hannes, dear Anil, I'm sorry for the delay of my reply. I agree with keeping Xen&Qubes targets in mind and the need for io-page lib= rary around mirage-*-xen, but if io-page can be restricted to a mirage-*xen= dependency, it can be replaced in a later move when non moving bytes with = Ocaml 5 can be assured. The first impact, as has been pointed out, is that now it will force some c= opies, and I don't have any performance measurements yet: lots of dlmalloc = allocations for small buffers vs. more bytes copies.=C2=A0I've tried to pur= ge Cstruct of a small unikernel, but I've run out of free time :'( As for a new intermediate layer that uses bytes instead of bigarrays and al= lows for cheap split/sub/shift, I'm sure this will make code updating and m= aintenance easier, but will also lead to a new dependency that will have to= be understood by users, as was the case for Cstruct with mirage-crypto.=C2= =A0So I'm in the 'bytes+offsets' team in the lower functions API, and trust= the code review for the lowest part of the stack. Best, Pierre Oct 2, 2024, 10:56 by hannes@mehnert.org: > Dear Anil, dear Pierre, > > I appreciate to move forward to use bytes/string where possible - mainly = guided by: > - nicer tooling support (statmemprof) [lots of tooling doesn't deal with = externally allocated memory well (doesn't take it into account, GC doesn't = nicely work with us when it comes to high memory pressure, have a look at t= he hacks for qubes-mirage-firewall, now https://github.com/mirage/mirage-qu= bes/blob/v1.0.0/lib/misc.ml#L2) > - finally we've time to think about "immutability" of data, and rethink t= he allocation disciplines (which leads to increased performance) -- as Sudh= a said in her FunOCaml (OCaml 5 GC) talk, the best thing to do is to alloca= te less > - Cstruct was a pretty nice abstraction, but has two layers of bounds che= cks, is an external library that is not too easy to comprehend (I've heard = several people complaining about the mirage-crypto interface before) > - OCaml standard library moved to have get_uint8, set_uint16_be etc. acce= ssors since 4.08, so we can just use them \o/ > - Bigarray allocation is expensive (calls to malloc, and esp. in MirageOS= we then use dlmalloc which is pretty slow) > > Now, there are certainly difficulties when porting code: > - Cstruct.shift and Cstruct.sub were cheap, now a String.sub is expensive= (allocates the whole data thing) > - In e.g. ethernet, we have `val write: t -> ?src:Macaddr.t -> Macaddr.t = -> Packet.proto -> ?size:int -> (Cstruct.t -> int) -> (unit, error) result = Lwt.t` > And here, the "fill" function (Cstruct.t -> int) is around to allow the = lowest layer to allocate the buffer, and then pass the buffer to-be-send on= to the upper layer -- but ethernet shifts the start by 14 bytes to have spa= ce for the ethernet header. > --> Replacing this with "bytes -> int" won't pan out, since we'd need som= e sort of "bytes -> off:int -> int" and ensure (by code review?) that "off"= is always used. A similar argument applies for the receiving side. > =3D=3D=3D> I'm still undecided whether we should give up on that defensiv= e coding practise, or have a fresh library that encapsulates the offset and= the underlying data (then again, shift would be cheap). > > What Anil says is for sure true, we have Xen supported (with the amazing = qubes-mirage-firewall, I guess still a very widely used unikernel), which r= equires non-moving page-aligned data. All other backends don't need page-al= ignment, neither non-moving since they copy the data anyways. And of course= we want to provide an IP stack that allows zero-copying on the receive and= send path. > > Now, a quick idea: > - what is the difference between bytes and bigarray, in terms of memory r= epresentation (of the underlying data)? I welcome any insight you may have. > - if it is equal, can't we come up with a library that unifies the access= to the data? would something as simple (and scary) as "let data : bytes = =3D Obj.magic cstruct.underlying_bigarray.underlying_memory" work? > - can we have a compile-time choice selecting the right type for the unde= rlying data (and thus the right functions are selected)? > > It is also related to bounds checks (and how to avoid them using types or= other optimizations), take a look into https://discuss.ocaml.org/t/bounds-= checks-for-string-and-bytes-when-retrieving-or-setting-subparts-thereof/ if= you're interested. > > > > Best, > > Hannes > > On 01/10/2024 17:26, Anil Madhavapeddy wrote: > >> Dear Pierre, >> >> While this is a good step forward, it is worth considering how to deal w= ith the requirements of some backends for non-moving IO pages to pass throu= gh to the kernel/unikernel. This interface will force a copy, but perhaps t= hat's ok. It should be possible to make OCaml 5 bytes non-moving with some = consideration for the garbage collector, but I haven't had a chance to thin= k through the details. >> >> best, >> Anil >> >> -- >> Anil Madhavapeddy, Professor of Planetary Computing >> Computer Laboratory, University of Cambridge (cst.cam.ac.uk) >> >>> On Oct 1, 2024, at 4:02=E2=80=AFPM, pierre.alain@tuta.io wrote: >>> >>> Dear all, >>> >>> Over the past few months, a large amount of work has been carried out t= o reduce the use of Cstruct.t. This was mainly driven by performances, as t= here were a lot of expensive allocations for small buffers (from 0 to a cou= ple of bytes). Indeed, there were big performance improvements with mirage-= crypto (which weren't limited to extracting Cstruct, but also algorithm imp= rovements and a deeper thinking about allocations, like allocating a buffer= once, and writing to it rather than allocating multiple buffers and concat= enating them). >>> >>> Now I'm wondering if this could be a time to extend this work of Cstruc= t removal with another API breakage. You can see at https://github.com/mira= ge/mirage-net/pull/25 a proposal for the new API. It's still conservative i= n term of "where each buffers would be allocated". Of course, if it is merg= ed, there will be the need for lot of PR to make dependent libraries compli= ant with the new interface. >>> >>> That's why I'm happy to ask your opinion on this possible step forward = :) >>> Best, >>> Pierre >>> >>> -- >>> P. >>> >> >> > > From mirageos-devel-bounces@lists.xenproject.org Wed Oct 09 19:16:42 2024 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 09 Oct 2024 19:16:42 +0000 Received: from list by lists.xenproject.org with outflank-mailman.815114.1228838 (Exim 4.92) (envelope-from ) id 1sycAg-0005JO-1Y; Wed, 09 Oct 2024 19:16:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 815114.1228838; Wed, 09 Oct 2024 19:16:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sycAf-0005JH-Uf; Wed, 09 Oct 2024 19:16:29 +0000 Received: by outflank-mailman (input) for mailman id 815114; Wed, 09 Oct 2024 19:16:28 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sycAd-0005JB-TV for mirageos-devel@lists.xenproject.org; Wed, 09 Oct 2024 19:16:28 +0000 Received: from mail.mehnert.org (mail.mehnert.org [213.73.89.200]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id fa839ff0-8672-11ef-a0bd-8be0dac302b0; Wed, 09 Oct 2024 21:16:26 +0200 (CEST) Received: from [192.168.42.80] (i5C74C28B.versanet.de [92.116.194.139]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "hannes@mehnert.org", Issuer "mehnert root CA" (not verified)) by mail.mehnert.org (Postfix) with ESMTPS id 25B4A2460C for ; Wed, 9 Oct 2024 21:16:25 +0200 (CEST) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: fa839ff0-8672-11ef-a0bd-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mehnert.org; s=mail; t=1728501385; bh=iXioFzSYizWNYZ+g30eCf5Z0TAoq80TpQnH5821lZ8g=; h=Date:Subject:To:References:From:In-Reply-To; b=Irl12iq7CgGw+Y/6oxeQx65KgazAkdI1GdcaGl3eDJiYXQl2z7WNVdnD/dZujYubX E/4kWNJiUFtrNiBfs8CPJXpxNl3Fn1Ko76XdliOeyxCRE64c14Mm+96WxW1hztWrIM MYdSjV6R4N2NrAQ5fqb3KOlCwM+8D4t0/E18U1dw9s/q9Ks8THuFd0Tp1L43gbLpve 6K5VmUgGjj2Zzt7DAY3gUDs1zl6UOY5H57WrY24fKILPkL7FO5Bq2JmKqUfE6wBAM4 qfF1lh9OvmS5F/NWeNECjDls3pMe4eKRtosYrPoxfZB799F+K664MwRjN4SZ8+EYa+ g3Wd3AFZdFKZw== Message-ID: Date: Wed, 9 Oct 2024 21:16:24 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [Request For Comments] Defunctorising MirageOS Content-Language: en-US To: mirageos-devel@lists.xenproject.org References: <8916395b-7f89-44b1-850d-b587f91817f5@mehnert.org> From: Hannes Mehnert Autocrypt: addr=hannes@mehnert.org; keydata= xsFLBEIw1AoBEADAtXwEV8F1DBpE9lnBTbHDNeZwDVp84MhxxIT5GUexGgbOWGSEWHhC3rYe FfGRUxF4M9P4fwxpxCS5YCvxoijWHeEf8nG5IkztVv5cw63E443XWHcCMc80YAwglZ2cSP4U GTNeKb9rqVPckk/PL348BYRawhzvZK+Bc+bUvbtPCfUXT1BWIxAR1dzsfpAQVNZ4bA06xOoP QJYVNgl/lWOmQgnSgb0dE2zsgddKTOj05ru7Q7LobB7WAUTRJVkZcXnrvI1SOt/WbPTyqF8l RBh94xCqFhv4SlqZVOTXxo9gw3LpDv/cYXRl/m7+/7Wljl3ziQ9cawA6O1mbw8nm7Sfa+TZl qo+5lXEenXG+MCbH0XnnL2I4BO6HSGDtKX6htTG2xs6w4r9mVxTGJuJcGrC0dxuz5j4jylt/ KOVn9IaRKzhj8ga7kWffMp+JYdrn43732weoFFJxm78mD2ij4UbJtNkQIIcTv8IBJajHy2P3 h1NuBIwwb7RmBav4oo0CKWoasIHFwjMSBpCzJ8QOHeO/F3TY3DZp7FTwViUgSXVJoewO9yFG ctX7MC27/F1IonU9/SJW0j+F3Vz32SfxUBrDnLYpO7/vwA8w+xmWLnl0iJN/8injz5+CigsP e7O66t4MtC9BVCuLu7a/ikH5nW0q6RyTW8of9eZIsuEyqF1ZPwAGKc0jSGFubmVzIE1laG5l cnQgPGhhbm5lc0BtZWhuZXJ0Lm9yZz7CwXQEEwECAB4FAkIw1A0CGwMGCwkIBwMCAxUCAwMW AgECHgECF4AACgkQvIlliN98KO5HYg//UD6gk4sFcNop/EQivcnpfPnHrrUddsBl9bovQSXb zIh5HY/8xhO5i87n5Aox9jYLcZwa6HJ3ElHMOa+n9AY4/+H8bd+BiHWTgEhEzcZqcYwyP2S2 0X/e/m/+1XYs5tldKNZb7ruYRv6rNyUAF1H8EtYNaJpmGtXYurkMhWhEgeP9YB7svmkUN+JO og91tNhN1Wd10/JfKIytNcpXmW6zij0f3MJw/kdwIsmfSUMPaiEli+eB7nU0uLZWf4C3MWTT NmwNznEya5K9McH1Wc/lO9+oB+zRXFBUM/v9YaiyPZo0JcwSRdVYKvKteyqnL/lnx7vtkOnA EC/bcmMvlWLI+Q4Vw2cr2FKcIpJVwswZ5snFqgDr4O5JB88aEAzPFzyWWeBlVqXc0DbDu8jD YmG3yp/xn5UJQSRy6eUcXICNjJyIwekUCznRmhtGwkGFCFEZH/s2fQ7nETxZcuiE4meRnVQE 9lOafI5D+dlsG3SlyN1x0YvrPismep7PwA6FX3cDyz2iUUj4xICLvRLU6kq892KuFmv75pop VAZjJMQqc8BG3oN2YkDcO4NEuOT9/r9muk/WH5Mqcs2BJEG6+yiQ13uMS5TxXiPFp3vKRlq0 MFnm7YRZr5aK6B/WGLOHnRRb2OdAzUgsj4Qiyqvh8Ab+x9wjLwGePxlA1akrF2hQItfOwUsE QjDUdAEQAOHG4vdGxU3eH5hYDLYRsQP6ofoU36pV8iFEtZRJ833L5p9GP2xFUGVDH8yTdkdf QR1prsCJXA7sE/gYBf3k9lGicJQmYNo3uW9Ngz787BhiQJyW/JXcutyTt9b/AZmfJaDo1p0C 8IEtoG7wt4+giFwAJ1brTJtyxlKOGcjWiKh1/dTh13muXSOPcCmhNs4Zm0YNjrhW9nIn1iik lpMRJCCxY1RNcU2VZXfTqq63UTaIrZ1lgYXWilnTdpXt5UEDYBw8Ee6tpPfQflC02e8hbDeD JEP9MTM9pmmPOwZQXP36hTryakKt1Kpw3hgC+Yx9q4wwaZ4XIiWUgopT5mlI+LhnzCgO05YN NcPrbsr6Js34gC3odNicD+C1jSdOXCqAPZZNiVx0PBjRv+LbBZhUkjQJxidvXmrp55pLm+Ua IVl3E/HpFY8kTaJBHP7jvLp+W4J9tP64Ijk5Y9F0z93JwMspG671xuomFsRxUtyO6vldd7qH 1yVzDX7Dd0fAzMDOPQJW6zLiixCmA0McaZdeBXapMJDDoZAPY4pCbRyJJXe0tfv9ufzJrM8Z JHylONdBiIKWw0JldXkUvIGafl1JDOHjP1XoDWrSDO8yFhBR3uWxJy9u1s7aKvonQb5IcYU1 nPu1Olg3doPugXyC0V05MIa68iKw+Kv8KtDDWyibndoTAAYpwsFfBBgBAgAJBQJCMNR1AhsM AAoJELyJZYjffCjuelUP/jlCsxLzu3fZpuORY2LsOQMd4nFHSZLUjauLxDUn8jE//32IIJ0v QV9ab4k7JCLOuYJTTd9aYD6rkITZIVhAcsR/FQZNgVOvGTj6tAmNyn385vMz0p4bLOOy5T0C KMLKzzS4Rt4XgtzvH2xDXSHfPsqS/t/5WFkO+aLgcPALldWGQPgRu5DNoCLr989gCGu5vmd4 XwMRBt/LmJGI0v0EypL3eRmlGaUw5k6N1hStu4EETzdikAzXP5KTuloEXq/caYeUs/SIb5zi XVC1ISW0CIwj5ATbMh8DMG4splXCsajtnJjsKJATBZIWV4XoNqtgV+pQn1ShmW36nUfVGqzX AQ+9i/M+CCkxBrb85Bk8I1CA1nBHNk5SQqER40VRp6vcmuxvIBGi6t8dDWsDQ2q3kd4RjjDZ kYjSie7176bb9t5MfUGjA9WckHuyi+vjy3+sC/nRzByhXf+8iZsO2no3xWZkGUWI8F2hhpzW VsXqvC27LZvJk53fJbpuSueN8a7JKfbKPDqoDSsRaEtcM7ig475tqA/ZCzv6mdqhEV5buoLu cpW7UgYzjNQQXeYZygGWc7FTV3dqLmF1MY2+RlydQbUDjcj1CJ+UmKyxgoLyf7ru0sznr7Tp K4WDnVeJdWX1mqoSupF/u5LON1vpzh3OIl5NNAuV68Hb5On/ALC+DwFX In-Reply-To: <8916395b-7f89-44b1-850d-b587f91817f5@mehnert.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Dear MirageOS developers, more discussion on the issue tracker lead me to a revised approach. I prepared a showcase for the changes to mirage itself at https://github.com/mirage/mirage/pull/1579 and would be happy to get some feedback. (Earlier discussions in https://github.com/mirage/mirage/issues/1513 and https://github.com/mirage/mirage/pull/1521 and https://github.com/mirage/mirage/pull/1529 in case you're interested.) The observation is: we only want a single (posix) clock, timer, (monotonic wall-clock) time, and random number generator in a unikernel. Now, we can solve this with dune-variants \o/ -- that's the good news. This also means we don't need to functorize every "device" with these resources, but we can just plug the right thing in when "Mirage.register" is called (you can develop your own clock, and push it into your unikernel - it only needs to stick to the existing interface). The consumer libraries (e.g. dns) can just depend on "mirage-clock" and use the available interface -- without knowing the implementation. So, a good start to read the changes is https://github.com/mirage/mirage/pull/1579/files#diff-79c838533b44a0761156ff031b0018fcc5f0e3837463a67484468a19c073756e If nobody stops me, I'll push forward and soon [tm] have a set of packages to use with this PR and updates to mirage-skeleton (so you can have a deeper look). Thanks for reading and your valuable feedback (either here, or via personal communication or on the issue tracker). Have a nice evening, Hannes From mirageos-devel-bounces@lists.xenproject.org Tue Oct 22 10:54:26 2024 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 22 Oct 2024 10:54:26 +0000 Received: from list by lists.xenproject.org with outflank-mailman.824094.1238184 (Exim 4.92) (envelope-from ) id 1t3CWk-00031j-It; Tue, 22 Oct 2024 10:54:14 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 824094.1238184; Tue, 22 Oct 2024 10:54:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t3CWk-00031c-FZ; Tue, 22 Oct 2024 10:54:14 +0000 Received: by outflank-mailman (input) for mailman id 824094; Tue, 22 Oct 2024 10:54:12 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t3CWi-00031U-5h for mirageos-devel@lists.xenproject.org; Tue, 22 Oct 2024 10:54:12 +0000 Received: from mail.mehnert.org (mail.mehnert.org [213.73.89.200]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f7374ebd-9063-11ef-99a3-01e77a169b0f; Tue, 22 Oct 2024 12:54:09 +0200 (CEST) Received: from [192.168.42.80] (i5C74C1AC.versanet.de [92.116.193.172]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "hannes@mehnert.org", Issuer "mehnert root CA" (not verified)) by mail.mehnert.org (Postfix) with ESMTPS id 95F1A20ED for ; Tue, 22 Oct 2024 12:54:08 +0200 (CEST) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: f7374ebd-9063-11ef-99a3-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mehnert.org; s=mail; t=1729594448; bh=YxSep3Xv/ZIG8C/FdWs4TNIElrpMPyAUiNrsqmBGR38=; h=Date:To:From:Subject; b=UcPofv6Oh5YNsxzXlj27p5cG5iqni+1oUTKm3twEedcy4hWT/EeDu6SZ2TP0Igb99 38swP75abRtRzME6/11DscTf1ah5mPBw9v3j7hvnvtfQ9nn5i5EFP2g0D5XkGMA7DM RVPCp93LakNiVQHt+IAiGmiYaf3HEO0lcMvFz/4R0xGB0mbfUsBdFmCwFKIunk8e3e M/b+eg/Fok3aiYY8okqMKO+KIaIqf82yCk4Erl759On1Nr34M8tZk20DpH3HnEuZr4 Q/jWt0Esnb+xIrisI6GxtfG074cuxv0fPxqdOg/Tl1u4lUAYE9NAtzvq9ZsVwf5oLt 02T+l/k81dlgA== Message-ID: Date: Tue, 22 Oct 2024 12:54:02 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: mirageos-devel@lists.xenproject.org Content-Language: en-US From: Hannes Mehnert Subject: A decade of Hello World Autocrypt: addr=hannes@mehnert.org; keydata= xsFLBEIw1AoBEADAtXwEV8F1DBpE9lnBTbHDNeZwDVp84MhxxIT5GUexGgbOWGSEWHhC3rYe FfGRUxF4M9P4fwxpxCS5YCvxoijWHeEf8nG5IkztVv5cw63E443XWHcCMc80YAwglZ2cSP4U GTNeKb9rqVPckk/PL348BYRawhzvZK+Bc+bUvbtPCfUXT1BWIxAR1dzsfpAQVNZ4bA06xOoP QJYVNgl/lWOmQgnSgb0dE2zsgddKTOj05ru7Q7LobB7WAUTRJVkZcXnrvI1SOt/WbPTyqF8l RBh94xCqFhv4SlqZVOTXxo9gw3LpDv/cYXRl/m7+/7Wljl3ziQ9cawA6O1mbw8nm7Sfa+TZl qo+5lXEenXG+MCbH0XnnL2I4BO6HSGDtKX6htTG2xs6w4r9mVxTGJuJcGrC0dxuz5j4jylt/ KOVn9IaRKzhj8ga7kWffMp+JYdrn43732weoFFJxm78mD2ij4UbJtNkQIIcTv8IBJajHy2P3 h1NuBIwwb7RmBav4oo0CKWoasIHFwjMSBpCzJ8QOHeO/F3TY3DZp7FTwViUgSXVJoewO9yFG ctX7MC27/F1IonU9/SJW0j+F3Vz32SfxUBrDnLYpO7/vwA8w+xmWLnl0iJN/8injz5+CigsP e7O66t4MtC9BVCuLu7a/ikH5nW0q6RyTW8of9eZIsuEyqF1ZPwAGKc0jSGFubmVzIE1laG5l cnQgPGhhbm5lc0BtZWhuZXJ0Lm9yZz7CwXQEEwECAB4FAkIw1A0CGwMGCwkIBwMCAxUCAwMW AgECHgECF4AACgkQvIlliN98KO5HYg//UD6gk4sFcNop/EQivcnpfPnHrrUddsBl9bovQSXb zIh5HY/8xhO5i87n5Aox9jYLcZwa6HJ3ElHMOa+n9AY4/+H8bd+BiHWTgEhEzcZqcYwyP2S2 0X/e/m/+1XYs5tldKNZb7ruYRv6rNyUAF1H8EtYNaJpmGtXYurkMhWhEgeP9YB7svmkUN+JO og91tNhN1Wd10/JfKIytNcpXmW6zij0f3MJw/kdwIsmfSUMPaiEli+eB7nU0uLZWf4C3MWTT NmwNznEya5K9McH1Wc/lO9+oB+zRXFBUM/v9YaiyPZo0JcwSRdVYKvKteyqnL/lnx7vtkOnA EC/bcmMvlWLI+Q4Vw2cr2FKcIpJVwswZ5snFqgDr4O5JB88aEAzPFzyWWeBlVqXc0DbDu8jD YmG3yp/xn5UJQSRy6eUcXICNjJyIwekUCznRmhtGwkGFCFEZH/s2fQ7nETxZcuiE4meRnVQE 9lOafI5D+dlsG3SlyN1x0YvrPismep7PwA6FX3cDyz2iUUj4xICLvRLU6kq892KuFmv75pop VAZjJMQqc8BG3oN2YkDcO4NEuOT9/r9muk/WH5Mqcs2BJEG6+yiQ13uMS5TxXiPFp3vKRlq0 MFnm7YRZr5aK6B/WGLOHnRRb2OdAzUgsj4Qiyqvh8Ab+x9wjLwGePxlA1akrF2hQItfOwUsE QjDUdAEQAOHG4vdGxU3eH5hYDLYRsQP6ofoU36pV8iFEtZRJ833L5p9GP2xFUGVDH8yTdkdf QR1prsCJXA7sE/gYBf3k9lGicJQmYNo3uW9Ngz787BhiQJyW/JXcutyTt9b/AZmfJaDo1p0C 8IEtoG7wt4+giFwAJ1brTJtyxlKOGcjWiKh1/dTh13muXSOPcCmhNs4Zm0YNjrhW9nIn1iik lpMRJCCxY1RNcU2VZXfTqq63UTaIrZ1lgYXWilnTdpXt5UEDYBw8Ee6tpPfQflC02e8hbDeD JEP9MTM9pmmPOwZQXP36hTryakKt1Kpw3hgC+Yx9q4wwaZ4XIiWUgopT5mlI+LhnzCgO05YN NcPrbsr6Js34gC3odNicD+C1jSdOXCqAPZZNiVx0PBjRv+LbBZhUkjQJxidvXmrp55pLm+Ua IVl3E/HpFY8kTaJBHP7jvLp+W4J9tP64Ijk5Y9F0z93JwMspG671xuomFsRxUtyO6vldd7qH 1yVzDX7Dd0fAzMDOPQJW6zLiixCmA0McaZdeBXapMJDDoZAPY4pCbRyJJXe0tfv9ufzJrM8Z JHylONdBiIKWw0JldXkUvIGafl1JDOHjP1XoDWrSDO8yFhBR3uWxJy9u1s7aKvonQb5IcYU1 nPu1Olg3doPugXyC0V05MIa68iKw+Kv8KtDDWyibndoTAAYpwsFfBBgBAgAJBQJCMNR1AhsM AAoJELyJZYjffCjuelUP/jlCsxLzu3fZpuORY2LsOQMd4nFHSZLUjauLxDUn8jE//32IIJ0v QV9ab4k7JCLOuYJTTd9aYD6rkITZIVhAcsR/FQZNgVOvGTj6tAmNyn385vMz0p4bLOOy5T0C KMLKzzS4Rt4XgtzvH2xDXSHfPsqS/t/5WFkO+aLgcPALldWGQPgRu5DNoCLr989gCGu5vmd4 XwMRBt/LmJGI0v0EypL3eRmlGaUw5k6N1hStu4EETzdikAzXP5KTuloEXq/caYeUs/SIb5zi XVC1ISW0CIwj5ATbMh8DMG4splXCsajtnJjsKJATBZIWV4XoNqtgV+pQn1ShmW36nUfVGqzX AQ+9i/M+CCkxBrb85Bk8I1CA1nBHNk5SQqER40VRp6vcmuxvIBGi6t8dDWsDQ2q3kd4RjjDZ kYjSie7176bb9t5MfUGjA9WckHuyi+vjy3+sC/nRzByhXf+8iZsO2no3xWZkGUWI8F2hhpzW VsXqvC27LZvJk53fJbpuSueN8a7JKfbKPDqoDSsRaEtcM7ig475tqA/ZCzv6mdqhEV5buoLu cpW7UgYzjNQQXeYZygGWc7FTV3dqLmF1MY2+RlydQbUDjcj1CJ+UmKyxgoLyf7ru0sznr7Tp K4WDnVeJdWX1mqoSupF/u5LON1vpzh3OIl5NNAuV68Hb5On/ALC+DwFX Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Dear valued MirageOS hacker, it is my pleasure - we just published a blog article on Mirage Runtime Arguments (https://blog.robur.coop/articles/arguments.html), which includes a (brief) history of Hello World and their changes. You can lean back and take 2 minutes to watch https://asciinema.org/a/681922 - the evolution of a decade of Hello World MirageOS unikernel. I'm sure there are earlier versions around that are missing -- if you have anything locally, don't hesitate to send them (preferably with a date and MirageOS release version) ;) Should we restart (bi?)weekly MirageOS meetings? My pleasure. Have a wonderful autumn day (at least in the Northern hemisphere), Hannes From mirageos-devel-bounces@lists.xenproject.org Tue Oct 22 11:11:08 2024 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 22 Oct 2024 11:11:08 +0000 Received: from list by lists.xenproject.org with outflank-mailman.824099.1238189 (Exim 4.92) (envelope-from ) id 1t3Cmz-0005qI-QY; Tue, 22 Oct 2024 11:11:01 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 824099.1238189; Tue, 22 Oct 2024 11:11:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t3Cmz-0005qB-N7; Tue, 22 Oct 2024 11:11:01 +0000 Received: by outflank-mailman (input) for mailman id 824099; Tue, 22 Oct 2024 11:11:00 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t3Cmx-0005q5-Js for mirageos-devel@lists.xenproject.org; Tue, 22 Oct 2024 11:11:00 +0000 Received: from mta3.cl.cam.ac.uk (mta3.cl.cam.ac.uk [2001:630:212:200::25:3]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4fdb10e6-9066-11ef-99a3-01e77a169b0f; Tue, 22 Oct 2024 13:10:57 +0200 (CEST) Received: from fauth-a1-smtp.messagingengine.com ([103.168.172.200]:43321) (dnseec=no) by mta3.cl.cam.ac.uk:587 [128.232.25.23] with esmtpsa auth as avsm2+fm (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) id 1t3Cmu-000omt-UQ (envelope-from ); Tue, 22 Oct 2024 12:10:57 +0100 Received: from phl-compute-11.internal (phl-compute-11.phl.internal [10.202.2.51]) by mailfauth.phl.internal (Postfix) with ESMTP id 079151200043; Tue, 22 Oct 2024 07:10:52 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-11.internal (MEProxy); Tue, 22 Oct 2024 07:10:53 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 22 Oct 2024 07:10:52 -0400 (EDT) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: 4fdb10e6-9066-11ef-99a3-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=cl.cam.ac.uk; s=mta3; h=References:To:Cc:In-Reply-To:Date:Subject: Mime-Version:Content-Type:Message-Id:From:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=+IMMcfNrDuAP0EuNTj9qY6qw6lNAXy7WA4IIKLN/gM0=; t=1729595457; x=1730459457; b=LDJ+lhwmRkn5PerWlqAFfVmZLlmePyV6ahtpb8xrcALmVL58bvvJBnKiJJzqMGSjYhD4ShcuXBW kj47IX65spVUmLQyTC0TiMvgFJual9M+0UAvbwJ19cr8gzZ20OO78f17aPY0TT/dBmzNk+7eetQeT bK6B6RNwDJmHz0e/UmxK9Uhi4Cz0u7I82jL2kjHKXMIt5/p6YgqPL0VgWhXUO15uGXouOI7yhvxuG A4G2/5rSU6duEvejhl+pk4kH6H2VHOrr54/zrVJ/rzYBYmRuVotRfvSdXLsguRU9N/CMqrBoVlp64 qNG3+RR07xhwqWTVAeu5LXKrzYETaWKwGWrQ==; X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrvdeihedgfeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhephffktgggufffjgevvfhfofesrgdtmherhhdtjeen ucfhrhhomheptehnihhlucforgguhhgrvhgrphgvugguhicuoegrvhhsmhdvsegtlhdrtg grmhdrrggtrdhukheqnecuggftrfgrthhtvghrnhepfeetkeefudehvedvheejueejjeff ffdtjeffhfdukeduvedtjeeludekteeluefgnecuffhomhgrihhnpehgihhthhhusgdrtg homhdptghonhhfihhgrdhmlhdphhgrnhgulhgvrhdrmhhlpdhrvggtohhilhdrohhrghdp tggrmhdrrggtrdhukhdprhhosghurhdrtghoohhppdgrshgtihhinhgvmhgrrdhorhhgne cuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomheprghvshhm vddomhgvshhmthhprghuthhhphgvrhhsohhnrghlihhthidquddvhedvkeefudejfedqvd ekudeljeegieelqdgrvhhsmhdvpeeptghlrdgtrghmrdgrtgdruhhksehfmhdrtghlrdgt rghmrdgrtgdruhhkpdhnsggprhgtphhtthhopedvpdhmohguvgepshhmthhpohhuthdprh gtphhtthhopehhrghnnhgvshesmhgvhhhnvghrthdrohhrghdprhgtphhtthhopehmihhr rghgvghoshdquggvvhgvlheslhhishhtshdrgigvnhhprhhojhgvtghtrdhorhhg X-ME-Proxy: Feedback-ID: i9ebc46d9:Fastmail From: Anil Madhavapeddy Message-Id: <38A69F6C-C4F9-44C8-9C9D-88B43E1D8BF0@cl.cam.ac.uk> Content-Type: multipart/alternative; boundary="Apple-Mail=_565480E5-1561-4A4E-B673-BB2529C69CF9" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.200.67\)) Subject: Re: A decade of Hello World Date: Tue, 22 Oct 2024 12:10:41 +0100 In-Reply-To: Cc: mirageos-devel@lists.xenproject.org To: Hannes Mehnert References: X-Mailer: Apple Mail (2.3826.200.67) X-cl-SRS: mirageos-devel@lists.xenproject.org vs mirageos-devel@lists.xenproject.org --Apple-Mail=_565480E5-1561-4A4E-B673-BB2529C69CF9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii I love this post! I couldn't resist spelunking through the git history = back to MirageOS 1.0 (pre functoria), and spotted the Hello World from = there: = https://github.com/mirage/mirage/blob/e7906f49c462a74cb39f4403bec32a04c17a= 6398/lib_test/console/config.ml = https://github.com/mirage/mirage/blob/e7906f49c462a74cb39f4403bec32a04c17a= 6398/lib_test/console/handler.ml which looks pleasingly similar to the 2.0 hello world. I think we = separated out the Lwt signatures, and then re-merged them back in a few = releases later.=20 I'll see if I can dig out the artefacts from the original ASPLOS paper = (in about 2012), which was (at that point) a 150kB DNS unikernel. It = used bytecode and a dead code elimination patch against OCaml 4.01 to = really squeeze out the unnecessary bytes, but it was still pretty = performant as well. -- Anil Madhavapeddy, Professor of Planetary Computing (anil.recoil.org) Computer Laboratory, University of Cambridge (www.cst.cam.ac.uk) > On 22 Oct 2024, at 11:54, Hannes Mehnert wrote: >=20 > Dear valued MirageOS hacker, >=20 > it is my pleasure - we just published a blog article on Mirage Runtime = Arguments (https://blog.robur.coop/articles/arguments.html), which = includes a (brief) history of Hello World and their changes. >=20 > You can lean back and take 2 minutes to watch = https://asciinema.org/a/681922 - the evolution of a decade of Hello = World MirageOS unikernel. I'm sure there are earlier versions around = that are missing -- if you have anything locally, don't hesitate to send = them (preferably with a date and MirageOS release version) ;) >=20 > Should we restart (bi?)weekly MirageOS meetings? >=20 >=20 > My pleasure. Have a wonderful autumn day (at least in the Northern = hemisphere), >=20 > Hannes >=20 --Apple-Mail=_565480E5-1561-4A4E-B673-BB2529C69CF9 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii I love this = post! I couldn't resist spelunking through the git history back to = MirageOS 1.0 (pre functoria), and spotted the Hello World from = there:

https://github.com/mirage/mirage/= blob/e7906f49c462a74cb39f4403bec32a04c17a6398/lib_test/console/config.ml

which looks pleasingly similar to the 2.0 = hello world. I think we separated out the Lwt signatures, and then = re-merged them back in a few releases = later. 

I'll see if I can dig out the = artefacts from the original ASPLOS paper (in about 2012), which was (at = that point) a 150kB DNS unikernel. It used bytecode and a dead code = elimination patch against OCaml 4.01 to really squeeze out the = unnecessary bytes, but it was still pretty performant as = well.

--
Anil Madhavapeddy, Professor = of Planetary Computing (anil.recoil.org)
Computer Laboratory, = University of Cambridge = (www.cst.cam.ac.uk)


On 22 Oct 2024, at 11:54, Hannes = Mehnert <hannes@mehnert.org> wrote:

Dear valued MirageOS = hacker,

it is my pleasure - we just published a blog article on = Mirage Runtime Arguments = (https://blog.robur.coop/articles/arguments.html), which includes a = (brief) history of Hello World and their changes.

You can lean = back and take 2 minutes to watch https://asciinema.org/a/681922 - the = evolution of a decade of Hello World MirageOS unikernel. I'm sure there = are earlier versions around that are missing -- if you have anything = locally, don't hesitate to send them (preferably with a date and = MirageOS release version) ;)

Should we restart (bi?)weekly = MirageOS meetings?


My pleasure. Have a wonderful autumn day = (at least in the Northern = hemisphere),

Hannes


= --Apple-Mail=_565480E5-1561-4A4E-B673-BB2529C69CF9-- From mirageos-devel-bounces@lists.xenproject.org Tue Oct 22 11:24:15 2024 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 22 Oct 2024 11:24:15 +0000 Received: from list by lists.xenproject.org with outflank-mailman.824116.1238204 (Exim 4.92) (envelope-from ) id 1t3Czj-0007w7-3U; Tue, 22 Oct 2024 11:24:11 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 824116.1238204; Tue, 22 Oct 2024 11:24:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t3Czj-0007w0-0q; Tue, 22 Oct 2024 11:24:11 +0000 Received: by outflank-mailman (input) for mailman id 824116; Tue, 22 Oct 2024 11:24:09 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t3Czh-0007vt-SX for mirageos-devel@lists.xenproject.org; Tue, 22 Oct 2024 11:24:09 +0000 Received: from mail.mehnert.org (mail.mehnert.org [213.73.89.200]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 26e69f03-9068-11ef-99a3-01e77a169b0f; Tue, 22 Oct 2024 13:24:07 +0200 (CEST) Received: from [192.168.42.80] (i5C74C1AC.versanet.de [92.116.193.172]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "hannes@mehnert.org", Issuer "mehnert root CA" (not verified)) by mail.mehnert.org (Postfix) with ESMTPS id C70979C2 for ; Tue, 22 Oct 2024 13:24:06 +0200 (CEST) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: 26e69f03-9068-11ef-99a3-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mehnert.org; s=mail; t=1729596246; bh=ScUJ5eN+SQuRUMSgRDSnQ3qVP1ekGpp6iZJEisXFhUk=; h=Date:Subject:To:References:From:In-Reply-To; b=WBvGDeHrCjkmTKlKM5Uk/Qwhd/3VoZEFLEpwutd400TGVvVntWmUwPaG2GXwl7U8T KodSwmZ5bT0l/07ISN/Nd1Ibg16VNhrgBPzTPbOtEznV7/WxB0xMQM3rJ9yzlPsSfY 6/FjJFSw17AX6yriWVSMNtciPIBDAfubDwDrir0nzxrltkE2TMwGOQAEGuM1ctac9p WYFisJIyjhLJO3B0UWcHEIUHao4lScREg8Te1skJPOGiTg+oUV6/8xCFdThyAqh6jH 1S3SOieKPmIShi5We9FfPoJUIkr7XEhSES3kJNYxA4SjuUGBUOQlALF+zw6AEforUk 95muCiZzg1T5g== Message-ID: <760f8b88-d826-4f81-b052-44751cc29073@mehnert.org> Date: Tue, 22 Oct 2024 13:24:06 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: A decade of Hello World Content-Language: en-US To: mirageos-devel@lists.xenproject.org References: <38A69F6C-C4F9-44C8-9C9D-88B43E1D8BF0@cl.cam.ac.uk> From: Hannes Mehnert Autocrypt: addr=hannes@mehnert.org; keydata= xsFLBEIw1AoBEADAtXwEV8F1DBpE9lnBTbHDNeZwDVp84MhxxIT5GUexGgbOWGSEWHhC3rYe FfGRUxF4M9P4fwxpxCS5YCvxoijWHeEf8nG5IkztVv5cw63E443XWHcCMc80YAwglZ2cSP4U GTNeKb9rqVPckk/PL348BYRawhzvZK+Bc+bUvbtPCfUXT1BWIxAR1dzsfpAQVNZ4bA06xOoP QJYVNgl/lWOmQgnSgb0dE2zsgddKTOj05ru7Q7LobB7WAUTRJVkZcXnrvI1SOt/WbPTyqF8l RBh94xCqFhv4SlqZVOTXxo9gw3LpDv/cYXRl/m7+/7Wljl3ziQ9cawA6O1mbw8nm7Sfa+TZl qo+5lXEenXG+MCbH0XnnL2I4BO6HSGDtKX6htTG2xs6w4r9mVxTGJuJcGrC0dxuz5j4jylt/ KOVn9IaRKzhj8ga7kWffMp+JYdrn43732weoFFJxm78mD2ij4UbJtNkQIIcTv8IBJajHy2P3 h1NuBIwwb7RmBav4oo0CKWoasIHFwjMSBpCzJ8QOHeO/F3TY3DZp7FTwViUgSXVJoewO9yFG ctX7MC27/F1IonU9/SJW0j+F3Vz32SfxUBrDnLYpO7/vwA8w+xmWLnl0iJN/8injz5+CigsP e7O66t4MtC9BVCuLu7a/ikH5nW0q6RyTW8of9eZIsuEyqF1ZPwAGKc0jSGFubmVzIE1laG5l cnQgPGhhbm5lc0BtZWhuZXJ0Lm9yZz7CwXQEEwECAB4FAkIw1A0CGwMGCwkIBwMCAxUCAwMW AgECHgECF4AACgkQvIlliN98KO5HYg//UD6gk4sFcNop/EQivcnpfPnHrrUddsBl9bovQSXb zIh5HY/8xhO5i87n5Aox9jYLcZwa6HJ3ElHMOa+n9AY4/+H8bd+BiHWTgEhEzcZqcYwyP2S2 0X/e/m/+1XYs5tldKNZb7ruYRv6rNyUAF1H8EtYNaJpmGtXYurkMhWhEgeP9YB7svmkUN+JO og91tNhN1Wd10/JfKIytNcpXmW6zij0f3MJw/kdwIsmfSUMPaiEli+eB7nU0uLZWf4C3MWTT NmwNznEya5K9McH1Wc/lO9+oB+zRXFBUM/v9YaiyPZo0JcwSRdVYKvKteyqnL/lnx7vtkOnA EC/bcmMvlWLI+Q4Vw2cr2FKcIpJVwswZ5snFqgDr4O5JB88aEAzPFzyWWeBlVqXc0DbDu8jD YmG3yp/xn5UJQSRy6eUcXICNjJyIwekUCznRmhtGwkGFCFEZH/s2fQ7nETxZcuiE4meRnVQE 9lOafI5D+dlsG3SlyN1x0YvrPismep7PwA6FX3cDyz2iUUj4xICLvRLU6kq892KuFmv75pop VAZjJMQqc8BG3oN2YkDcO4NEuOT9/r9muk/WH5Mqcs2BJEG6+yiQ13uMS5TxXiPFp3vKRlq0 MFnm7YRZr5aK6B/WGLOHnRRb2OdAzUgsj4Qiyqvh8Ab+x9wjLwGePxlA1akrF2hQItfOwUsE QjDUdAEQAOHG4vdGxU3eH5hYDLYRsQP6ofoU36pV8iFEtZRJ833L5p9GP2xFUGVDH8yTdkdf QR1prsCJXA7sE/gYBf3k9lGicJQmYNo3uW9Ngz787BhiQJyW/JXcutyTt9b/AZmfJaDo1p0C 8IEtoG7wt4+giFwAJ1brTJtyxlKOGcjWiKh1/dTh13muXSOPcCmhNs4Zm0YNjrhW9nIn1iik lpMRJCCxY1RNcU2VZXfTqq63UTaIrZ1lgYXWilnTdpXt5UEDYBw8Ee6tpPfQflC02e8hbDeD JEP9MTM9pmmPOwZQXP36hTryakKt1Kpw3hgC+Yx9q4wwaZ4XIiWUgopT5mlI+LhnzCgO05YN NcPrbsr6Js34gC3odNicD+C1jSdOXCqAPZZNiVx0PBjRv+LbBZhUkjQJxidvXmrp55pLm+Ua IVl3E/HpFY8kTaJBHP7jvLp+W4J9tP64Ijk5Y9F0z93JwMspG671xuomFsRxUtyO6vldd7qH 1yVzDX7Dd0fAzMDOPQJW6zLiixCmA0McaZdeBXapMJDDoZAPY4pCbRyJJXe0tfv9ufzJrM8Z JHylONdBiIKWw0JldXkUvIGafl1JDOHjP1XoDWrSDO8yFhBR3uWxJy9u1s7aKvonQb5IcYU1 nPu1Olg3doPugXyC0V05MIa68iKw+Kv8KtDDWyibndoTAAYpwsFfBBgBAgAJBQJCMNR1AhsM AAoJELyJZYjffCjuelUP/jlCsxLzu3fZpuORY2LsOQMd4nFHSZLUjauLxDUn8jE//32IIJ0v QV9ab4k7JCLOuYJTTd9aYD6rkITZIVhAcsR/FQZNgVOvGTj6tAmNyn385vMz0p4bLOOy5T0C KMLKzzS4Rt4XgtzvH2xDXSHfPsqS/t/5WFkO+aLgcPALldWGQPgRu5DNoCLr989gCGu5vmd4 XwMRBt/LmJGI0v0EypL3eRmlGaUw5k6N1hStu4EETzdikAzXP5KTuloEXq/caYeUs/SIb5zi XVC1ISW0CIwj5ATbMh8DMG4splXCsajtnJjsKJATBZIWV4XoNqtgV+pQn1ShmW36nUfVGqzX AQ+9i/M+CCkxBrb85Bk8I1CA1nBHNk5SQqER40VRp6vcmuxvIBGi6t8dDWsDQ2q3kd4RjjDZ kYjSie7176bb9t5MfUGjA9WckHuyi+vjy3+sC/nRzByhXf+8iZsO2no3xWZkGUWI8F2hhpzW VsXqvC27LZvJk53fJbpuSueN8a7JKfbKPDqoDSsRaEtcM7ig475tqA/ZCzv6mdqhEV5buoLu cpW7UgYzjNQQXeYZygGWc7FTV3dqLmF1MY2+RlydQbUDjcj1CJ+UmKyxgoLyf7ru0sznr7Tp K4WDnVeJdWX1mqoSupF/u5LON1vpzh3OIl5NNAuV68Hb5On/ALC+DwFX In-Reply-To: <38A69F6C-C4F9-44C8-9C9D-88B43E1D8BF0@cl.cam.ac.uk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Great, thanks for your exhumation. Since I was focusing on runtime arguments, I discarded those hello world unikernels that are only printing hello world. ;) Hannes On 22/10/2024 13:10, Anil Madhavapeddy wrote: > I love this post! I couldn't resist spelunking through the git history back to MirageOS 1.0 (pre functoria), and spotted the Hello World from there: > > https://github.com/mirage/mirage/blob/e7906f49c462a74cb39f4403bec32a04c17a6398/lib_test/console/config.ml > https://github.com/mirage/mirage/blob/e7906f49c462a74cb39f4403bec32a04c17a6398/lib_test/console/handler.ml > > which looks pleasingly similar to the 2.0 hello world. I think we separated out the Lwt signatures, and then re-merged them back in a few releases later. > > I'll see if I can dig out the artefacts from the original ASPLOS paper (in about 2012), which was (at that point) a 150kB DNS unikernel. It used bytecode and a dead code elimination patch against OCaml 4.01 to really squeeze out the unnecessary bytes, but it was still pretty performant as well. > > -- > Anil Madhavapeddy, Professor of Planetary Computing (anil.recoil.org) > Computer Laboratory, University of Cambridge (www.cst.cam.ac.uk) > > >> On 22 Oct 2024, at 11:54, Hannes Mehnert wrote: >> >> Dear valued MirageOS hacker, >> >> it is my pleasure - we just published a blog article on Mirage Runtime Arguments (https://blog.robur.coop/articles/arguments.html), which includes a (brief) history of Hello World and their changes. >> >> You can lean back and take 2 minutes to watch https://asciinema.org/a/681922 - the evolution of a decade of Hello World MirageOS unikernel. I'm sure there are earlier versions around that are missing -- if you have anything locally, don't hesitate to send them (preferably with a date and MirageOS release version) ;) >> >> Should we restart (bi?)weekly MirageOS meetings? >> >> >> My pleasure. Have a wonderful autumn day (at least in the Northern hemisphere), >> >> Hannes >> > >