From mirageos-devel-bounces@lists.xenproject.org Thu Sep 01 12:49:46 2022 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Thu, 01 Sep 2022 12:49:46 +0000 Received: from list by lists.xenproject.org with outflank-mailman.396577.636761 (Exim 4.92) (envelope-from ) id 1oTjdW-0005fT-Oe; Thu, 01 Sep 2022 12:49:34 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 396577.636761; Thu, 01 Sep 2022 12:49:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTjdW-0005fM-Lz; Thu, 01 Sep 2022 12:49:34 +0000 Received: by outflank-mailman (input) for mailman id 396577; Thu, 01 Sep 2022 12:49:33 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTjdV-0005ex-HM for mirageos-devel@lists.xenproject.org; Thu, 01 Sep 2022 12:49:33 +0000 Received: from mail-yb1-xb31.google.com (mail-yb1-xb31.google.com [2607:f8b0:4864:20::b31]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 85a7f37c-29f4-11ed-82f2-63bd783d45fa; Thu, 01 Sep 2022 14:49:32 +0200 (CEST) Received: by mail-yb1-xb31.google.com with SMTP id y197so8441107yby.13 for ; Thu, 01 Sep 2022 05:49:31 -0700 (PDT) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: 85a7f37c-29f4-11ed-82f2-63bd783d45fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=CeeyL0x8XjcLcevQH/rR4IpQU8mKuMyWe8EN/QakKzE=; b=O9003/8jRVhjtn9iSixVeJIlVhGSd+XYOwX2ISC7ETHluk4e194f2SStaHDh1Do3Rk CyN8UmaQfxiytAo2FYPwy1kmE4egizhe0cAcdMy9Iw5hpKLzrKuDchn835fQkxBGa+to KMW1hVOIOJINtANFKHJlCgE1H8xxPipHilxYFjSQRwqkLXSpLLCXdbRIZluEjSng94PJ JmWY7sNhcCJmjUmnfvYQYkiG7eA8I3nBDi60/HyFpq8zY02qbD716U5IRdJv/67xZuPz hZlB98I9pvAQ2XaqEJp6KZXythQD7z6REaEvuBJe78BFMsHKy5ImiLeB0qNnLQXeWy5c PgJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=CeeyL0x8XjcLcevQH/rR4IpQU8mKuMyWe8EN/QakKzE=; b=zAho55TtIpYg1oBPtRo9wvhshxyLZq76fLEQ7PJy5P20roEh9v75U0UNGdtN1aOuJH hFXlA12Sn7pBixdFIuYHuQ2sOxjPpLscRZ2ddxsMViuo2KZh0QIvjrDvJ+0GKpLHJZyf z/FfaBhHxctXl4WlL7SmB/X3jwwV+KOuNBz2+YigdkAwwdr9l9trUWXnmIheUhtFj99R bmHLQqOoACEq56AxJiUEmCEBQJYBEN1wKdWWsJ+jRAdGsRoSo/VtfzBnu6kEATG8i0I0 DHaFrZyTPvqDG40v6WMeGgoqz1NZXh3evKIojOWXGSjYJLjJy2Q2HUECaDxsGOK7QHhV v47w== X-Gm-Message-State: ACgBeo3uC0Sfs3vXNX4dPe3dh0UD4RfxcQzg0Ve0gehJLTjCyLp752tD U0wgyCEnHfjYMxQubRh9daeuByXkHsbjO4HzSDFkShVFS6s= X-Google-Smtp-Source: AA6agR4HPvqHPgSM+FGtswZnjWwErL4pyYPkH/sWgpLeaQ6R70Cxa9MKsNdM5K0HZ71x9mVqPefs4XBAnKBUNDQXkEU= X-Received: by 2002:a25:7c7:0:b0:696:237:d000 with SMTP id 190-20020a2507c7000000b006960237d000mr17478465ybh.626.1662036570514; Thu, 01 Sep 2022 05:49:30 -0700 (PDT) MIME-Version: 1.0 From: =?UTF-8?Q?Reynir_Bj=C3=B6rnsson?= Date: Thu, 1 Sep 2022 14:49:19 +0200 Message-ID: Subject: UncensordDNS and DNS brownout today To: mirageos-devel@lists.xenproject.org Content-Type: text/plain; charset="UTF-8" Dear list, The default DNS server used in Mirage applications is uncensoreddns.org. A while ago they announced that they would be turning off DNS queries over plaintext and set today as the date[1]. We had heard a rumor this was on the table so we in Robur worked on implementing DNS over TLS which we released in November[2]. If you are experiencing some of your unikernels failing to resolve hostnames this may be why. Thankfully, today it is just a 12-hour brownout and the real phase out will be on October 1[0]. Best, Reynir [0]: https://blog.uncensoreddns.org/blog/40-cleartext-dns-turned-off/ [1]: https://blog.uncensoreddns.org/blog/39-the-unfriendly-internet-turning-off-cleartext-lookups-in-september/ [2]: https://github.com/mirage/ocaml-dns/blob/main/CHANGES.md#v610-2021-11-10 From mirageos-devel-bounces@lists.xenproject.org Mon Sep 05 13:01:06 2022 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Mon, 05 Sep 2022 13:01:06 +0000 Received: from list by lists.xenproject.org with outflank-mailman.398843.639817 (Exim 4.92) (envelope-from ) id 1oVBii-0005yF-4F; Mon, 05 Sep 2022 13:00:56 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 398843.639817; Mon, 05 Sep 2022 13:00:56 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVBii-0005y8-0J; Mon, 05 Sep 2022 13:00:56 +0000 Received: by outflank-mailman (input) for mailman id 398843; Mon, 05 Sep 2022 13:00:55 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVBih-0005w2-8V for mirageos-devel@lists.xenproject.org; Mon, 05 Sep 2022 13:00:55 +0000 Received: from mail.mehnert.org (mail.mehnert.org [213.73.89.200]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id c5ef49f2-2d1a-11ed-af93-0125da4c0113; Mon, 05 Sep 2022 15:00:54 +0200 (CEST) Received: from [192.168.42.80] (dslb-178-008-101-109.178.008.pools.vodafone-ip.de [178.8.101.109]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "hannes@mehnert.org", Issuer "mehnert root CA" (not verified)) by mail.mehnert.org (Postfix) with ESMTPS id 673931E51A for ; Mon, 5 Sep 2022 15:00:52 +0200 (CEST) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: c5ef49f2-2d1a-11ed-af93-0125da4c0113 Message-ID: <54f22a23-693a-d86f-3912-6695c2f540c3@mehnert.org> Date: Mon, 5 Sep 2022 15:00:52 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 To: mirageos-devel@lists.xenproject.org Content-Language: en-US From: Hannes Mehnert Subject: upcoming MirageOS meeting 2022-09-07 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hey, on this Wednesday, September 7th, starting at 14:00 CEST in https://whereby.com/ocamllabs, we'll have a MirageOS meeting again. We'll take notes at https://pad.data.coop/kLbooYhzS-CUUd4vxp0qaA# The agenda so far has some API adjustments, and removing some deprecated code -- feel free to add your other discussion points :) - mirage-flow "shutdown" https://github.com/mirage/mirage-flow/pull/48 - mirage-flow "read" https://github.com/mirage/mirage-flow/issues/46 - mirage-kv API enhancements (partial read, partial write, rename, size) https://github.com/mirage/mirage-kv/pull/28 -- implementation status? - sunsetting Tcpip.Stack.V4 and Tcpip.Stack.V6 (in favour of Tcpip.Stack.V4V6) https://github.com/mirage/mirage/issues/1339 Best and see you on Wednesday! Hannes From mirageos-devel-bounces@lists.xenproject.org Wed Sep 07 13:55:26 2022 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 07 Sep 2022 13:55:26 +0000 Received: from list by lists.xenproject.org with outflank-mailman.401828.643777 (Exim 4.92) (envelope-from ) id 1oVvWJ-00037z-Ao; Wed, 07 Sep 2022 13:55:11 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 401828.643777; Wed, 07 Sep 2022 13:55:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVvWJ-00037s-7O; Wed, 07 Sep 2022 13:55:11 +0000 Received: by outflank-mailman (input) for mailman id 401828; Wed, 07 Sep 2022 13:55:09 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVvWH-00037m-S0 for mirageos-devel@lists.xenproject.org; Wed, 07 Sep 2022 13:55:09 +0000 Received: from mail.mehnert.org (mail.mehnert.org [213.73.89.200]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ad7c679f-2eb4-11ed-af93-0125da4c0113; Wed, 07 Sep 2022 15:55:08 +0200 (CEST) Received: from [192.168.42.80] (dslb-178-008-099-255.178.008.pools.vodafone-ip.de [178.8.99.255]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "hannes@mehnert.org", Issuer "mehnert root CA" (not verified)) by mail.mehnert.org (Postfix) with ESMTPS id DEC32E875 for ; Wed, 7 Sep 2022 15:55:04 +0200 (CEST) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: ad7c679f-2eb4-11ed-af93-0125da4c0113 Message-ID: <47a2c4d9-40e0-9ff7-97e6-bb378c48d5d1@mehnert.org> Date: Wed, 7 Sep 2022 15:55:03 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Subject: Re: upcoming MirageOS meeting 2022-09-07 Content-Language: en-US From: Hannes Mehnert To: mirageos-devel@lists.xenproject.org References: <54f22a23-693a-d86f-3912-6695c2f540c3@mehnert.org> In-Reply-To: <54f22a23-693a-d86f-3912-6695c2f540c3@mehnert.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hello again. below are the notes. The next meeting is 2022-09-21 at 14:00 CEST in whereby.com/ocamllabs. I'm glad we took some decisions and started to merge some of the PRs. # MirageOS meeting 2022-09-07 participants: lucas, hannes, pierre, reynir, christiano, talex5, taka, dinosaure ## qubes-mirage-firewall release (0.8) - it is out, this time with mirage4 and PVH support - fixes memory issues we had before (now using the footprint / dlmalloc from ocaml-solo5) - still some memory issue when ocaml wants to allocate more memory than what's available (https://github.com/mirage/qubes-mirage-firewall/issues/143) - unikernel is too slow (can't deal with bandwidth) - also ongoing work: qrexec version 3 https://github.com/mirage/mirage-qubes/pull/60 - some hardcoded DNS servers in there (see https://github.com/mirage/qubes-mirage-firewall/pull/142) - remove UI code improves performance - what is the packet rate? 1'000? 10'000? not clear, using iperf3 - old stats at https://github.com/mirage/qubes-mirage-firewall/pull/45, also https://github.com/mirage/qubes-mirage-firewall/issues/130 ## mirage-flow "shutdown" https://github.com/mirage/mirage-flow/pull/48 - no comments ## mirage-flow "read" https://github.com/mirage/mirage-flow/issues/46 - maybe have both? - eio: the reader has to provide the buffer by default, but the source can offer alternative optimisations. - jane street has an annotation that a writer can only use the buffer until "read" is finished -> propose a second function with the implementation thereof ## mirage-flow "write" (documentation update!) for `write`: h2 reuses the buffer (so there's need for a copy); mirage-flow takes ownership of the buffer (thus deallocates it eventually) ## mirage-kv API enhancements (partial read, partial write, rename, size) https://github.com/mirage/mirage-kv/pull/28 - how hard is it to implement for irmin? - let's merge and get some bounds for irmin etc. ## sunsetting Tcpip.Stack.V4 and Tcpip.Stack.V6 (in favour of Tcpip.Stack.V4V6) https://github.com/mirage/mirage/issues/1339 - agreed, let's merge ## tls-eio https://github.com/mirleft/ocaml-tls/pull/451 - CI is failing: add a "ocaml < 5" annotation to tls-async - could get a review - requires mirage-crypto-rng-eio (merge & release) On 05/09/2022 15:00, Hannes Mehnert wrote: > Hey, > > on this Wednesday, September 7th, starting at 14:00 CEST in > https://whereby.com/ocamllabs, we'll have a MirageOS meeting again. > > We'll take notes at https://pad.data.coop/kLbooYhzS-CUUd4vxp0qaA# > > The agenda so far has some API adjustments, and removing some deprecated > code -- feel free to add your other discussion points :) > > - mirage-flow "shutdown" https://github.com/mirage/mirage-flow/pull/48 > - mirage-flow "read" https://github.com/mirage/mirage-flow/issues/46 > - mirage-kv API enhancements (partial read, partial write, rename, size) > https://github.com/mirage/mirage-kv/pull/28 -- implementation status? > - sunsetting Tcpip.Stack.V4 and Tcpip.Stack.V6 (in favour of > Tcpip.Stack.V4V6) https://github.com/mirage/mirage/issues/1339 > > > Best and see you on Wednesday! > > Hannes > From mirageos-devel-bounces@lists.xenproject.org Wed Sep 07 14:25:45 2022 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 07 Sep 2022 14:25:45 +0000 Received: from list by lists.xenproject.org with outflank-mailman.401874.643836 (Exim 4.92) (envelope-from ) id 1oVvzn-000156-RF; Wed, 07 Sep 2022 14:25:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 401874.643836; Wed, 07 Sep 2022 14:25:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVvzn-00014z-Of; Wed, 07 Sep 2022 14:25:39 +0000 Received: by outflank-mailman (input) for mailman id 401874; Wed, 07 Sep 2022 14:25:39 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVvzn-00014t-6R for mirageos-devel@lists.xenproject.org; Wed, 07 Sep 2022 14:25:39 +0000 Received: from w1.tutanota.de (w1.tutanota.de [81.3.6.162]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f107bb96-2eb8-11ed-af93-0125da4c0113; Wed, 07 Sep 2022 16:25:37 +0200 (CEST) Received: from w3.tutanota.de (unknown [192.168.1.164]) by w1.tutanota.de (Postfix) with ESMTP id CBDFDFBF93B; Wed, 7 Sep 2022 14:25:36 +0000 (UTC) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: f107bb96-2eb8-11ed-af93-0125da4c0113 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1662560736; s=s1; d=tuta.io; h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Content-Transfer-Encoding:Cc:Cc:Date:Date:In-Reply-To:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:References:Sender; bh=SpGtS7Lm5ab38cesIf+wvOREAVqDm6rywbS5bLgdsj4=; b=xN55BSTtCkH/8GTAQVrNejlcpoxBvGe10tMPqreLUvSxzLVI1/Zk0Gv6AxqtjeEG UmXxutwSeaC307Kfg/gciz0n+DTc58KK8bwOoiNtG7cYjwu0/yVK6CCOfdNmL82W3vI B2aR6vzD3nAbj8hWXAd+8wtBo7QlTZ0Dfz9LLEDHLtT5j5qlHRs+/8bZS8N0UjP4kyC ZQsqm2Da253Eks89YoBlrIT1hDj1IUVnFHuUiB5+uXHa91NVaRdrX1SdCtfuzwwQjOD 4AhPt8ALcLltWgXJzeRLU+mklcfMY61CLJf1j2zKzxVMtsFW4v9D74QdjzEzGLV3EvB WejlrWxMDg== Date: Wed, 7 Sep 2022 16:25:36 +0200 (CEST) From: pierre.alain@tuta.io To: Hannes Mehnert Cc: mirageos-devel@lists.xenproject.org Message-ID: In-Reply-To: <47a2c4d9-40e0-9ff7-97e6-bb378c48d5d1@mehnert.org> References: <54f22a23-693a-d86f-3912-6695c2f540c3@mehnert.org> <47a2c4d9-40e0-9ff7-97e6-bb378c48d5d1@mehnert.org> Subject: Re: upcoming MirageOS meeting 2022-09-07 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi all, Regarding the UDP #pkt/s with qubes-mirage-firewall, I did a quick test (wi= th fortunately no hang or unfortunately as it doesn't give any additional c= lue). As I'm not currently at home I cannot test with my FTTH connexion but= I think it's still relevant as I managed to get a CPU saturation with mira= ge. With mirageOS as firewall (CPU is 100%): $ iperf3 -c lon.speedtest.clouvider.net -p 5203 -u -b 0 Connecting to host lon.speedtest.clouvider.net, port 5203 [=C2=A0 5] local 10.137.0.20 port 56373 connected to 5.180.211.133 port 520= 3 [ ID] Interval=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Transfer=C2=A0=C2=A0=C2=A0=C2=A0 Bitrate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Total Datagrams [=C2=A0 5]=C2=A0=C2=A0 0.00-1.00=C2=A0=C2=A0 sec=C2=A0 51.9 MBytes=C2=A0=C2= =A0 436 Mbits/sec=C2=A0 38090=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 1.00-2.00=C2=A0=C2=A0 sec=C2=A0 71.1 MBytes=C2=A0=C2= =A0 597 Mbits/sec=C2=A0 52160=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 2.00-3.00=C2=A0=C2=A0 sec=C2=A0 71.8 MBytes=C2=A0=C2= =A0 603 Mbits/sec=C2=A0 52680=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 3.00-4.00=C2=A0=C2=A0 sec=C2=A0 73.7 MBytes=C2=A0=C2= =A0 618 Mbits/sec=C2=A0 54030=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 4.00-5.00=C2=A0=C2=A0 sec=C2=A0 73.8 MBytes=C2=A0=C2= =A0 619 Mbits/sec=C2=A0 54080=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 5.00-6.00=C2=A0=C2=A0 sec=C2=A0 60.0 MBytes=C2=A0=C2= =A0 504 Mbits/sec=C2=A0 44030=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 6.00-7.00=C2=A0=C2=A0 sec=C2=A0 45.0 MBytes=C2=A0=C2= =A0 377 Mbits/sec=C2=A0 32990=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 7.00-8.00=C2=A0=C2=A0 sec=C2=A0 73.6 MBytes=C2=A0=C2= =A0 617 Mbits/sec=C2=A0 53950=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 8.00-9.00=C2=A0=C2=A0 sec=C2=A0 70.8 MBytes=C2=A0=C2= =A0 594 Mbits/sec=C2=A0 51920=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 9.00-10.00=C2=A0 sec=C2=A0 70.6 MBytes=C2=A0=C2=A0 5= 92 Mbits/sec=C2=A0 51760=C2=A0 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Transfer=C2=A0=C2=A0=C2=A0=C2=A0 Bitrate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Jitter=C2=A0=C2=A0=C2=A0 Lost/Total Datagrams [=C2=A0 5]=C2=A0=C2=A0 0.00-10.00=C2=A0 sec=C2=A0=C2=A0 662 MBytes=C2=A0=C2= =A0 556 Mbits/sec=C2=A0 0.000 ms=C2=A0 0/485690 (0%)=C2=A0 sender [=C2=A0 5]=C2=A0=C2=A0 0.00-10.00=C2=A0 sec=C2=A0 57.2 MBytes=C2=A0 48.0 Mb= its/sec=C2=A0 0.269 ms=C2=A0 437001/478947 (91%)=C2=A0 receiver with linux as firewall (CPU is around 80%): $ iperf3 -c lon.speedtest.clouvider.net -p 5203 -u -b 0 Connecting to host lon.speedtest.clouvider.net, port 5203 [=C2=A0 5] local 10.137.0.21 port 49539 connected to 5.180.211.133 port 520= 3 [ ID] Interval=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Transfer=C2=A0=C2=A0=C2=A0=C2=A0 Bitrate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Total Datagrams [=C2=A0 5]=C2=A0=C2=A0 0.00-1.00=C2=A0=C2=A0 sec=C2=A0 82.2 MBytes=C2=A0=C2= =A0 689 Mbits/sec=C2=A0 60240=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 1.00-2.00=C2=A0=C2=A0 sec=C2=A0 85.7 MBytes=C2=A0=C2= =A0 719 Mbits/sec=C2=A0 62850=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 2.00-3.00=C2=A0=C2=A0 sec=C2=A0 85.2 MBytes=C2=A0=C2= =A0 715 Mbits/sec=C2=A0 62460=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 3.00-4.00=C2=A0=C2=A0 sec=C2=A0 79.0 MBytes=C2=A0=C2= =A0 663 Mbits/sec=C2=A0 57920=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 4.00-5.00=C2=A0=C2=A0 sec=C2=A0 84.8 MBytes=C2=A0=C2= =A0 712 Mbits/sec=C2=A0 62210=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 5.00-6.00=C2=A0=C2=A0 sec=C2=A0 80.7 MBytes=C2=A0=C2= =A0 676 Mbits/sec=C2=A0 59140=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 6.00-7.00=C2=A0=C2=A0 sec=C2=A0 80.3 MBytes=C2=A0=C2= =A0 674 Mbits/sec=C2=A0 58880=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 7.00-8.00=C2=A0=C2=A0 sec=C2=A0 80.2 MBytes=C2=A0=C2= =A0 673 Mbits/sec=C2=A0 58810=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 8.00-9.00=C2=A0=C2=A0 sec=C2=A0 80.2 MBytes=C2=A0=C2= =A0 673 Mbits/sec=C2=A0 58830=C2=A0 [=C2=A0 5]=C2=A0=C2=A0 9.00-10.00=C2=A0 sec=C2=A0 77.5 MBytes=C2=A0=C2=A0 6= 50 Mbits/sec=C2=A0 56820=C2=A0 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Transfer=C2=A0=C2=A0=C2=A0=C2=A0 Bitrate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Jitter=C2=A0=C2=A0=C2=A0 Lost/Total Datagrams [=C2=A0 5]=C2=A0=C2=A0 0.00-10.00=C2=A0 sec=C2=A0=C2=A0 816 MBytes=C2=A0=C2= =A0 684 Mbits/sec=C2=A0 0.000 ms=C2=A0 0/598160 (0%)=C2=A0 sender [=C2=A0 5]=C2=A0=C2=A0 0.00-10.00=C2=A0 sec=C2=A0 57.2 MBytes=C2=A0 47.9 Mb= its/sec=C2=A0 0.272 ms=C2=A0 549868/591780 (93%)=C2=A0 receiver As Christiano said, there may be a way for optimizations where solo5 does t= he IO, as here, once the NAT is done, it's only a matter of copying data fr= om one page to another. This should also be reproducible with a simple nat unikernel such as https:= //github.com/mirage/mirage-nat/tree/main/example . I'll try that later. Best, Pierre --=20 P. From mirageos-devel-bounces@lists.xenproject.org Wed Sep 07 15:18:34 2022 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 07 Sep 2022 15:18:34 +0000 Received: from list by lists.xenproject.org with outflank-mailman.401955.643932 (Exim 4.92) (envelope-from ) id 1oVwov-0002zB-0r; Wed, 07 Sep 2022 15:18:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 401955.643932; Wed, 07 Sep 2022 15:18:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVwou-0002z4-UH; Wed, 07 Sep 2022 15:18:28 +0000 Received: by outflank-mailman (input) for mailman id 401955; Wed, 07 Sep 2022 15:18:28 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVwou-0002yy-7Z for mirageos-devel@lists.xenproject.org; Wed, 07 Sep 2022 15:18:28 +0000 Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [2a00:1450:4864:20::134]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 51e40eea-2ec0-11ed-af93-0125da4c0113; Wed, 07 Sep 2022 17:18:26 +0200 (CEST) Received: by mail-lf1-x134.google.com with SMTP id q21so8738411lfo.0 for ; Wed, 07 Sep 2022 08:18:26 -0700 (PDT) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: 51e40eea-2ec0-11ed-af93-0125da4c0113 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=haesbaert-org.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=vuyjO7woFWlLNkMLOUYSYOXF3U33oePTAAtx4zw7d04=; b=Ucq6ZM5AqW5YspXUaFss6fB4W8emjxXLdDw//5T6Q4TLPqVJSS0KeJdtP9VIiWDz1a COjkO69EYOWF7SvTsHlJqbr2yjDOnQUjtqxY7FAVMxo/cvvNDPPoztnghpeCg6hLv/C4 cEKibutoCJ6MQfzUC2Z5lUwCc9oGWGjZ7qhXnhCJ1g+fv/X5THJ29lRlI8mSkJHjgJsv lntk9Z8+Sl79KJR4/6DUPf+mFjg7EmTha5ofUtQtkDDM8u43zSeuGO1U6SJOnMr8M7QD uNoBTc+do2EYlH0112z+q0pAyJvVBsKkFqd3jE/Iv4k0Doi9bywwqi3bZiSD21dd30Z+ J/fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=vuyjO7woFWlLNkMLOUYSYOXF3U33oePTAAtx4zw7d04=; b=3OTyGY625jdHdiWojy7S0a4cAI9zKAWhF7wkprKTbnfnnWETvYfCqE3iFZqL80jAXn 8S1cW71yGJIyLWMizGnNzWlMvr5JhXWRkcThp0LsyDahR3GD6VKbl//ag+ybLJbIfqJs 9xVtteEsC4z6Q3juSx7SX/pQkrNlqiQx/ADkh6rZ6LT6r3y0/uuKC+rftRn4sFEv/fQ5 Hzngciaya5pmz0ZfXJpRA0swkGxG0em+SZRP0VVq/YHFL9pAQguKnFmKtU7IYjByUBL3 TfrNMPI4etIDgILzjjwFabRcS9sY8NQuyNGBKkZpS+9GXOdM8moDTVF2CUs/3yZlr7hw 8HlQ== X-Gm-Message-State: ACgBeo2ZL+bcGsxfej00XXHJ/RSZgn1Qut5TAh8neL6r7hQjJb8gD6xb ImAi681B4eOJFm1aIkGNzWffUX9yEmJGXGajk9XyJpQA X-Google-Smtp-Source: AA6agR5Qu5YL4179N/Gl761UujeWu+F77InLjJ+b48vEbbIFxemnMkIEx9swWCKxf1mQv+ktPgcLM3R6ZmPhUnIJJiY= X-Received: by 2002:ac2:4f90:0:b0:496:91da:fb45 with SMTP id z16-20020ac24f90000000b0049691dafb45mr1343099lfs.550.1662563905582; Wed, 07 Sep 2022 08:18:25 -0700 (PDT) MIME-Version: 1.0 References: <54f22a23-693a-d86f-3912-6695c2f540c3@mehnert.org> <47a2c4d9-40e0-9ff7-97e6-bb378c48d5d1@mehnert.org> In-Reply-To: From: "Christiano F. Haesbaert" Date: Wed, 7 Sep 2022 17:18:14 +0200 Message-ID: Subject: Re: upcoming MirageOS meeting 2022-09-07 To: pierre.alain@tuta.io Cc: Hannes Mehnert , mirageos-devel Content-Type: multipart/alternative; boundary="000000000000023c4105e817d315" --000000000000023c4105e817d315 Content-Type: text/plain; charset="UTF-8" That looks painfully slow, it's sad that iperf doesn't report packets per second but that's around ~5kpps at 1460B/frame. I'm also surprised your sender is not saturating gigabit, but I'd have to check how iperf knows it was able to send out the packets, usually no one cares about the sender as long as you saturate the link. I think I feel I owe an explanation since I frequently talk about how solo5 IO is slow but never explain why/how. First of all I don't mean this as a bashing, I love solo5, the code was never intended to be optimized for network performance. Also take this with a grain of salt, there can be multiple things involved and it might be a bug somewhere completely unrelated, the truth is I haven't run tests enough to understand how much of the solo5 IO can be blamed, I just repeat this because "there might be nothing wrong". I had a quick look at the xen code for the first time now, and it's quite different from the rest, it has very little to do with how solo5 does IO the ring management and IO code is in ocaml and I can't really reason about it without a lot of time. At this point I'd try to turn the firewall into an "expensive cable" just copy packets from input to output and get some idea of the baseline. On Wed, 7 Sept 2022 at 16:25, wrote: > Hi all, > Regarding the UDP #pkt/s with qubes-mirage-firewall, I did a quick test > (with fortunately no hang or unfortunately as it doesn't give any > additional clue). As I'm not currently at home I cannot test with my FTTH > connexion but I think it's still relevant as I managed to get a CPU > saturation with mirage. > > With mirageOS as firewall (CPU is 100%): > $ iperf3 -c lon.speedtest.clouvider.net -p 5203 -u -b 0 > Connecting to host lon.speedtest.clouvider.net, port 5203 > [ 5] local 10.137.0.20 port 56373 connected to 5.180.211.133 port 5203 > [ ID] Interval Transfer Bitrate Total Datagrams > [ 5] 0.00-1.00 sec 51.9 MBytes 436 Mbits/sec 38090 > [ 5] 1.00-2.00 sec 71.1 MBytes 597 Mbits/sec 52160 > [ 5] 2.00-3.00 sec 71.8 MBytes 603 Mbits/sec 52680 > [ 5] 3.00-4.00 sec 73.7 MBytes 618 Mbits/sec 54030 > [ 5] 4.00-5.00 sec 73.8 MBytes 619 Mbits/sec 54080 > [ 5] 5.00-6.00 sec 60.0 MBytes 504 Mbits/sec 44030 > [ 5] 6.00-7.00 sec 45.0 MBytes 377 Mbits/sec 32990 > [ 5] 7.00-8.00 sec 73.6 MBytes 617 Mbits/sec 53950 > [ 5] 8.00-9.00 sec 70.8 MBytes 594 Mbits/sec 51920 > [ 5] 9.00-10.00 sec 70.6 MBytes 592 Mbits/sec 51760 > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Jitter Lost/Total > Datagrams > [ 5] 0.00-10.00 sec 662 MBytes 556 Mbits/sec 0.000 ms 0/485690 > (0%) sender > [ 5] 0.00-10.00 sec 57.2 MBytes 48.0 Mbits/sec 0.269 ms > 437001/478947 (91%) receiver > > with linux as firewall (CPU is around 80%): > $ iperf3 -c lon.speedtest.clouvider.net -p 5203 -u -b 0 > Connecting to host lon.speedtest.clouvider.net, port 5203 > [ 5] local 10.137.0.21 port 49539 connected to 5.180.211.133 port 5203 > [ ID] Interval Transfer Bitrate Total Datagrams > [ 5] 0.00-1.00 sec 82.2 MBytes 689 Mbits/sec 60240 > [ 5] 1.00-2.00 sec 85.7 MBytes 719 Mbits/sec 62850 > [ 5] 2.00-3.00 sec 85.2 MBytes 715 Mbits/sec 62460 > [ 5] 3.00-4.00 sec 79.0 MBytes 663 Mbits/sec 57920 > [ 5] 4.00-5.00 sec 84.8 MBytes 712 Mbits/sec 62210 > [ 5] 5.00-6.00 sec 80.7 MBytes 676 Mbits/sec 59140 > [ 5] 6.00-7.00 sec 80.3 MBytes 674 Mbits/sec 58880 > [ 5] 7.00-8.00 sec 80.2 MBytes 673 Mbits/sec 58810 > [ 5] 8.00-9.00 sec 80.2 MBytes 673 Mbits/sec 58830 > [ 5] 9.00-10.00 sec 77.5 MBytes 650 Mbits/sec 56820 > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Jitter Lost/Total > Datagrams > [ 5] 0.00-10.00 sec 816 MBytes 684 Mbits/sec 0.000 ms 0/598160 > (0%) sender > [ 5] 0.00-10.00 sec 57.2 MBytes 47.9 Mbits/sec 0.272 ms > 549868/591780 (93%) receiver > > As Christiano said, there may be a way for optimizations where solo5 does > the IO, as here, once the NAT is done, it's only a matter of copying data > from one page to another. > This should also be reproducible with a simple nat unikernel such as > https://github.com/mirage/mirage-nat/tree/main/example . I'll try that > later. > > Best, > Pierre > -- > P. > > --000000000000023c4105e817d315 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
That looks painfully slow, it's sad that iperf doesn&#= 39;t report packets per second but that's around ~5kpps at 1460B/frame.=
I'm also surprised your sender is not saturating gigabit, but I= 9;d have to check how iperf knows it was able to send out the packets, usua= lly no one cares about the sender as long as you saturate the link.

I think I feel I owe an explanation since I frequently ta= lk about how solo5 IO is slow but never explain=C2=A0why/how. First of all = I don't mean this as a bashing, I love solo5, the code was never intend= ed to be optimized for network performance. Also take this with a grain of = salt, there can be multiple things involved and it might be a bug somewhere= completely unrelated, the truth is I haven't run tests enough to under= stand how much of the solo5 IO can be blamed, I just repeat this because &q= uot;there might be nothing wrong".

I had a qu= ick look at the xen code for the first time now, and it's quite differe= nt from the rest, it has very little to do with how solo5 does IO the ring = management and IO code is in ocaml and I can't really reason about it w= ithout a lot of time.
At this point I'd try to turn the firew= all into an "expensive cable" just copy packets from input to out= put and get some idea of the baseline.


On Wed, 7 Sept 2= 022 at 16:25, <pierre.alain@tuta= .io> wrote:
Hi all,
Regarding the UDP #pkt/s with qubes-mirage-firewall, I did a quick test (wi= th fortunately no hang or unfortunately as it doesn't give any addition= al clue). As I'm not currently at home I cannot test with my FTTH conne= xion but I think it's still relevant as I managed to get a CPU saturati= on with mirage.

With mirageOS as firewall (CPU is 100%):
$ iperf3 -c lon.speedtest.clouvider.net -p 5203 -u -b 0
Connecting to host lon.speedtest.clouvider.net, port 5203
[=C2=A0 5] local 10.137.0.20 port 56373 connected to 5.180.211.133 port 520= 3
[ ID] Interval=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Transfer=C2=A0=C2=A0=C2=A0=C2=A0 Bitrate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Total Datagrams
[=C2=A0 5]=C2=A0=C2=A0 0.00-1.00=C2=A0=C2=A0 sec=C2=A0 51.9 MBytes=C2=A0=C2= =A0 436 Mbits/sec=C2=A0 38090=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 1.00-2.00=C2=A0=C2=A0 sec=C2=A0 71.1 MBytes=C2=A0=C2= =A0 597 Mbits/sec=C2=A0 52160=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 2.00-3.00=C2=A0=C2=A0 sec=C2=A0 71.8 MBytes=C2=A0=C2= =A0 603 Mbits/sec=C2=A0 52680=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 3.00-4.00=C2=A0=C2=A0 sec=C2=A0 73.7 MBytes=C2=A0=C2= =A0 618 Mbits/sec=C2=A0 54030=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 4.00-5.00=C2=A0=C2=A0 sec=C2=A0 73.8 MBytes=C2=A0=C2= =A0 619 Mbits/sec=C2=A0 54080=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 5.00-6.00=C2=A0=C2=A0 sec=C2=A0 60.0 MBytes=C2=A0=C2= =A0 504 Mbits/sec=C2=A0 44030=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 6.00-7.00=C2=A0=C2=A0 sec=C2=A0 45.0 MBytes=C2=A0=C2= =A0 377 Mbits/sec=C2=A0 32990=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 7.00-8.00=C2=A0=C2=A0 sec=C2=A0 73.6 MBytes=C2=A0=C2= =A0 617 Mbits/sec=C2=A0 53950=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 8.00-9.00=C2=A0=C2=A0 sec=C2=A0 70.8 MBytes=C2=A0=C2= =A0 594 Mbits/sec=C2=A0 51920=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 9.00-10.00=C2=A0 sec=C2=A0 70.6 MBytes=C2=A0=C2=A0 5= 92 Mbits/sec=C2=A0 51760=C2=A0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Transfer=C2=A0=C2=A0=C2=A0=C2=A0 Bitrate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Jitter=C2=A0=C2=A0=C2=A0 Lost/Total Datagrams
[=C2=A0 5]=C2=A0=C2=A0 0.00-10.00=C2=A0 sec=C2=A0=C2=A0 662 MBytes=C2=A0=C2= =A0 556 Mbits/sec=C2=A0 0.000 ms=C2=A0 0/485690 (0%)=C2=A0 sender
[=C2=A0 5]=C2=A0=C2=A0 0.00-10.00=C2=A0 sec=C2=A0 57.2 MBytes=C2=A0 48.0 Mb= its/sec=C2=A0 0.269 ms=C2=A0 437001/478947 (91%)=C2=A0 receiver

with linux as firewall (CPU is around 80%):
$ iperf3 -c lon.speedtest.clouvider.net -p 5203 -u -b 0
Connecting to host lon.speedtest.clouvider.net, port 5203
[=C2=A0 5] local 10.137.0.21 port 49539 connected to 5.180.211.133 port 520= 3
[ ID] Interval=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Transfer=C2=A0=C2=A0=C2=A0=C2=A0 Bitrate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Total Datagrams
[=C2=A0 5]=C2=A0=C2=A0 0.00-1.00=C2=A0=C2=A0 sec=C2=A0 82.2 MBytes=C2=A0=C2= =A0 689 Mbits/sec=C2=A0 60240=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 1.00-2.00=C2=A0=C2=A0 sec=C2=A0 85.7 MBytes=C2=A0=C2= =A0 719 Mbits/sec=C2=A0 62850=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 2.00-3.00=C2=A0=C2=A0 sec=C2=A0 85.2 MBytes=C2=A0=C2= =A0 715 Mbits/sec=C2=A0 62460=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 3.00-4.00=C2=A0=C2=A0 sec=C2=A0 79.0 MBytes=C2=A0=C2= =A0 663 Mbits/sec=C2=A0 57920=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 4.00-5.00=C2=A0=C2=A0 sec=C2=A0 84.8 MBytes=C2=A0=C2= =A0 712 Mbits/sec=C2=A0 62210=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 5.00-6.00=C2=A0=C2=A0 sec=C2=A0 80.7 MBytes=C2=A0=C2= =A0 676 Mbits/sec=C2=A0 59140=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 6.00-7.00=C2=A0=C2=A0 sec=C2=A0 80.3 MBytes=C2=A0=C2= =A0 674 Mbits/sec=C2=A0 58880=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 7.00-8.00=C2=A0=C2=A0 sec=C2=A0 80.2 MBytes=C2=A0=C2= =A0 673 Mbits/sec=C2=A0 58810=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 8.00-9.00=C2=A0=C2=A0 sec=C2=A0 80.2 MBytes=C2=A0=C2= =A0 673 Mbits/sec=C2=A0 58830=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 9.00-10.00=C2=A0 sec=C2=A0 77.5 MBytes=C2=A0=C2=A0 6= 50 Mbits/sec=C2=A0 56820=C2=A0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Transfer=C2=A0=C2=A0=C2=A0=C2=A0 Bitrate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Jitter=C2=A0=C2=A0=C2=A0 Lost/Total Datagrams
[=C2=A0 5]=C2=A0=C2=A0 0.00-10.00=C2=A0 sec=C2=A0=C2=A0 816 MBytes=C2=A0=C2= =A0 684 Mbits/sec=C2=A0 0.000 ms=C2=A0 0/598160 (0%)=C2=A0 sender
[=C2=A0 5]=C2=A0=C2=A0 0.00-10.00=C2=A0 sec=C2=A0 57.2 MBytes=C2=A0 47.9 Mb= its/sec=C2=A0 0.272 ms=C2=A0 549868/591780 (93%)=C2=A0 receiver

As Christiano said, there may be a way for optimizations where solo5 does t= he IO, as here, once the NAT is done, it's only a matter of copying dat= a from one page to another.
This should also be reproducible with a simple nat unikernel such as https://github.com/mirage/mirage-nat/tree/main/examp= le . I'll try that later.

Best,
Pierre
--
P.

--000000000000023c4105e817d315-- From mirageos-devel-bounces@lists.xenproject.org Wed Sep 07 15:37:28 2022 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 07 Sep 2022 15:37:28 +0000 Received: from list by lists.xenproject.org with outflank-mailman.401981.643970 (Exim 4.92) (envelope-from ) id 1oVx7F-0006ro-A6; Wed, 07 Sep 2022 15:37:25 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 401981.643970; Wed, 07 Sep 2022 15:37:25 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVx7F-0006rh-6w; Wed, 07 Sep 2022 15:37:25 +0000 Received: by outflank-mailman (input) for mailman id 401981; Wed, 07 Sep 2022 15:37:24 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVx7E-0006oz-LI for mirageos-devel@lists.xenproject.org; Wed, 07 Sep 2022 15:37:24 +0000 Received: from mail-ot1-x331.google.com (mail-ot1-x331.google.com [2607:f8b0:4864:20::331]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f762760f-2ec2-11ed-af93-0125da4c0113; Wed, 07 Sep 2022 17:37:23 +0200 (CEST) Received: by mail-ot1-x331.google.com with SMTP id br15-20020a056830390f00b0061c9d73b8bdso10517970otb.6 for ; Wed, 07 Sep 2022 08:37:23 -0700 (PDT) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: f762760f-2ec2-11ed-af93-0125da4c0113 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=mCzhWRu+xL2NpzpOhY7RUBwr1SCJ0DJoiFYfKmb/k84=; b=iM8izr1hg+HxOKUYDB6egq/6m++pYsxicQOtAgA+JTfrdgXZKunkcHHYBv04J8QOgs hJKRgQBwOSVY91vWEbblXsu4Fperyc6FFaV8SOVHEE1QS30KzOt5AGbGOb7G6BXkguEB qDJkpfBIHa2fNZB/HJhviPwKj2ORXnc5wEbi7/v7VV5K5vqdpI0V+P6+o3KC48zWzL9l VFPHBgIkCPXwTxpR6Ut8nAbppFb4GL9wYCCVGyUFBxraLlDVEZH4pZpV5pPWZZz9BK3Y PKt8Rd0SmPG8vEafO8pqyteZpP+zH1FUYyBamhhnS2ydD7+dHRCCghn+b+DIDGahYaKX KYkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=mCzhWRu+xL2NpzpOhY7RUBwr1SCJ0DJoiFYfKmb/k84=; b=ghvzAo/zmYfj3JmJyx5btnPThjtN1JlVQyP/kWVyeQnRpEBmirjGdmKrOVK7jOlhgI C7+NsLw9U1J5Nn9G/MefqgPCcl3gwa5lZtG806y2O6CPdc6A3+grshatNuBRRjASwgRR g3vxJeHfins/M3ZM02GSR0bgblbDuJzdut1I/FmQzbvsSLG1puslyJ+zhwMO4gfs3W7d cjvnTfGMTjBHNCqdWSxrdoMnd1JFkGMTUm6M03yekNnykm5jRjpZkpIWFaksxvZOMRHC tWzp6C9s1WPG6uYzAlqzqAUT3u3wxcgkP7sSgO7P/txk8FKgwgw++pBO2muGD2bVXnKo h3Vg== X-Gm-Message-State: ACgBeo3M9hFIMn/sMr0eNqiywyANG8JrS9TR7EP/yRLbkBMZ7+Cls9Be loXdy15rqRoFhhgDi+Nui+3JcGJzaKpekLBS1O4= X-Google-Smtp-Source: AA6agR7dxlx0IQu+ddlYkH9DUlZzPpr+ptervvzqyi1lZnafnqPlZByuYHGEzlOWiI/HWpLyavQ+gFDX6SGUJyivffM= X-Received: by 2002:a9d:188:0:b0:637:1879:c6 with SMTP id e8-20020a9d0188000000b00637187900c6mr1570339ote.269.1662565042039; Wed, 07 Sep 2022 08:37:22 -0700 (PDT) MIME-Version: 1.0 References: <54f22a23-693a-d86f-3912-6695c2f540c3@mehnert.org> <47a2c4d9-40e0-9ff7-97e6-bb378c48d5d1@mehnert.org> In-Reply-To: From: Romain Calascibetta Date: Wed, 7 Sep 2022 17:37:09 +0200 Message-ID: Subject: Re: upcoming MirageOS meeting 2022-09-07 To: "Christiano F. Haesbaert" Cc: pierre.alain@tuta.io, Hannes Mehnert , mirageos-devel Content-Type: multipart/alternative; boundary="000000000000bf306605e818160f" --000000000000bf306605e818160f Content-Type: text/plain; charset="UTF-8" Hi, Just to inform you that someone (@rand00) started to make a unikernel which permits testing a connection and seeing the bandwidth. It's useful to test multiple backends and probably have more informations about where is the bottleneck. https://github.com/rand00/connest It still is experimental but at least it helps me to find some limitations about my internet connection. It may be worthwhile to concentrate our efforts on the latter in order to finally have a tool to replace our good old mirage-iperf (https://github.com/dimosped/iperf-mirage). On Wed, Sep 7, 2022 at 5:18 PM Christiano F. Haesbaert < haesbaert@haesbaert.org> wrote: > That looks painfully slow, it's sad that iperf doesn't report packets per > second but that's around ~5kpps at 1460B/frame. > I'm also surprised your sender is not saturating gigabit, but I'd have to > check how iperf knows it was able to send out the packets, usually no one > cares about the sender as long as you saturate the link. > > I think I feel I owe an explanation since I frequently talk about how > solo5 IO is slow but never explain why/how. First of all I don't mean this > as a bashing, I love solo5, the code was never intended to be optimized for > network performance. Also take this with a grain of salt, there can be > multiple things involved and it might be a bug somewhere completely > unrelated, the truth is I haven't run tests enough to understand how much > of the solo5 IO can be blamed, I just repeat this because "there might be > nothing wrong". > > I had a quick look at the xen code for the first time now, and it's quite > different from the rest, it has very little to do with how solo5 does IO > the ring management and IO code is in ocaml and I can't really reason about > it without a lot of time. > At this point I'd try to turn the firewall into an "expensive cable" just > copy packets from input to output and get some idea of the baseline. > > > On Wed, 7 Sept 2022 at 16:25, wrote: > >> Hi all, >> Regarding the UDP #pkt/s with qubes-mirage-firewall, I did a quick test >> (with fortunately no hang or unfortunately as it doesn't give any >> additional clue). As I'm not currently at home I cannot test with my FTTH >> connexion but I think it's still relevant as I managed to get a CPU >> saturation with mirage. >> >> With mirageOS as firewall (CPU is 100%): >> $ iperf3 -c lon.speedtest.clouvider.net -p 5203 -u -b 0 >> Connecting to host lon.speedtest.clouvider.net, port 5203 >> [ 5] local 10.137.0.20 port 56373 connected to 5.180.211.133 port 5203 >> [ ID] Interval Transfer Bitrate Total Datagrams >> [ 5] 0.00-1.00 sec 51.9 MBytes 436 Mbits/sec 38090 >> [ 5] 1.00-2.00 sec 71.1 MBytes 597 Mbits/sec 52160 >> [ 5] 2.00-3.00 sec 71.8 MBytes 603 Mbits/sec 52680 >> [ 5] 3.00-4.00 sec 73.7 MBytes 618 Mbits/sec 54030 >> [ 5] 4.00-5.00 sec 73.8 MBytes 619 Mbits/sec 54080 >> [ 5] 5.00-6.00 sec 60.0 MBytes 504 Mbits/sec 44030 >> [ 5] 6.00-7.00 sec 45.0 MBytes 377 Mbits/sec 32990 >> [ 5] 7.00-8.00 sec 73.6 MBytes 617 Mbits/sec 53950 >> [ 5] 8.00-9.00 sec 70.8 MBytes 594 Mbits/sec 51920 >> [ 5] 9.00-10.00 sec 70.6 MBytes 592 Mbits/sec 51760 >> - - - - - - - - - - - - - - - - - - - - - - - - - >> [ ID] Interval Transfer Bitrate Jitter >> Lost/Total Datagrams >> [ 5] 0.00-10.00 sec 662 MBytes 556 Mbits/sec 0.000 ms 0/485690 >> (0%) sender >> [ 5] 0.00-10.00 sec 57.2 MBytes 48.0 Mbits/sec 0.269 ms >> 437001/478947 (91%) receiver >> >> with linux as firewall (CPU is around 80%): >> $ iperf3 -c lon.speedtest.clouvider.net -p 5203 -u -b 0 >> Connecting to host lon.speedtest.clouvider.net, port 5203 >> [ 5] local 10.137.0.21 port 49539 connected to 5.180.211.133 port 5203 >> [ ID] Interval Transfer Bitrate Total Datagrams >> [ 5] 0.00-1.00 sec 82.2 MBytes 689 Mbits/sec 60240 >> [ 5] 1.00-2.00 sec 85.7 MBytes 719 Mbits/sec 62850 >> [ 5] 2.00-3.00 sec 85.2 MBytes 715 Mbits/sec 62460 >> [ 5] 3.00-4.00 sec 79.0 MBytes 663 Mbits/sec 57920 >> [ 5] 4.00-5.00 sec 84.8 MBytes 712 Mbits/sec 62210 >> [ 5] 5.00-6.00 sec 80.7 MBytes 676 Mbits/sec 59140 >> [ 5] 6.00-7.00 sec 80.3 MBytes 674 Mbits/sec 58880 >> [ 5] 7.00-8.00 sec 80.2 MBytes 673 Mbits/sec 58810 >> [ 5] 8.00-9.00 sec 80.2 MBytes 673 Mbits/sec 58830 >> [ 5] 9.00-10.00 sec 77.5 MBytes 650 Mbits/sec 56820 >> - - - - - - - - - - - - - - - - - - - - - - - - - >> [ ID] Interval Transfer Bitrate Jitter >> Lost/Total Datagrams >> [ 5] 0.00-10.00 sec 816 MBytes 684 Mbits/sec 0.000 ms 0/598160 >> (0%) sender >> [ 5] 0.00-10.00 sec 57.2 MBytes 47.9 Mbits/sec 0.272 ms >> 549868/591780 (93%) receiver >> >> As Christiano said, there may be a way for optimizations where solo5 does >> the IO, as here, once the NAT is done, it's only a matter of copying data >> from one page to another. >> This should also be reproducible with a simple nat unikernel such as >> https://github.com/mirage/mirage-nat/tree/main/example . I'll try that >> later. >> >> Best, >> Pierre >> -- >> P. >> >> -- Romain Calascibetta - http://din.osau.re/ --000000000000bf306605e818160f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

Just to inform you t= hat someone (@rand00) started to make a unikernel which permits testing a c= onnection
and seeing the bandwidth. It's useful to test multi= ple backends and probably have more informations about
where is t= he bottleneck.


I= t still is experimental but at least it helps me to find some limitations a= bout my internet connection. It may
be worthwhile to concentrate = our efforts on the latter in order to finally have a tool to replace our go= od old

On Wed, Sep 7,= 2022 at 5:18 PM Christiano F. Haesbaert <haesbaert@haesbaert.org> wrote:
That looks painfully s= low, it's sad that iperf doesn't report packets per second but that= 's around ~5kpps at 1460B/frame.
I'm also surprised your sender= is not saturating gigabit, but I'd have to check how iperf knows it wa= s able to send out the packets, usually no one cares about the sender as lo= ng as you saturate the link.

I think I feel I owe = an explanation since I frequently talk about how solo5 IO is slow but never= explain=C2=A0why/how. First of all I don't mean this as a bashing, I l= ove solo5, the code was never intended to be optimized for network performa= nce. Also take this with a grain of salt, there can be multiple things invo= lved and it might be a bug somewhere completely unrelated, the truth is I h= aven't run tests enough to understand how much of the solo5 IO can be b= lamed, I just repeat this because "there might be nothing wrong".=

I had a quick look at the xen code for the first = time now, and it's quite different from the rest, it has very little to= do with how solo5 does IO the ring management and IO code is in ocaml and = I can't really reason about it without a lot of time.
At this= point I'd try to turn the firewall into an "expensive cable"= just copy packets from input to output and get some idea of the baseline.<= /div>


On Wed, 7 Sept 2022 at 16:25, <pierre.alain@tuta.io> wrote:
Hi all,
Regarding the UDP #pkt/s with qubes-mirage-firewall, I did a quick test (wi= th fortunately no hang or unfortunately as it doesn't give any addition= al clue). As I'm not currently at home I cannot test with my FTTH conne= xion but I think it's still relevant as I managed to get a CPU saturati= on with mirage.

With mirageOS as firewall (CPU is 100%):
$ iperf3 -c lon.speedtest.clouvider.net -p 5203 -u -b 0
Connecting to host lon.speedtest.clouvider.net, port 5203
[=C2=A0 5] local 10.137.0.20 port 56373 connected to 5.180.211.133 port 520= 3
[ ID] Interval=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Transfer=C2=A0=C2=A0=C2=A0=C2=A0 Bitrate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Total Datagrams
[=C2=A0 5]=C2=A0=C2=A0 0.00-1.00=C2=A0=C2=A0 sec=C2=A0 51.9 MBytes=C2=A0=C2= =A0 436 Mbits/sec=C2=A0 38090=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 1.00-2.00=C2=A0=C2=A0 sec=C2=A0 71.1 MBytes=C2=A0=C2= =A0 597 Mbits/sec=C2=A0 52160=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 2.00-3.00=C2=A0=C2=A0 sec=C2=A0 71.8 MBytes=C2=A0=C2= =A0 603 Mbits/sec=C2=A0 52680=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 3.00-4.00=C2=A0=C2=A0 sec=C2=A0 73.7 MBytes=C2=A0=C2= =A0 618 Mbits/sec=C2=A0 54030=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 4.00-5.00=C2=A0=C2=A0 sec=C2=A0 73.8 MBytes=C2=A0=C2= =A0 619 Mbits/sec=C2=A0 54080=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 5.00-6.00=C2=A0=C2=A0 sec=C2=A0 60.0 MBytes=C2=A0=C2= =A0 504 Mbits/sec=C2=A0 44030=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 6.00-7.00=C2=A0=C2=A0 sec=C2=A0 45.0 MBytes=C2=A0=C2= =A0 377 Mbits/sec=C2=A0 32990=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 7.00-8.00=C2=A0=C2=A0 sec=C2=A0 73.6 MBytes=C2=A0=C2= =A0 617 Mbits/sec=C2=A0 53950=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 8.00-9.00=C2=A0=C2=A0 sec=C2=A0 70.8 MBytes=C2=A0=C2= =A0 594 Mbits/sec=C2=A0 51920=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 9.00-10.00=C2=A0 sec=C2=A0 70.6 MBytes=C2=A0=C2=A0 5= 92 Mbits/sec=C2=A0 51760=C2=A0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Transfer=C2=A0=C2=A0=C2=A0=C2=A0 Bitrate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Jitter=C2=A0=C2=A0=C2=A0 Lost/Total Datagrams
[=C2=A0 5]=C2=A0=C2=A0 0.00-10.00=C2=A0 sec=C2=A0=C2=A0 662 MBytes=C2=A0=C2= =A0 556 Mbits/sec=C2=A0 0.000 ms=C2=A0 0/485690 (0%)=C2=A0 sender
[=C2=A0 5]=C2=A0=C2=A0 0.00-10.00=C2=A0 sec=C2=A0 57.2 MBytes=C2=A0 48.0 Mb= its/sec=C2=A0 0.269 ms=C2=A0 437001/478947 (91%)=C2=A0 receiver

with linux as firewall (CPU is around 80%):
$ iperf3 -c lon.speedtest.clouvider.net -p 5203 -u -b 0
Connecting to host lon.speedtest.clouvider.net, port 5203
[=C2=A0 5] local 10.137.0.21 port 49539 connected to 5.180.211.133 port 520= 3
[ ID] Interval=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Transfer=C2=A0=C2=A0=C2=A0=C2=A0 Bitrate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Total Datagrams
[=C2=A0 5]=C2=A0=C2=A0 0.00-1.00=C2=A0=C2=A0 sec=C2=A0 82.2 MBytes=C2=A0=C2= =A0 689 Mbits/sec=C2=A0 60240=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 1.00-2.00=C2=A0=C2=A0 sec=C2=A0 85.7 MBytes=C2=A0=C2= =A0 719 Mbits/sec=C2=A0 62850=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 2.00-3.00=C2=A0=C2=A0 sec=C2=A0 85.2 MBytes=C2=A0=C2= =A0 715 Mbits/sec=C2=A0 62460=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 3.00-4.00=C2=A0=C2=A0 sec=C2=A0 79.0 MBytes=C2=A0=C2= =A0 663 Mbits/sec=C2=A0 57920=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 4.00-5.00=C2=A0=C2=A0 sec=C2=A0 84.8 MBytes=C2=A0=C2= =A0 712 Mbits/sec=C2=A0 62210=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 5.00-6.00=C2=A0=C2=A0 sec=C2=A0 80.7 MBytes=C2=A0=C2= =A0 676 Mbits/sec=C2=A0 59140=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 6.00-7.00=C2=A0=C2=A0 sec=C2=A0 80.3 MBytes=C2=A0=C2= =A0 674 Mbits/sec=C2=A0 58880=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 7.00-8.00=C2=A0=C2=A0 sec=C2=A0 80.2 MBytes=C2=A0=C2= =A0 673 Mbits/sec=C2=A0 58810=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 8.00-9.00=C2=A0=C2=A0 sec=C2=A0 80.2 MBytes=C2=A0=C2= =A0 673 Mbits/sec=C2=A0 58830=C2=A0
[=C2=A0 5]=C2=A0=C2=A0 9.00-10.00=C2=A0 sec=C2=A0 77.5 MBytes=C2=A0=C2=A0 6= 50 Mbits/sec=C2=A0 56820=C2=A0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Transfer=C2=A0=C2=A0=C2=A0=C2=A0 Bitrate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Jitter=C2=A0=C2=A0=C2=A0 Lost/Total Datagrams
[=C2=A0 5]=C2=A0=C2=A0 0.00-10.00=C2=A0 sec=C2=A0=C2=A0 816 MBytes=C2=A0=C2= =A0 684 Mbits/sec=C2=A0 0.000 ms=C2=A0 0/598160 (0%)=C2=A0 sender
[=C2=A0 5]=C2=A0=C2=A0 0.00-10.00=C2=A0 sec=C2=A0 57.2 MBytes=C2=A0 47.9 Mb= its/sec=C2=A0 0.272 ms=C2=A0 549868/591780 (93%)=C2=A0 receiver

As Christiano said, there may be a way for optimizations where solo5 does t= he IO, as here, once the NAT is done, it's only a matter of copying dat= a from one page to another.
This should also be reproducible with a simple nat unikernel such as https://github.com/mirage/mirage-nat/tree/main/examp= le . I'll try that later.

Best,
Pierre
--
P.



--
Romain Calascibetta - http://din.osau.re/
--000000000000bf306605e818160f-- From mirageos-devel-bounces@lists.xenproject.org Wed Sep 07 16:21:33 2022 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 07 Sep 2022 16:21:33 +0000 Received: from list by lists.xenproject.org with outflank-mailman.402015.644001 (Exim 4.92) (envelope-from ) id 1oVxno-0005rn-3P; Wed, 07 Sep 2022 16:21:24 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 402015.644001; Wed, 07 Sep 2022 16:21:24 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVxno-0005rg-0F; Wed, 07 Sep 2022 16:21:24 +0000 Received: by outflank-mailman (input) for mailman id 402015; Wed, 07 Sep 2022 16:21:22 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVxnm-0005rX-CN for mirageos-devel@lists.xenproject.org; Wed, 07 Sep 2022 16:21:22 +0000 Received: from w1.tutanota.de (w1.tutanota.de [81.3.6.162]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 1b3cc8aa-2ec9-11ed-af93-0125da4c0113; Wed, 07 Sep 2022 18:21:20 +0200 (CEST) Received: from w3.tutanota.de (unknown [192.168.1.164]) by w1.tutanota.de (Postfix) with ESMTP id 3D395FA0A89; Wed, 7 Sep 2022 16:21:19 +0000 (UTC) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: 1b3cc8aa-2ec9-11ed-af93-0125da4c0113 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1662567678; s=s1; d=tuta.io; h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Content-Transfer-Encoding:Cc:Cc:Date:Date:In-Reply-To:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:References:Sender; bh=TBTDjPrS3hs5Ya4IkZPJnA06w7ZIJEuZGJ77ik0yPb8=; b=DfBKKbemP73jj+n4rjcfAOFOGFW9vM9wcQYH05kOBZQqgAtVANv0z8VraKMFV2mP 1e3blTjg+1yMmHx2UQNNguJo8pJWWXsyG6jxpBAeQmIsB5oJVzGHP3JB12w9qZVM/NY i0/4wzTkuIg4KlImx0plHWT3+UuCqPOsGo4LQGk2wrOPiA/f+/lF4hVquqMzzzn5IrH Jkg3PZcgXrLWm/mzGe05skrnVi2fzznZB3IW2iTa0A2fAiO/YHGIz8v9w+Dh1XktPgd jOlZGE1e8GAfV45v4d6535E2h2H/H8tthtYxVDjKA0Ak1Le0HFUMHe5BxTxVkOJGru5 bQ0Sp38PGQ== Date: Wed, 7 Sep 2022 18:21:18 +0200 (CEST) From: pierre.alain@tuta.io To: "Christiano F. Haesbaert" Cc: mirageos-devel Message-ID: In-Reply-To: References: <54f22a23-693a-d86f-3912-6695c2f540c3@mehnert.org> <47a2c4d9-40e0-9ff7-97e6-bb378c48d5d1@mehnert.org> Subject: Re: upcoming MirageOS meeting 2022-09-07 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Sep 7, 2022, 17:18 by haesbaert@haesbaert.org: > That looks painfully slow, it's sad that iperf doesn't report packets per= second but that's around ~5kpps at 1460B/frame. > I'm also surprised your sender is not saturating gigabit, but I'd have to= check how iperf knows it was able to send out the packets, usually no one = cares about the sender as long as you saturate the link. > This connexion is shared in the hotel and not as fast as a home one but I d= on't have anything better :) For the giga (un-)saturation, this probably can be explained by the sys-net= vm where packets also have to transit in both cases (and take its taxe). > I think I feel I owe an explanation since I frequently talk about how sol= o5 IO is slow but never explain=C2=A0why/how. First of all I don't mean thi= s as a bashing, I love solo5, the code was never intended to be optimized f= or network performance. Also take this with a grain of salt, there can be m= ultiple things involved and it might be a bug somewhere completely unrelate= d, the truth is I haven't run tests enough to understand how much of the so= lo5 IO can be blamed, I just repeat this because "there might be nothing wr= ong". > I didn't take your comment, in any case, as a bashing against solo5 but as = an idea where to look for evolution and improvements, which, to me, is the = right spirit :) > I had a quick look at the xen code for the first time now, and it's quite= different from the rest, it has very little to do with how solo5 does IO t= he ring management and IO code is in ocaml and I can't really reason about = it without a lot of time. > At this point I'd try to turn the firewall into an "expensive cable" just= copy packets from input to output and get some idea of the baseline. > > Not sure if you mean xen code as the hypersivor or mirage-xen code. The sam= e xen code (as the hypervisor) is involved in both linux and mirage cases a= nd should have the same behavior. Best, Pierre From mirageos-devel-bounces@lists.xenproject.org Wed Sep 07 16:40:42 2022 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 07 Sep 2022 16:40:42 +0000 Received: from list by lists.xenproject.org with outflank-mailman.402094.644094 (Exim 4.92) (envelope-from ) id 1oVy6Q-0004Wx-JA; Wed, 07 Sep 2022 16:40:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 402094.644094; Wed, 07 Sep 2022 16:40:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVy6Q-0004Wq-Fj; Wed, 07 Sep 2022 16:40:38 +0000 Received: by outflank-mailman (input) for mailman id 402094; Wed, 07 Sep 2022 16:40:36 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVy6O-0004We-Ob for mirageos-devel@lists.xenproject.org; Wed, 07 Sep 2022 16:40:36 +0000 Received: from mail.mehnert.org (mail.mehnert.org [213.73.89.200]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id cba7017d-2ecb-11ed-af93-0125da4c0113; Wed, 07 Sep 2022 18:40:35 +0200 (CEST) Received: from [192.168.42.80] (dslb-178-008-099-255.178.008.pools.vodafone-ip.de [178.8.99.255]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "hannes@mehnert.org", Issuer "mehnert root CA" (not verified)) by mail.mehnert.org (Postfix) with ESMTPS id 00A9CA2BE for ; Wed, 7 Sep 2022 18:40:33 +0200 (CEST) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: cba7017d-2ecb-11ed-af93-0125da4c0113 Message-ID: Date: Wed, 7 Sep 2022 18:40:33 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Subject: Re: upcoming MirageOS meeting 2022-09-07 Content-Language: en-US To: mirageos-devel@lists.xenproject.org References: <54f22a23-693a-d86f-3912-6695c2f540c3@mehnert.org> <47a2c4d9-40e0-9ff7-97e6-bb378c48d5d1@mehnert.org> From: Hannes Mehnert In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Eventually related is this old issue in mirage: https://github.com/mirage/mirage/issues/240 On 07/09/2022 17:18, Christiano F. Haesbaert wrote: > That looks painfully slow, it's sad that iperf doesn't report packets per > second but that's around ~5kpps at 1460B/frame. > I'm also surprised your sender is not saturating gigabit, but I'd have to > check how iperf knows it was able to send out the packets, usually no one > cares about the sender as long as you saturate the link. > > I think I feel I owe an explanation since I frequently talk about how solo5 > IO is slow but never explain why/how. First of all I don't mean this as a > bashing, I love solo5, the code was never intended to be optimized for > network performance. Also take this with a grain of salt, there can be > multiple things involved and it might be a bug somewhere completely > unrelated, the truth is I haven't run tests enough to understand how much > of the solo5 IO can be blamed, I just repeat this because "there might be > nothing wrong". > > I had a quick look at the xen code for the first time now, and it's quite > different from the rest, it has very little to do with how solo5 does IO > the ring management and IO code is in ocaml and I can't really reason about > it without a lot of time. > At this point I'd try to turn the firewall into an "expensive cable" just > copy packets from input to output and get some idea of the baseline. > > > On Wed, 7 Sept 2022 at 16:25, wrote: > >> Hi all, >> Regarding the UDP #pkt/s with qubes-mirage-firewall, I did a quick test >> (with fortunately no hang or unfortunately as it doesn't give any >> additional clue). As I'm not currently at home I cannot test with my FTTH >> connexion but I think it's still relevant as I managed to get a CPU >> saturation with mirage. >> >> With mirageOS as firewall (CPU is 100%): >> $ iperf3 -c lon.speedtest.clouvider.net -p 5203 -u -b 0 >> Connecting to host lon.speedtest.clouvider.net, port 5203 >> [ 5] local 10.137.0.20 port 56373 connected to 5.180.211.133 port 5203 >> [ ID] Interval Transfer Bitrate Total Datagrams >> [ 5] 0.00-1.00 sec 51.9 MBytes 436 Mbits/sec 38090 >> [ 5] 1.00-2.00 sec 71.1 MBytes 597 Mbits/sec 52160 >> [ 5] 2.00-3.00 sec 71.8 MBytes 603 Mbits/sec 52680 >> [ 5] 3.00-4.00 sec 73.7 MBytes 618 Mbits/sec 54030 >> [ 5] 4.00-5.00 sec 73.8 MBytes 619 Mbits/sec 54080 >> [ 5] 5.00-6.00 sec 60.0 MBytes 504 Mbits/sec 44030 >> [ 5] 6.00-7.00 sec 45.0 MBytes 377 Mbits/sec 32990 >> [ 5] 7.00-8.00 sec 73.6 MBytes 617 Mbits/sec 53950 >> [ 5] 8.00-9.00 sec 70.8 MBytes 594 Mbits/sec 51920 >> [ 5] 9.00-10.00 sec 70.6 MBytes 592 Mbits/sec 51760 >> - - - - - - - - - - - - - - - - - - - - - - - - - >> [ ID] Interval Transfer Bitrate Jitter Lost/Total >> Datagrams >> [ 5] 0.00-10.00 sec 662 MBytes 556 Mbits/sec 0.000 ms 0/485690 >> (0%) sender >> [ 5] 0.00-10.00 sec 57.2 MBytes 48.0 Mbits/sec 0.269 ms >> 437001/478947 (91%) receiver >> >> with linux as firewall (CPU is around 80%): >> $ iperf3 -c lon.speedtest.clouvider.net -p 5203 -u -b 0 >> Connecting to host lon.speedtest.clouvider.net, port 5203 >> [ 5] local 10.137.0.21 port 49539 connected to 5.180.211.133 port 5203 >> [ ID] Interval Transfer Bitrate Total Datagrams >> [ 5] 0.00-1.00 sec 82.2 MBytes 689 Mbits/sec 60240 >> [ 5] 1.00-2.00 sec 85.7 MBytes 719 Mbits/sec 62850 >> [ 5] 2.00-3.00 sec 85.2 MBytes 715 Mbits/sec 62460 >> [ 5] 3.00-4.00 sec 79.0 MBytes 663 Mbits/sec 57920 >> [ 5] 4.00-5.00 sec 84.8 MBytes 712 Mbits/sec 62210 >> [ 5] 5.00-6.00 sec 80.7 MBytes 676 Mbits/sec 59140 >> [ 5] 6.00-7.00 sec 80.3 MBytes 674 Mbits/sec 58880 >> [ 5] 7.00-8.00 sec 80.2 MBytes 673 Mbits/sec 58810 >> [ 5] 8.00-9.00 sec 80.2 MBytes 673 Mbits/sec 58830 >> [ 5] 9.00-10.00 sec 77.5 MBytes 650 Mbits/sec 56820 >> - - - - - - - - - - - - - - - - - - - - - - - - - >> [ ID] Interval Transfer Bitrate Jitter Lost/Total >> Datagrams >> [ 5] 0.00-10.00 sec 816 MBytes 684 Mbits/sec 0.000 ms 0/598160 >> (0%) sender >> [ 5] 0.00-10.00 sec 57.2 MBytes 47.9 Mbits/sec 0.272 ms >> 549868/591780 (93%) receiver >> >> As Christiano said, there may be a way for optimizations where solo5 does >> the IO, as here, once the NAT is done, it's only a matter of copying data >> from one page to another. >> This should also be reproducible with a simple nat unikernel such as >> https://github.com/mirage/mirage-nat/tree/main/example . I'll try that >> later. >> >> Best, >> Pierre >> -- >> P. >> >> > From mirageos-devel-bounces@lists.xenproject.org Tue Sep 20 14:15:22 2022 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 20 Sep 2022 14:15:22 +0000 Received: from list by lists.xenproject.org with outflank-mailman.409455.652405 (Exim 4.92) (envelope-from ) id 1oae1n-0000YC-9G; Tue, 20 Sep 2022 14:15:11 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 409455.652405; Tue, 20 Sep 2022 14:15:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oae1n-0000Y5-5y; Tue, 20 Sep 2022 14:15:11 +0000 Received: by outflank-mailman (input) for mailman id 409455; Tue, 20 Sep 2022 14:15:10 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oae1m-0000Xz-9H for mirageos-devel@lists.xenproject.org; Tue, 20 Sep 2022 14:15:10 +0000 Received: from mail.mehnert.org (mail.mehnert.org [213.73.89.200]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9f8e4bb0-38ee-11ed-bad8-01ff208a15ba; Tue, 20 Sep 2022 16:15:07 +0200 (CEST) Received: from [192.168.42.80] (dslb-094-223-120-014.094.223.pools.vodafone-ip.de [94.223.120.14]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "hannes@mehnert.org", Issuer "mehnert root CA" (not verified)) by mail.mehnert.org (Postfix) with ESMTPS id D53399230 for ; Tue, 20 Sep 2022 16:15:04 +0200 (CEST) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: 9f8e4bb0-38ee-11ed-bad8-01ff208a15ba Message-ID: Date: Tue, 20 Sep 2022 16:15:04 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Content-Language: en-US To: mirageos-devel@lists.xenproject.org From: Hannes Mehnert Subject: upcoming MirageOS meeting 2022-09-21 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hey, tomorrow from 14:00 CEST we'll have the next mirageos meeting at https://whereby.com/ocamlllabs (notes https://pad.data.coop/aOHbvgwaR46JQXMbqgoKlQ#) If you've agenda items, feel free to put them in the pad. Best and see you tomorrow, Hannes From mirageos-devel-bounces@lists.xenproject.org Wed Sep 21 13:26:45 2022 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 21 Sep 2022 13:26:45 +0000 Received: from list by lists.xenproject.org with outflank-mailman.409797.652768 (Exim 4.92) (envelope-from ) id 1oazkG-0004lt-Qn; Wed, 21 Sep 2022 13:26:32 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 409797.652768; Wed, 21 Sep 2022 13:26:32 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oazkG-0004lm-O0; Wed, 21 Sep 2022 13:26:32 +0000 Received: by outflank-mailman (input) for mailman id 409797; Wed, 21 Sep 2022 13:26:31 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oazkF-0004lg-Ha for mirageos-devel@lists.xenproject.org; Wed, 21 Sep 2022 13:26:31 +0000 Received: from mail.mehnert.org (mail.mehnert.org [213.73.89.200]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ff2d8e3d-39b0-11ed-9647-05401a9f4f97; Wed, 21 Sep 2022 15:26:30 +0200 (CEST) Received: from [192.168.42.80] (dslb-094-223-109-217.094.223.pools.vodafone-ip.de [94.223.109.217]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "hannes@mehnert.org", Issuer "mehnert root CA" (not verified)) by mail.mehnert.org (Postfix) with ESMTPS id E2FD5F81A for ; Wed, 21 Sep 2022 15:26:26 +0200 (CEST) X-BeenThere: mirageos-devel@lists.xenproject.org List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Precedence: list Sender: "MirageOS-devel" X-Inumbo-ID: ff2d8e3d-39b0-11ed-9647-05401a9f4f97 Message-ID: <62b4c3f8-0d6b-4017-8e46-5f87146d1636@mehnert.org> Date: Wed, 21 Sep 2022 15:26:26 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Subject: Re: upcoming MirageOS meeting 2022-09-21 Content-Language: en-US From: Hannes Mehnert To: mirageos-devel@lists.xenproject.org References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hello, below is the notes we took during the meeting. The next one is in three weeks (Oct 12th). Best, Hannes # mirage 2022-09-21 participants: christiano, reynir, hannes, thomas leonard, rand, anil, dave, mindy ## mirage-kv changes - released as mirage-kv 5.0 - get_partial, size, set_partial, rename - at the moment, none mirage-kv implementation supports the new interface - ocaml-tar has a PR that makes it a Mirage_kv.RW (e.g. used by https://git.robur.io/robur/opam-mirror) - TODO: setup and write a blog entry (for mirage.io) about opam-mirror ## mirage-flow changes - dave scott approved the shutdown PR (https://github.com/mirage/mirage-flow/pull/48) - read semantics (and buffer ownership, https://github.com/mirage/mirage-flow/issues/46) - reading eio rationale (https://github.com/ocaml-multicore/eio/blob/main/doc/rationale.md#indicating-end-of-file), the `` Ok `Eof `` signalling is bad for performance -- TODO: benchmark this claim ## use eio in mirageos? - just use eio as a dropin, select a backend - for performance and getting rid of functors, plus direct style - lucas worked at it: https://github.com/TheLortex/mirage-monorepo - the eio core is independent of Unix - to use eio in MirageOS, solo5 needs some support (christiano will work on it) - instead of Lwt_main, it could run Eio_main (and use Lwt_eio), to migrate incrementally - what is needed for OCaml 5 & MirageOS? -- a pthread implementation (in ocaml-solo5?) -- an eio backend, and hooking up the OCaml TCP/IP stack as a net implementation -- domain support (and eventually domain overcommit) [eventually preemption?] -- structured concurrency (for eio, for mirage) ## mirage-block sector alignment - mirage-block read and write is done aligned on sectors - on unix, if the image file is not sector-aligned, some 0s are added at the end - on solo5, only full sectors can be read and written (the last bytes couldn't be read or written) - detour: mirage-block-unix has a buffered option - https://github.com/mirage/mirage-block-unix/pull/117 - https://github.com/Solo5/solo5/pull/527 --> good to merge and release :D ## ocaml-mbr - also got revived and revised - ready to release (but is the check "at most one active partition needed") ## tls-eio - https://github.com/mirleft/ocaml-tls/pull/451 - please review and merge - there's a kludge to remove tls-async from CI for OCaml 5.0 ## move cstruct.t to use bytes (at least for OCaml 5.0) - faster and slab allocated ## eio webserver+LE? (anil) - we're pretty close: cohttp, tls - someone has a eio&let's encrypt stuff? - mindy is looking it up - tlstunnel, contruno, unipi, (canopy) ## data center from packet.net is demolished, we need to move mirage.io by end of november ## next call: Oct 12th 14:00 CEST