Xen project Mailing List

This is the report of the MirageOS meeting (as available here: https://pad.data.coop/wGS4r8RyTKqQ73mcw7FrwA?both)

# MirageOS community call

### 25/03/2024 (next)

- What do we want Mirage5 to look like? And when do we want to make a release?
- runtime args
- Ocaml 5 support
- dune pkg?
- No more Lwt?

### 11/03/2024
- Present: Kaushik, Samuel, dinosaure, Magnus, Pierre, Alfred, Thomas, Hannes, Shakthi.
- PR review
- new API for runtime arguments: https://github.com/mirage/mirage/pull/1493 -- ready to merge. Late feedback is useful, please test and review if possible - plan is to merge today. After it's merged it's also possible to make changes (although before merge is better)
- solo5+ocaml5: https://github.com/mirage/ocaml-solo5/pull/124
- samuel had sound issues, post update on pr and discuss next time
- current state: reworked the build process and moved to support OCaml 5.2, not yet ready for a review
- Unikraft+mirage plans
- tarides has grant with unikraft to work with them on mirage, still early
- current plan is to first look at solo5 and try to unlock what's blocking ocaml5 and look at the build system hacks and see if we can tweak them to help with unikraft integration.
- see if we can reuse small C subset of unikraft instead of custom solo5 calls, starting slowly
- two main blockers/unknowns;
- if we use unikraft, what do we gain over solo5? Maybe performance, as we can do io without vmexit. But we don't know what we lose in terms of security. Security roadmap for unikraft, but still wip.
- unikraft comes with existing tooling - have to use a cli tool to generate a small c project that contains what you need. Cli tool will link application etc. Similar to what mirage does, so we have to figure out which tools drives which part of the process. Have to think about this to get a nice user experience
- goal is not to drop solo5, it's working well and we don't want to completely switch to unikraft - just want to provide a new backend and clean up solo5 bindings in the process
- hannes: main goal is performance? thomas: goal is to extend the device model, want to support more types of devices - e.g. what about things like pci and gpu passthrough. This doesn't fit with the solo5 model. Performance is also interesting; if we build a unikernel focussed on performance, how much further can we go. But we don't want to lose security benefits. Unikraft has a team that maintains it and can help develop new features
- magnus: timeframe? thomas: this quarter to explore, figure out what's possible. Expect to have some initial targets at end of year
- magnus: new architectures? thomas: exploring what pthreads and smp can gives us with ocaml5
- alfred: what is the status of solo5, likely to evolve with pthreads? thomas: currently maintained, but smp may nto fit the philosophy of solo5. romain: I maintain solo5 and I agree that we don't want to have multicore support there, we don't have the resources to make it. Also, do we actually need multiple cores for unikernels? In my view, no, one core is enough. Would like to have ocaml5 support (with effects and so on), but multicore not important. Thomas: I think that makes sense, keep the solo5 runtime super simple for security reasons - then experiment with smp support with unikraft and mirage.
- How can we improve the security story around MirageOS? What are the security issues with mirage?
- thomas: when we talk about mirage, usually we say that we are more secure than other solutions; type safety and memory safety. But then a question is whether everything is ocaml, what about c bindings etc? I started aggregating a list of security features in solo5, it's important that we demonstrate that we focus on security not only in ocaml. It would be nice to have a place where we talk about security end to end in mirage.
- alfred: important to document, when we had security review of includeos this would have helped us by providing context for test scripts etc
- hannes: I'm also interested in the security. Several security audits of mirage, if these are public, we should link to them. Alfred, could be interesting to see the review report for includeos as well. Was also some work on improving issues in solo5 that are documented in the changelog.
- thomas: yes, would be great to put what is currently documented in changelogs, PRs etc in one location
- IncludeOS / Mirage
- alfred: experimenting with building a high performance game server as a unikernel. Interested in iouring experiments and how to support thousands of players. Not possible currently with solo5. From current experiments seems really hard to be able to send enough data in a single process. Multicore also has a cost, so not clear that it's the only best/solution.
- hannes: have you looked at the solo5 virtio code? alfred: yes, but I think you still have the functional interface where you have to do one packet at the time. Solo5 doesn't expose the ring buffers into mirage, I think you need to expose virtio to mirage or iouring. Hannes: yes, the api requires calls to send
- thomas: we have sendv some places? hannes: only at the mirage-flow layer (so, TCP / TLS), not at the lower layers
- solo5 netmap support: https://github.com/solo5-netmap/solo5
- VPN (hannes)
- hannes: we have been working on vpn lately. working on reducing allocations, saw performance improvements also in mirage-crypto. Robur also interested in improving performance in general, but focussing on single thread first
- performance numbers for MirageVPN: https://github.com/robur-coop/miragevpn/issues/206#issuecomment-1980849179
- mirage-crypto performance: https://discuss.ocaml.org/t/ann-mirage-crypto-0-11-3-with-more-speed-for-elliptic-curves-and-the-future-roadmap-of-mirage-crypto/ and announcement (with numbers https://github.com/mirage/mirage-crypto/releases/tag/v0.11.3)
- alfred: shared memory available for multiple unikernels to coordinate? hannes: not in solo5 - you could use block devices or something like that, but no real interface available - available in xen. Could be useful to add with the right usecase. Thomas: perhaps something to add outside solo5? alfred: we could do polling for high performance, then block device might work
- Next meeting will be in two weeks - March 25th

### 26/02/2024
- Present: dinosaure, Pierre, Hannes, Virgile, Thomas G. Thomas L., Samuel, Shakthi, Magnus
- Agenda:
- Meta:
- Should we use Zoom or something else?
- Pierre: Some universities cannot use Zoom on Linux.
- Dinosaure: Prefers to use Jitsi.
- Virgile: Can use Renater (which uses Jitsi) if someone has a renater account.
- Pierre-Alain will investigate.
- Hannes: there is the public instance jit.si where no registration is necessary (as used e.g. in the opam-repository meetings)
- Should we use https://pad.data.coop/ or something else? => Pad seems good!
- Anyone willing to help organising these calls? (send , taking notes, etc.)
- Next meeting will be organised by Magnus :-) Thanks (and Virgile is also happy to help)
- MirageOS retreat update?
- 16 people signed up! early-birds registration seems to work fine
- still space available - even for a few days and/or for the week-end
- mix of people that have been there before, and new people!
- PR updates
- solo5+ocaml5: https://github.com/mirage/ocaml-solo5/pull/124
- Samuel and Fabrice are trying to revive the PR. Cleaning up the x-compiler story for the OCaml compiler in a state that could be upstream. The goal is to clean up all the pieces are glued together. Still trying to understand how the various details are working (like memory allocation). Target: OCaml 5.2 (which include compaction) as many things have changed.
- Hannes: there are 2 or 3 PRs that pretend to do something with OCaml 5. Some of them have a lots of comments from Christiano and others. Is there a plan to merge/review those comments too? Some stuff implemented in C were a bit brittle. Would be sad if we lose these reviews.
- Samuel: plan is probably to open a new PR that adress all of the comments (#122, #124, #129). Many things have changed in 5.2.
- Dinosaure: what's the issue with x-compilation? Samuel: many ways in the way ocaml-solo5 works is to patch the OCaml build system (with a few things broken as a result). Things are very brittle. We would like to have something more solid. Dinosaure: this seems independant? Could we decouple those concerns? Samuel: indeed. currently trying to understand how things work but the idea is to reduce the maintenance as well. and to understand what is needed for solo5 vs. x-compilation. Dinosaure: see ocaml-solo5#123 (most issues are related `configure`). Samuel: will talk to Sebastien next
- mirage runtime keys v2: https://github.com/mirage/mirage/pull/1493
- Thomas:
- use cmdliner directly, instead of a custom fork
- keys are split into configuration time keys and runtime keys
- the runtime ones are only defined at runtime
- it is a breaking big change
- Hannes:
- configure-time: you select what libraries you want to use in the unikernel (keys)
- runtime-time: you select some runtime parameters (ip address, etc.)
- better type for key
- issue: type errors, locations in generated code
- Happy with the current state but do not have time to review fully
- Thomas:
- updated existing unikernels
- worked on error locations (you need to pass `__LOC__`)
- there's a OCaml compiler patch that does this automatically
- ~all examples and documentation will need to be updated to reflect the new interface: big workload
- Hannes: can we remove the dependency between mirage and mirage-runtime
- Dinosaure: the idea looks good, and the possibility to add custom types (via cmdliner) is nice
- Thomas: will make progress for the next call
- MirageOS relies on "opam-monorepo", what is the schedule switching to "dune pkg" (last was "Q1/2024" - anyone actively working on that)?
- the dune and opam team worked for the last year to include opam-lib into dune
- dune will be able to compute (make a lock file), download packages it needs locally, and compile each opam package (even if the package is not using dune)
- single tool to define dependencies, whenever you modify the dune file, the opam packages are updated
- rules will be cached
- it is similar to opam-monorepo
- packages need to be relocatable (not all are, e.g. the compiler)
- there'll be a small overlay (similar to opam-monorepo overlay) for a small set of packages that are not possible to build with "dune pkg"
- timeline: first alpha was planned end of Q1, now scheduled for May
- what needs to be done for mirage? we haven't tested it yet. need to discuss and test once an alpha is around (thomas will test even earlier)
- for now we need to keep opam-monorepo
- Thomas: are there urgent pains to address?
- Hannes: hard to use old versions of dunes, ...
- Hannes: Is there any specification of how dune pkg will download/organize etc., for the purpose of reproducible builds? -- Thomas: it created a lock file and downloads using opam-lib
- Hannes: carton/git issue with opam-monorepo -- Thomas: this will be fixed in "dune pkg"
- Dune pkg milestone: https://github.com/ocaml/dune/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Package+Management+MVP%22
- dune+orb (reproducible build integration): https://github.com/ocaml/dune/issues/9548
- Performance considerations for MirageOS (let's take mirage-crypto as example)
- EC NIST curves pre-computed tables -- https://github.com/mirage/mirage-crypto/pull/191 shows a speedup of 4x
- mirage-crypto symmetric cipher (AES-GCM / Poly1305-Chacha20) around 10x slower than OpenSSL (see https://github.com/mirage/mirage-crypto/pull/203, a speedup of ~2.5x for chacha using string/bytes)
- cstruct.t vs bytes -- https://blog.robur.coop/articles/speeding-ec-string.html shows a speedup of 2.5x
- or replace the underlying structure of cstruct from bigarrays to bytes? "Just" have to take care of IO-pages which still need aligned addresses
- for Xen interfaces we need page-aligned non-moving memory areas
- Dinosaure: don't use _systematically_ Cstruct‧t and probably we should use bytes more systematically and asking ourselves about particularities of bigarray
- Thomas: Patrick has started a Bstruct library: https://github.com/ocaml-multicore/ocaml-uring/pull/101
- mirage-flow + shutdown (https://github.com/mirage/mirage-flow/pull/48)
- Hannes: released to opam-repository as mirage-flow 4.x
- Hannes: anyone eager to review: https://github.com/mirage/mirage-tcpip/pull/512
- Thomas: will ping Dave
- Next meeting - https://meet.jit.si/mirageos-call, 9 CET in two weeks (March 11th)

### Moved to next meeting
- Unikraft+mirage plans
- How can we improve the security story around MirageOS? What are the security issues with mirage?
- What do we want Mirage5 to look like?
- Ocaml 5 support
- No more Lwt?
- IncludeOS / Mirage

Re: MirageOS meeting 2024-03-11 - 9:00 CET