Re: A unikernel for building unikernels

[Hannes Mehnert <hannes@xxxxxxxxxxx>]

Dear Nick,

On 16/12/2024 11:42, Nick Betteridge wrote:
> Does anyone know of any unikernels that can build unikernels for the 
> local platform, using something like ocaml-monorepo and, perhaps, irmin?
I'm afraid I'm not aware of such a unikernel - though that'd be great, 
one that is able to compile itself (let's keep the C objects for the 
OCaml runtime as given).
> Hannes has suggested looking at something like ocamlnat [1], which would 
What I meant was not that ocamlnat, but the one provided by the OCaml 
compiler distribution -- on the 4.14 branch the toplevel/native 
directory (https://github.com/ocaml/ocaml/tree/4.14/toplevel/native) -- 
make ocamlnat
> be one way of tackling the issue, but I was wondering if anyone had 
> considered or implemented any other approaches to solve this  - I would 
> really love to move away from using the relatively bulky Docker 
> infrastructure and associated maintenance.
What we use is actually docker (or FreeBSD jails):
- we use the `mirage configure` generated opam file(s) as input (in our 
unikernel-repo https://git.robur.coop/robur/unikernel-repo)
- we then use builder (https://github.com/robur-coop/builder) as a 
client/server architecture to schedule builds from the scheduler 
(server) to the workers (that are polling for builds)
  - here, we once say 'builder-client orb-build hello 
mirage-unikernel-hello-hvt' (where "hello" is the job name, and 
"mirage-unikernel-hello-hvt" the opam name), and then get a daily build 
thereof
  - manually we schedule a build with "builder-client execute hello"
  - the builder-worker is running within a jail/docker container to 
have a think layer of dependencies
- the builder driver is orb (https://github.com/robur-coop/orb) that 
basically uses the opam API to execute the build while preserving 
information required for reproducing the build
Now, another stepping stone is builder-web 
(https://git.robur.coop/robur/builder-web/) where builder uploads the 
builds and we have a nice web interface (https://builds.robur.coop) with 
failed builds (https://builds.robur.coop/failed-builds), some 
visualizations (binary size, dependencies), a database of the builds (we 
can compare builds - i.e. 
https://builds.robur.coop/compare/c43f6d82-9a28-4482-94e9-cf14e50816c0/920b68e8-1ed7-47ee-a5a4-eacef52e1f92 
-- and if you look on the first build, it resulted in a 7.75MB hvt 
image, the second only a 6.97MB -- while the difference is an update of 
charrua that dropped sexplib dependencies), and finally also binary 
package repositories (pkg.robur.coop / apt.robur.coop) for the utilities 
just mentioned: orb. builder, builder-web, (and also albatross, solo5).
So, on our build machines we use these package repositories to "apt 
update && apt upgrade". Bootstrapping may be a bit tedious - that's why 
we provide the binaries for common platforms / OS (if yours is missing, 
don't hesitate to complain).

Back to the 'self-building MirageOS unikernel' - I guess apart from the 
C compiler, and the linking step this could be achieved with a 
reasonable effort.

Hope that helped,

Hannes