Re: [MirageOS-devel] Patches for building mirage libraries with `opam2nix`

[Tim Cuthbertson <tim@xxxxxxxxxxx>]

On Fri, Sep 4, 2015 at 8:22 PM, Thomas Leonard <talex5@xxxxxxxxx> wrote:
> On 1 September 2015 at 08:27, Tim Cuthbertson <tim@xxxxxxxxxxx> wrote:
>> Hi all,
>>
>> A little while ago I managed to get mirage's xen libraries and
>> dependencies building in nix [0]. This worked, but I didn't fancy
>> keeping those packages up to date manually. So I also started working
>> on `opam2nix`[1], a tool which generates nix expressions from an opam
>> repository. It's still work in progress, but has managed to build most
>> things I've thrown at it so far.
>>
>> The main problem is when build scripts assume that they'll be
>> installed into `opam config var prefix`/lib/<library>, or that
>> pkgconfig specs will be found in `opam config var
>> prefix`/lib/pkgconfig. In the case of `opam2nix`, dependencies can be
>> found via `ocamlfind query`, while the destination will be a
>> completely different path which is populated in $PREFIX,
>> $OCAMLFIND_DESTDIR and opam variables (like %{lib}% etc.)
>>
>> I've made a number of changes to make these build scripts more lenient
>> - e.g. rather than always overriding $PKG_CONFIG_PATH, they'll only do
>> that if all needed dependencies aren't already present. And also
>> respecting $PREFIX if it's set (but falling back to `opam config var
>> prefix` if not):
>>
>> --------
>>
>> gmp-xen:
>> https://github.com/gfxmonk/opam-repository/commits/mirage-fixes
>>
>> io-page:
>> https://github.com/gfxmonk/io-page/commits/master
>>
>> mirage-platform:
>> https://github.com/gfxmonk/mirage-platform/commits/master
>> (note: the second of those two commits
>> [6121921ef6e666f021a61f3570840108927f90d8] removes something that I
>> _think_ was unnecessary, but it's worth verifying that with someone
>> who knows more about how the libraries depend on each other)
>>
>> tcpip:
>> https://github.com/gfxmonk/mirage-tcpip/commits/master
>>
>> zarith-xen:
>> I had to rewrite the zarith-xen build script, because I couldn't come
>> up with a simple enough change which would support both the `opam` and
>> `opam2nix` cases. So this is less of a patch than a note that it
>> doesn't currently work, and maybe someone will have better ideas about
>> how to make it work:
>> https://github.com/gfxmonk/opam2nix/blob/71887935dc30a744aa26f1f5af75678355d2d74e/src/nix/overrides/zarith-xen/install.sh
>>
>> --------
>>
>> It'd be great if the maintainers of these libraries are happy to merge
>> my changes, or otherwise discuss other ways of making these libraries
>> build in an I-can't-believe-it's-not-opam environment ;). I thought
>> I'd discuss it here rather than creating individual pull requests,
>> because this should provide a lot more context.
>>
>> The main downside is that these changes make debugging a little
>> trickier, since the build scripts are slightly more complex. Also, it
>> might mean that a dependency could be picked up from a system package
>> rather than from `opam`.
>>
>> It may also be a better idea in general to just feed more information
>> to the scripts directly from the `opam` file (via %{lib}% and friends)
>> rather than have them assume too much about how they'll be installed.
>> I didn't want to go down this route because it makes for more drastic
>> changes, but I'm happy to try it if you're game.
>
> Yeah, it is a bit concerning that system packages will override the
> current opam switch's packages. The problem is that local builds of
> OCaml programs are typically started by simply running "make", so
> there's no way to know whether the user wants to use opam or not.
>
> I agree that build scripts shouldn't be calling back into opam
> explicitly (as these do currently).
>
> Can we get "opam config env" to set PKG_CONFIG_PATH (as it does for
> e.g. MANPATH already)? Then we assume that a user who initialises opam
> in their login script prefers to use opam's build environment (which
> is already true for other aspects of the environment).

That sounds good to me. And opam's build process would need to set
this as well, right? i.e. I'm assuming all of these should work:

1) user is in `opam config env` shell, runs `make` -> only uses opam deps
2) user is in system shell, runs `make` -> system deps
3) user is in system shell, runs `opam install this-local-package` ->
only uses opam deps

Hopefully the same opam code path can cover both (1) and (3).

> If you build a
> Nix package in this environment, would it build that in a clean
> environment?

Yep, nix builds are in a clean environment, so PKG_CONFIG_PATH during
a nix build contains only your (transitive) build dependencies.

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel