Re: [MirageOS-devel] Patches for building mirage libraries with `opam2nix`

[Thomas Leonard <talex5@xxxxxxxxx>]

On 4 September 2015 at 12:29, Tim Cuthbertson <tim@xxxxxxxxxxx> wrote:
> On Fri, Sep 4, 2015 at 8:22 PM, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>> On 1 September 2015 at 08:27, Tim Cuthbertson <tim@xxxxxxxxxxx> wrote:
>>> Hi all,
>>>
>>> A little while ago I managed to get mirage's xen libraries and
>>> dependencies building in nix [0]. This worked, but I didn't fancy
>>> keeping those packages up to date manually. So I also started working
>>> on `opam2nix`[1], a tool which generates nix expressions from an opam
>>> repository. It's still work in progress, but has managed to build most
>>> things I've thrown at it so far.
>>>
>>> The main problem is when build scripts assume that they'll be
>>> installed into `opam config var prefix`/lib/<library>, or that
>>> pkgconfig specs will be found in `opam config var
>>> prefix`/lib/pkgconfig. In the case of `opam2nix`, dependencies can be
>>> found via `ocamlfind query`, while the destination will be a
>>> completely different path which is populated in $PREFIX,
>>> $OCAMLFIND_DESTDIR and opam variables (like %{lib}% etc.)
>>>
>>> I've made a number of changes to make these build scripts more lenient
>>> - e.g. rather than always overriding $PKG_CONFIG_PATH, they'll only do
>>> that if all needed dependencies aren't already present. And also
>>> respecting $PREFIX if it's set (but falling back to `opam config var
>>> prefix` if not):
>>>
>>> --------
>>>
>>> gmp-xen:
>>> https://github.com/gfxmonk/opam-repository/commits/mirage-fixes
>>>
>>> io-page:
>>> https://github.com/gfxmonk/io-page/commits/master
>>>
>>> mirage-platform:
>>> https://github.com/gfxmonk/mirage-platform/commits/master
>>> (note: the second of those two commits
>>> [6121921ef6e666f021a61f3570840108927f90d8] removes something that I
>>> _think_ was unnecessary, but it's worth verifying that with someone
>>> who knows more about how the libraries depend on each other)
>>>
>>> tcpip:
>>> https://github.com/gfxmonk/mirage-tcpip/commits/master
>>>
>>> zarith-xen:
>>> I had to rewrite the zarith-xen build script, because I couldn't come
>>> up with a simple enough change which would support both the `opam` and
>>> `opam2nix` cases. So this is less of a patch than a note that it
>>> doesn't currently work, and maybe someone will have better ideas about
>>> how to make it work:
>>> https://github.com/gfxmonk/opam2nix/blob/71887935dc30a744aa26f1f5af75678355d2d74e/src/nix/overrides/zarith-xen/install.sh
>>>
>>> --------
>>>
>>> It'd be great if the maintainers of these libraries are happy to merge
>>> my changes, or otherwise discuss other ways of making these libraries
>>> build in an I-can't-believe-it's-not-opam environment ;). I thought
>>> I'd discuss it here rather than creating individual pull requests,
>>> because this should provide a lot more context.
>>>
>>> The main downside is that these changes make debugging a little
>>> trickier, since the build scripts are slightly more complex. Also, it
>>> might mean that a dependency could be picked up from a system package
>>> rather than from `opam`.
>>>
>>> It may also be a better idea in general to just feed more information
>>> to the scripts directly from the `opam` file (via %{lib}% and friends)
>>> rather than have them assume too much about how they'll be installed.
>>> I didn't want to go down this route because it makes for more drastic
>>> changes, but I'm happy to try it if you're game.
>>
>> Yeah, it is a bit concerning that system packages will override the
>> current opam switch's packages. The problem is that local builds of
>> OCaml programs are typically started by simply running "make", so
>> there's no way to know whether the user wants to use opam or not.
>>
>> I agree that build scripts shouldn't be calling back into opam
>> explicitly (as these do currently).
>>
>> Can we get "opam config env" to set PKG_CONFIG_PATH (as it does for
>> e.g. MANPATH already)? Then we assume that a user who initialises opam
>> in their login script prefers to use opam's build environment (which
>> is already true for other aspects of the environment).
>
> That sounds good to me. And opam's build process would need to set
> this as well, right? i.e. I'm assuming all of these should work:
>
> 1) user is in `opam config env` shell, runs `make` -> only uses opam deps
> 2) user is in system shell, runs `make` -> system deps
> 3) user is in system shell, runs `opam install this-local-package` ->
> only uses opam deps
>
> Hopefully the same opam code path can cover both (1) and (3).

Yes, I think it does this automatically.

>> If you build a
>> Nix package in this environment, would it build that in a clean
>> environment?
>
> Yep, nix builds are in a clean environment, so PKG_CONFIG_PATH during
> a nix build contains only your (transitive) build dependencies.



-- 
Dr Thomas Leonard        http://roscidus.com/blog/
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel