Re: [MirageOS-devel] Compiling gmp-xen with nixpkgs

[Tim Cuthbertson <tim@xxxxxxxxxxx>]

Yeah true, there are quite a lot of mirage packages. There are already
npm2nix, pypi2nix and cabal2nix tools which spit out nix packages from
upstream definitions, so I'm confident that an opam2nix would be
possible too. I believe all you'd really need to do is interpret the
opam repo data to extract dependencies & build commands, wrapping the
opam tool itself shouldn't be necessary.

I'm actually not sure how much of `nixpkgs` is derived automatically
using tools like this, and how many are updated manually (or via
ad-hoc scripts). For now I'll plod along manually because some of the
packages need special care (i.e. hacks), but it's definitely something
I'll look into once I have things working, or when I tire of all this
manual packaging ;).

Cheers,
 - Tim.

On Tue, Jun 16, 2015 at 6:44 PM, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
> Phew, I'm glad there's no real error beyond the test cases.
>
> Any packaging are always appreciated, especially to Nix.  However, it
> may be worth just wrapping OPAM somehow to keep them in sync.  Since
> the package database moves quite fast, we've had problems in the past
> with users grabbing older versions and running into issues.
>
> Edwin TÃrÃk has been putting together an outline for a strategy to
> convert OPAM files to Deb/RPM:
> https://github.com/ocaml/opam/wiki/opam2%7Brpm,deb%7D
>
> ...but I'm wondering what a first-order approximation in Nix might look
> like.  Could it wrap OPAM and set appropriate bin/lib dirs for a Mirage
> installation using the latest OPAM repository?
>
> -anil
>
>> On 16 Jun 2015, at 01:53, Tim Cuthbertson <tim@xxxxxxxxxxx> wrote:
>>
>> Thanks, guys.
>>
>> I tracked down where `opam` stored its build logs (which doesn't seem
>> to be documented anywhere, so I resorted to `lsof`), and did a diff of
>> the outputs. Turns out I was doing exactly what `opam` does, but I
>> forgot that `nix` by default runs `make test` after building. So
>> actually it had built fine, but the tests (as Thomas suspected) simply
>> don't compile in this configuration. So that built just fine once I
>> disabled the tests - I can't tell if it all works yet as I've got some
>> more missing dependencies to deal with, but it's looking good.
>>
>> When I'm done, how do you folks feel about me submitting these nix
>> packages to the official nixpkgs repo? I assume you only want to
>> support `opam` officially, but nix users are used to being on an
>> unsupported environment ;). I'm hoping the churn of new packages
>> shouldn't be too hard for me to keep up with at this point? Making the
>> 20+ new packages I needed for this took me less than a day, and
>> updates to existing packages should be pretty simple.
>>
>> Cheers,
>> - Tim.
>>
>> On Tue, Jun 16, 2015 at 8:17 AM, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>>> On 15 Jun 2015, at 17:48, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>>>>
>>>> On 15 June 2015 at 13:58, Tim Cuthbertson <tim@xxxxxxxxxxx> wrote:
>>>>> Hi all,
>>>>>
>>>>> After getting acquainted with some of the sample applications, I've
>>>>> been trying to get the server component of a relatively small project
>>>>> of mine[0] running under mirage.
>>>>>
>>>>> After refactoring the server code to use the mirage interfaces rather
>>>>> than Unix & friends directly, I've got it compiling to the xen
>>>>> backend.
>>>>>
>>>>> I've been using nix[1] to build my project, so while getting stuff
>>>>> working I have been using opam instead to maximise my chance of it
>>>>> working. But now that it does, I'd like to get it working under `nix`
>>>>> as well. Most of it is just grunt work, as the mirage libraries &
>>>>> dependencies seem to build without issue (yay!), but when it comes to
>>>>> linking stub libraries I'm having some trouble.
>>>>>
>>>>> As I understand it, some (most?) c stubs will need cross-compilation
>>>>> for a xen (minios?) target before they can be linked into a xen
>>>>> unikernel. You've made *-xen opam package variants for the libraries
>>>>> needed for nocrypto - zarith, gmp, etc.
>>>>>
>>>>> In trying to recreate these I feel like I'm doing just the same as the
>>>>> opam packages - based on the opam repo I'm applying the exact same
>>>>> patches, running the same build script (modulo $PKG_CONFIG_PATH), etc.
>>>>> But I still can't get gmp to build. It gets to the linking stage, and
>>>>> I get a screenful of undefined symbols, primarily "_ctype" and
>>>>> "printk". I've uploaded the full log[2]. if that helps.
>>>>>
>>>>> I can't really tell which side of the problem this is - should gmp not
>>>>> be referencing these symbols (maybe I'm missing some compile flags)?
>>>>> or should the symbols be found somewhere, and I'm failing to link in
>>>>> some important library?
>>>>>
>>>>> In either case it's hard to tell where to go from here, since I can't
>>>>> see anything too suspicious in the logs, and I'm trying my best to do
>>>>> the same thing as `opam` would. Any pointers?
>>>>
>>>> Hi Tim,
>>>>
>>>> I don't know much about the gmp build process, but if you have a good
>>>> (opam) build and a bad (nix) one, I guess you can just compare the two
>>>> environments. e.g. are the missing symbols present in the good build?
>>>> Does the nix one build if you point PKG_CONFIG_PATH at the opam
>>>> installation, etc?
>>>>
>>>> Many of the errors are coming from libtests.a - should it even be
>>>> building the tests for a cross-compile (I don't know)?
>>>>
>>>
>>> One common reason this breaks under Nix (or similar non-traditional
>>> paths) is that there is an rpath which is used by the linker to find
>>> the library in a standard environment, but not when Nix is explicitly
>>> managing paths.
>>>
>>> It's easiest to compare the raw build commands and bisect them for
>>> differences to track this down, as Thomas notes.
>>>
>>> -anil
>>
>

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel