Re: [MirageOS-devel] Compiling C components for mirage-xen

[Thomas Leonard <talex5@xxxxxxxxx>]

On 10 January 2016 at 10:15, Tim Cuthbertson <tim@xxxxxxxxxxx> wrote:
> On Fri, Jan 8, 2016 at 11:14 PM, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>> On 8 January 2016 at 11:19, Tim Cuthbertson <tim@xxxxxxxxxxx> wrote:
>>> I'd like to use `Bcrypt` from the `safepass` opam module in passe[0].
>>> There is sadly no `safepass-xen` package, and trying to just use
>>> `safepass` results in link errors:
>>>
>>>
>>> <...>/lib/safepass/libsafepass_stubs.a(crypt_blowfish.o): In function
>>> `BF_crypt':
>>> (.text+0x1b97): undefined reference to `__errno_location'
>>> <...>/lib/safepass/libsafepass_stubs.a(crypt_blowfish.o): In function
>>> `BF_crypt':
>>> (.text+0x2112): undefined reference to `__errno_location'
>>> <...>/lib/safepass/libsafepass_stubs.a(crypt_blowfish.o): In function
>>> `_crypt_blowfish_rn':
>>> (.text+0x21ea): undefined reference to `__errno_location'
>>> <...>/lib/safepass/libsafepass_stubs.a(crypt_blowfish.o): In function
>>> `_crypt_gensalt_blowfish_rn':
>>> (.text+0x2575): undefined reference to `__errno_location'
>>> <...>/lib/safepass/libsafepass_stubs.a(crypt_blowfish.o): In function
>>> `_crypt_gensalt_blowfish_rn':
>>> (.text+0x25a6): undefined reference to `__errno_location'
>>> <...>/lib/safepass/libsafepass_stubs.a(crypt_blowfish.o):(.text+0x25b7):
>>> more undefined references to `__errno_location' follow
>>>
>>>
>>> I'm assuming I'd need to customise the build process tp produce an
>>> appropriate C library as with `gmp-xen`, `zarith-xen` etc. The trouble
>>> is twofold:
>>>
>>>  - I don't really know what the thing I'm doing is. Is mirage-xen a
>>> different architecture? A different ABI? Do C libraries need to be
>>> cross-compiled, or just compiled with particular flags so that they'll
>>> work on xen-minios instead of gnu/linux?
>>
>> C libraries need to be compiled with different headers and flags (in
>> particular, -mno-red-zone). Using libraries compiled for Linux will
>> often appear to work, but may fail when interrupts occur, so the build
>> makes it hard to use them by accident.
>>
>> However, OCaml binaries can be shared (ocamlopt doesn't use a red zone
>> and the OCaml ABI is the same).
>>
>>>  - The existing scripts seem to be pretty ad-hoc - zarith gets away
>>> mainly with particular CFLAGS
>>> (https://github.com/ocaml/opam-repository/blob/master/packages/zarith-xen/zarith-xen.1.4/files/mirage-install.sh),
>>
>> It's using pkg-config to get the important flags. The others are
>> optimisations (except -fno-builtin, which is either not needed or
>> should be supplied by Mini-OS).
>>
>>> while gmp looks like a full cross-compilation using `--target` (and
>>> then does some other sneaky things).
>>>
>>> safepass uses OASIS for its build system, which doesn't seem to have
>>> any cross-compilation functionality built in as far as I could tell.
>>> Can anyone give me pointers on what I'd need to do to safepass' build
>>> scripts to get this working in mirage-xen, or at the least some
>>> clarification on why it doesn't work as-is and how cross compilation
>>> works for C libraries used in mirage-xen?
>>>
>>> [0]: https://github.com/timbertson/passe
>>
>> The io-page library provides a fairly simple template:
>>
>> https://github.com/mirage/io-page/blob/master/_oasis
>>
>> It builds io_page_unix and io_page_xen libraries. The C flags are
>> added to the main META file with:
>>
>> XMETAExtraLines: xen_linkopts = "-lio_page_xen_stubs"
>>
>> There's a command to get the Xen flags:
>>
>> PostConfCommand: ocaml postconf.ml
>>
>> The Xen source file (stub_alloc_pages_xen.c) is just a symlink to the
>> Unix one (you need a separate file because OASIS can't compile the
>> same source path with different flags).
>
> Thanks, Thomas. - that helps quite a lot
>
> I think I've followed the same pattern now, but still haven't been
> able to get safepass building.
>
> I'm using the following for CCOpt:
>
> `pkg-config --cflags --static mirage-xen-ocaml`
>
> Now the linking phase fails with:
>
> + ocamlfind ocamlmklib -o src/safepass_stubs src/crypt_blowfish.o
> src/bcrypt_stub.o
> ld: errno: TLS definition in /nix/store/<...>-glibc-2.21/lib/libc.so.6
> section .tbss mismatches non-TLS reference in src/crypt_blowfish.o
> /nix/store/<...>-glibc-2.21/lib/libc.so.6: could not read symbols: Bad value
> collect2: error: ld returned 1 exit status
> Command exited with code 2.
> E: Failure("Command ''ocamlbuild' src/libsafepass_stubs.a
> src/dllsafepass_stubs.so src/safepass.cma src/safepass.cmxa
> src/safepass.a src/safepass.cmxs -tag debug' terminated with error
> code 10")

Try running it as "ocamlfind ocamlmklib -verbose" to see what commands
it's actually using. For io-page, I get:

$ ocamlfind ocamlmklib -verbose -o lib/io_page_xen_stubs
lib/stub_alloc_pages_xen.o
Effective set of compiler predicates: autolink,byte,native
+ ocamlmklib -ocamlc ocamlc.opt -ocamlopt ocamlopt.opt -verbose -o
lib/io_page_xen_stubs lib/stub_alloc_pages_xen.o
+ gcc -shared  -o lib/dllio_page_xen_stubs.so lib/stub_alloc_pages_xen.o
+ ar rc lib/libio_page_xen_stubs.a  lib/stub_alloc_pages_xen.o; ranlib
lib/libio_page_xen_stubs.a

So it doesn't seem to be using ld at all, just ar.

> This seems to be something special about errno being thread-local in
> glibc. Googling turns up a lot of advice on fixing this problem, but
> all of it assumes you're just building on linux for linux.
> bcrypt_stubs.c does <errno.h>, which I believe is being provided by
> minios-xen-0.8/include/minios-xen/errno.h.
>
> However, at link time I don't know where errno is supposed to come
> from. It seems to be coming from glibc, but shouldn't it be coming
> from minios instead given that's where the errno header comes from? I
> poked around with `nm` in minios' built files but couldn't find
> anything that actually defined errno.

It's from mirage-xen:

https://github.com/mirage/mirage-platform/blob/c035439de703edbfdd096044afdbab85f4cbf61f/bindings/main.c#L30

>  -- other things I've tried:
>
> 1) I tried using `pkg-config --cflags mirage-xen-ocaml` for CCLib,
> however `ocamlfind ocamlmklib` didn't seem to like that - it didn't
> understand "-m elf_x86_64" nor
> "-T/nix/store/<...>-minios-xen-0.8/lib/minios-xen/libminios.lds"
>
> 2) I notice that minios-xen/errno.h has:
>
> #ifdef HAVE_LIBC
> #include <mini-os/sched.h>
> extern int errno;
> #define ERRNO
> #define errno (get_current()->reent._errno)
> #endif
>
> So just to see what happened, I tried adding -DHAVE_LIBC. But then I
> get errors because sys/reent.h can't be found, so that's probably not
> a useful path to explore.
>
> Any advice where to go from here?
>
> Thanks,
>  - Tim.



-- 
Dr Thomas Leonard        http://roscidus.com/blog/
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel