Re: [MirageOS-devel] Building mirage-www fails

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

On 22 Oct 2014, at 18:13, Nik Sultana <ns441@xxxxxxxxx> wrote:

> Thanks Anil, I could build the Ubuntu 14.04 (Trusty) container following your 
> instructions, with one modification: referencing "avsm/docker-opam" instead 
> of "avsm/docker".

Sorry, my typo.

> 
> I haven't tested the other containers since I don't have ready access to a 
> CentOS box. This patch will work if the other distributions use a version of 
> sh that's compatible to that used in Trusty.
> 

To be clear: the point of the Docker container is to give you a CentOS 
environment.  You can just use the CentOS tag, even from an Ubuntu host.

> I tested the problem with the semantics of && under -e by replacing the line
>  make configure && make depend && make build
> with
>  false && make configure && make depend && make build
> and verified that I still got "MIRAGE WORKS" printed at the end after running
>  ./build.sh ubuntu-trusty-4.01.0 mirage-skeleton
> 
> Then I unfolded
>  false && make configure && make depend && make build
> into
>  false
>  make configure
>  make depend
>  make build
> and reran, and verified that I did NOT get "MIRAGE WORKS" printed at the end.
> 
> Finally I removed "false", and reran.
> I did get "MIRAGE WORKS" printed at the end.
> 
> At the risk of asking an over-asked question: what's the procedure for the 
> PR, do I need to quote an issue number etc, or can I just go ahead and create 
> a PR from a fork?

Just go ahead and send a pull request to the mirage/is-mirage-broken repo from 
your fork.  Getting the code fixed always takes priority over the form of the 
patch :-)

thanks!
Anil


> 
> Nik
> 
> On 2014-10-22 14:03, Anil Madhavapeddy wrote:
>> CCing the devel list so everyone knows how this works.
>> If you have a Linux box with Docker installed, you can clone the
>> precise environment by:
>> $ sudo docker pull avsm/docker
>> This gives you access to a number of 'tags' (which are predefined
>> builds of OPAM/OCaml on various filesystems). Full list of tags are:
>> https://registry.hub.docker.com/u/avsm/docker-opam/tags/manage/
>> You can use a tag directly via:
>> $ sudo docker build -t avsm/docker:ubuntu-trusty-4.01.0 <cmd>
>> The scripts referenced below already do this, and are all in:
>> https://github.com/mirage/is-mirage-broken
>> The cron.sh is the one that's run regularly and sync with the
>> Docker registry and builds every combination.
>> To recreate one of them, just run build.sh directly; e.g.
>> ./build.sh ubuntu-trusty-4.01.0 mirage-skeleton
>> If you already have a suitable environment with OPAM and OCaml
>> installed system-wide, you can just run the shell scripts in
>> is-mirage-broken/scripts/ directly to fix them.
>> -anil
>> On 22 Oct 2014, at 13:55, Nik Sultana <ns441@xxxxxxxxx> wrote:
>>> Hi Anil, is this something I can try locally to see if the modified script 
>>> works, or do I need to run the script on a build server somewhere? Feel 
>>> free to point me to any "BUILD" notes i could use for this, in case I can 
>>> test it and make a PR, if it's still an open issue.
>>> Best,
>>> Nik
>>> On 2014-10-21 16:18, Richard Mortier wrote:
>>>> if you read the docs i quoted, that's exactly as documented:
>>>>>>>>>       • The -e setting shall be ignored when executing the compound 
>>>>>>>>> list
>>>>>>>>> following the while, until, if, or elif reserved word, a pipeline
>>>>>>>>> beginning with the ! reserved word, or any command of an AND-OR list
>>>>>>>>> other than the last.
>>>> it might be rather counter-intuitive and of dubious merit, but it's
>>>> the documented behaviour...
>>>> On 21 Oct 2014, at 16:01, Nik Sultana <ns441@xxxxxxxxx> wrote:
>>>>> I think Richard might be right. The semantics of && seems to be a bit
>>>>> counterintuitive in this case. Consider this script:
>>>>> #!/bin/sh
>>>>> set -e #line1
>>>>> false && echo hello #line2
>>>>> echo hello2
>>>>> false #line4
>>>>> echo world
>>>>> It gives the output:
>>>>> hello2
>>>>> which is odd, because we expect the script to be terminated at line2 --
>>>>> the line "echo hello2" should never be reached. Instead, the script
>>>>> continues on and is only terminated at line4.
>>>>> Just to test the effect of "set -e", commenting line1 and running the
>>>>> script gives the output:
>>>>> hello2
>>>>> world
>>>>> On 2014-10-21 15:49, Anil Madhavapeddy wrote:
>>>>>> The && should catch that failure.  Consider:
>>>>>> #!/bin/sh -ex
>>>>>> exit 1 && echo hello
>>>>>> echo world
>>>>>> Executing that:
>>>>>> $ ./test.sh
>>>>>> + exit 1
>>>>>> If the make depend fails, I'd expect the && to fail, and the resulting
>>>>>> non-zero exit code to cause the whole shell to exit.
>>>>>> -anil
>>>>>> On 21 Oct 2014, at 15:37, Richard Mortier
>>>>>> <Richard.Mortier@xxxxxxxxxxxxxxxx> wrote:
>>>>>>> but isn't it the "make depend" step that's failing as far as the
>>>>>>> is-mirage-broken/scripts/mirage-www script is concerned -- hence it
>>>>>>> won't exit according to point 2. below (-e is ignored if it's any
>>>>>>> command of an AND-OR list other than the last).
>>>>>>> with -e on, seems expected practice is to issue the commands
>>>>>>> individually i.e.
>>>>>>> make depend
>>>>>>> make configure
>>>>>>> make build
>>>>>>> ...and let the -e fail them if required. (assuming you don't want to
>>>>>>> accumulate info about failures of depend/configure on other platforms
>>>>>>> before bailing out.)
>>>>>>> On 21 Oct 2014, at 15:18, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>>>>>>>> It's here:
>>>>>>>> https://github.com/mirage/is-mirage-broken/blob/master/scripts/mirage-www
>>>>>>>> And the `make` invocations do seem to exit with a non-zero exit code.
>>>>>>>> -ani
>>>>>>>> On 21 Oct 2014, at 15:02, Richard Mortier
>>>>>>>> <Richard.Mortier@xxxxxxxxxxxxxxxx> wrote:
>>>>>>>>> not sure precisely where the script being executed actually is but
>>>>>>>>> could it be because it's a command in a multi-command pipeline?
>>>>>>>>> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
>>>>>>>>> -e
>>>>>>>>> When this option is on, when any command fails (for any of the
>>>>>>>>> reasons listed in Consequences of Shell Errors or by returning an
>>>>>>>>> exit status greater than zero), the shell immediately shall exit
>>>>>>>>> with the following exceptions:
>>>>>>>>>       • The failure of any individual command in a multi-command 
>>>>>>>>> pipeline
>>>>>>>>> shall not cause the shell to exit. Only the failure of the pipeline
>>>>>>>>> itself shall be considered.
>>>>>>>>>       • The -e setting shall be ignored when executing the compound 
>>>>>>>>> list
>>>>>>>>> following the while, until, if, or elif reserved word, a pipeline
>>>>>>>>> beginning with the ! reserved word, or any command of an AND-OR list
>>>>>>>>> other than the last.
>>>>>>>>>       • If the exit status of a compound command other than a subshell
>>>>>>>>> command was the result of a failure while -e was being ignored, then
>>>>>>>>> -e shall not apply to this command.
>>>>>>>>> This requirement applies to the shell environment and each subshell
>>>>>>>>> environment separately. For example, in:
>>>>>>>>> set -e; (false; echo one) | cat; echo two
>>>>>>>>> the false command causes the subshell to exit without executing echo
>>>>>>>>> one; however, echo two is executed because the exit status of the
>>>>>>>>> pipeline (false; echo one) | cat is zero.
>>>>>>>>> On 21 Oct 2014, at 14:31, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>>>>>>>>>> Great!  For the rest of the list, it looks like mirage-dev is
>>>>>>>>>> working with the external OPAM solver (aspcud, on Ubuntu-trusty),
>>>>>>>>>> but not with the internal one (CentOS builds):
>>>>>>>>>> https://github.com/mirage/is-mirage-broken/tree/master/logs
>>>>>>>>>> This is probably due to our excessive use of depopts; I will
>>>>>>>>>> replace those with explicit virtual packages (e.g. dns-mirage) that
>>>>>>>>>> encode all the dependencies they need.
>>>>>>>>>> More mysteriously, the `set -e` in the shell script isn't causing
>>>>>>>>>> it to terminate early, so the `MIRAGE WORKS` string is always
>>>>>>>>>> printed.  We need a is-mirage-working-working script to fix our
>>>>>>>>>> is-mirage-working script :-)
>>>>>>>>>> -anil
>>>>>>>>>> On 20 Oct 2014, at 15:09, Luke Dunstan <lukedunstan81@xxxxxxxxx>
>>>>>>>>>> wrote:
>>>>>>>>>>> Thanks, I was able to compile and run mirage-www on a Docker image
>>>>>>>>>>> with 4.01.
>>>>>>>>>>> Luke
>>>>>>>>>>> On 20 October 2014 09:28, Anil Madhavapeddy <anil@xxxxxxxxxx>
>>>>>>>>>>> wrote:
>>>>>>>>>>> On 19 Oct 2014, at 03:17, Luke Dunstan <lukedunstan81@xxxxxxxxx>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> I tried to build mirage-www as per
>>>>>>>>>>>> http://openmirage.org/wiki/mirage-www
>>>>>>>>>>>> $ env NET=socket FS=crunch mirage configure --unix
>>>>>>>>>>>> (success)
>>>>>>>>>>>> $ make depend
>>>>>>>>>>>> (success*)
>>>>>>>>>>>> $ make
>>>>>>>>>>>> camlbuild -classic-display -use-ocamlfind -pkgs
>>>>>>>>>>>> lwt.syntax,conduit.mirage,cow.syntax,cowabloga,cstruct,io-page,io-page.unix,lwt,mirage-console.unix,mirage-fs-unix,mirage-http,mirage-types,mirage-types.lwt,tcpip.stack-socket
>>>>>>>>>>>> -tags "syntax(camlp4o),annot,bin_annot,strict_sequence,principal"
>>>>>>>>>>>> -cflag -g -lflags -g,-linkpkg main.native
>>>>>>>>>>>> ocamlfind ocamlc -c -g -annot -bin-annot -principal
>>>>>>>>>>>> -strict-sequence -package tcpip.stack-socket -package
>>>>>>>>>>>> mirage-types.lwt -package mirage-types -package mirage-http
>>>>>>>>>>>> -package mirage-fs-unix -package mirage-console.unix -package lwt
>>>>>>>>>>>> -package io-page.unix -package io-page -package cstruct -package
>>>>>>>>>>>> cowabloga -package cow.syntax -package conduit.mirage -package
>>>>>>>>>>>> lwt.syntax -syntax camlp4o -o main.cmo main.ml
>>>>>>>>>>>> + ocamlfind ocamlc -c -g -annot -bin-annot -principal
>>>>>>>>>>>> -strict-sequence -package tcpip.stack-socket -package
>>>>>>>>>>>> mirage-types.lwt -package mirage-types -package mirage-http
>>>>>>>>>>>> -package mirage-fs-unix -package mirage-console.unix -package lwt
>>>>>>>>>>>> -package io-page.unix -package io-page -package cstruct -package
>>>>>>>>>>>> cowabloga -package cow.syntax -package conduit.mirage -package
>>>>>>>>>>>> lwt.syntax -syntax camlp4o -o main.cmo main.ml
>>>>>>>>>>>> File "main.ml", line 98, characters 2-13:
>>>>>>>>>>>> Error: Unbound module OS
>>>>>>>>>>>> Command exited with code 2.
>>>>>>>>>>>> make: *** [main.native] Error 10
>>>>>>>>>>>> To try to ensure that this was reproducible I also tried starting
>>>>>>>>>>>> from the "avsm/docker-opam:ubuntu-trusty-4.01.0" Docker image and
>>>>>>>>>>>> got the same error. However, I had to change the ounit package
>>>>>>>>>>>> URL because forge.ocamlcore.org is apparently down.
>>>>>>>>>>>> Also, I had to "apt-get install libssl-dev" to get past the
>>>>>>>>>>>> "configure" / "make depend" step because it was apparently
>>>>>>>>>>>> required to compile ssl-0.4.7, so maybe that is related to the
>>>>>>>>>>>> problem?
>>>>>>>>>>> Hi Luke,
>>>>>>>>>>> Thanks for the bug report, and bonus points for trying the Docker
>>>>>>>>>>> repository out.  I've fixed the underlying issue that was causing
>>>>>>>>>>> the unbound OS module to show up -- it's because of some recent
>>>>>>>>>>> shuffling of dependencies in console handling, and this resulted
>>>>>>>>>>> in mirage-unix no longer being implicitly depended on.  I've
>>>>>>>>>>> modified the mirage tool to explicitly add the dependency in.
>>>>>>>>>>> In order to help keep things working across the increasing number
>>>>>>>>>>> of distributions, I've also created a couple of cron jobs that
>>>>>>>>>>> automatically build things and publish the logs
>>>>>>>>>>> - The Docker registry now has OPAM tags for Ubuntu/CentOS on OCaml
>>>>>>>>>>> 4.01 and 4.02:
>>>>>>>>>>> https://registry.hub.docker.com/u/avsm/docker-opam/builds_history/15669/
>>>>>>>>>>> https://github.com/avsm/docker-opam has the sources
>>>>>>>>>>> - https://github.com/mirage/is-mirage-broken runs as a cronjob
>>>>>>>>>>> (cron.sh) on blobs.openmirage.org daily and pushes the logs to the
>>>>>>>>>>> logs/ directory in this repo.  When that's completed running (and
>>>>>>>>>>> we've fixed the build breakages!), it should be easy to get it on
>>>>>>>>>>> the website as a badge so we can see the status of repos at a
>>>>>>>>>>> glance.
>>>>>>>>>>> About to hop on a flight now, so I've left the cron job running
>>>>>>>>>>> and will check it out tomorrow.  If anyone else can test out the
>>>>>>>>>>> Docker images in the meanwhile, feel free...
>>>>>>>>>>> -anil
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> MirageOS-devel mailing list
>>>>>>>>>>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>>>>>>>>>>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>>>>>>>>>> _______________________________________________
>>>>>>>>>> MirageOS-devel mailing list
>>>>>>>>>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>>>>>>>>>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>>>>>>>>> --
>>>>>>>>> Cheers,
>>>>>>>>> R.
>>>>>>>>> _______________________________________________
>>>>>>>>> MirageOS-devel mailing list
>>>>>>>>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>>>>>>>>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>>>>>>> --
>>>>>>> Cheers,
>>>>>>> R.
>>>>>>> _______________________________________________
>>>>>>> MirageOS-devel mailing list
>>>>>>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>>>>>>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>>>>>> _______________________________________________
>>>>>> MirageOS-devel mailing list
>>>>>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>>>>>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
> 


_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel