Re: [MirageOS-devel] [opam-devel] experimental opam bulk build logs on GitHub

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

I was discussing this very problem with Thomas in fact!  Right now we deliberately do a separate opam install for each package, and do not cover depopts at all.

The plan is to integrate the scripts into using Irmin to record the log outputs, since git is hitting congestion collapse with >20 workers (GitHub rejects the push due to another process having the lock).  With Irmin, we can record all the logs into a memory instance, and have one pushing process to GitHub.

When we do have Irmin support, we can calculate a static schedule of all the packages that should be built, and just have spare CPUs pick up some work.  It would be great to build the risky packages (i.e. early up the dependency chain) first, and then move to the leaf packages.

I'm not sure if re-using packages and snapshotting is a good idea in general, but it's very useful for common ones like ocamlfind, Lwt or Core.  Depopts are a big problem here though, particularly for Lwt...

-anil

On 23 May 2014, at 09:50, Louis Gesbert <louis.gesbert@xxxxxxxxxxxx> wrote:

Very nice !

 

I am wondering, do you 'opam install' every package individually ? We may play some clever tricks with the solver here, like `opam upgrade --criteria=-notuptodate,+new` which will attempt to install as many packages as possible, after which we could iterate on a universe reduced to the not-yet-installed packages. Depopts would be less predictable, but we can't realistically test all possible combinations of those anyway.

 

Would be nice to find a common place to aggregate such results.

 

Best,

Louis

 

 

Le mardi 20 mai 2014, 17:40:04 Anil Madhavapeddy a écrit :

> Hi,

>

> I've written some scripts that use Docker and OPAM to run bulk builds across the entire package repository, and the results are logged and pushed to GitHub daily.

>

> If anyone wants to improve the health of the repository, please look at:

>

> https://github.com/ocaml/opam-bulk-logs

>

> in the err/ directories, and submit pull requests fix the failures.

>

> Source scripts are still in my personal GitHub; I'll move these to the ocaml/ repo.

>

> cheers,

> Anil

> _______________________________________________

> opam-devel mailing list

> opam-devel@xxxxxxxxxxxxxxx

> http://lists.ocaml.org/listinfo/opam-devel

 

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel