Re: [Xen-devel] Test Meetup at Developer Summit

[Doug Goldstein <cardoe@xxxxxxxxxx>]

On 9/5/16 6:19 AM, Lars Kurth wrote:
> == Attendees ==
> Lars Kurth
> George Dunlap
> Doug Goldstein
> Andrew Cooper
> Paul Durrant
> 
> There were a few others, which I may have missed

I know Jonathan Creekmore was involved as well.

> 
> I tried to transcribe from a recording we had at lunch, but due to background 
> noise I didn't get everything. Please add/correct, if I got something wrong. 
> There was a bit of discussion, but I tried to capture the key points only 
> (e.g. I omitted clarifying questions)

Thanks Lars! Sorry for the delay in responses I went immediately from
the Summit to vacation for the past week.

> 
> == Discussion: Travis Improvements ==
> 
> Doug, wanted to discuss changes to the project's Travis set-up
> * Reconfiguring Travis set-up such that it launches docker containers to do 
> very basic smoke testing of individual commits on a number of different build 
> configurations
> * Right now this would require running a script running inside a script
> * The key question is what matrix to support: e.g. which compiler versions, 
> whether it should cover deb builds, ...
> * It is basically just a build test, with a very basic run-test within an 
> emulated qemu environment to check whether we boot. Aka booting a small image 
> that is cross-compiled against various environments that produces "Hello 
> World" on the first available serial-port
> * If "Hello World" was produced, then Travis would decide the build was 
> successful
> 
> George: over OSSTEST what value would this add? 
> Doug: feedback about 7-8 minutes after a push with some confidence that a 
> build was not broken, faster than OSSTEST does now. Obviously this would not 
> replace OSSTEST, but should reduce the number of build-related issues in 
> OSSTEST
> Andrew: This should help quite a lot with avoiding spotting build related 
> issues earlier
> Doug: Would help avoid unnecessary manual build testing
> George: Should help committers also 
> Lars: Are there any extra requirements for people to use?
> Andrew: Needs an account and set-up on GitHub

If we decide to stick with GitHub and Travis then I plan on documenting
this so its easier to people to do this on their own.

> 
> == Discussion: GitHub vs. GitLab for Travis ==
> The second set of topics was to move from GitHub to GitLab for Build-testing 
> due to the code availability issues with GitHub. Doug uses GitLab internally 
> and is also more functional that GitHub. 

The concern when I initiated the GitHub + Travis bits were that GitHub
and Travis were not free as in speech but only free as in beer. GitLab
is free as in speech [1] and if you use their hosted service [2] its
free as in beer as well.

They've made outstanding strides this year with their integrated CI
service called GitLab CI. At my company we now use this for all of our
projects since it allows you to define multiple steps and dependencies
of those steps. Steps can be flagged to have to run within specific
Docker containers which allows us to have a matrix of build environments
to build test with. It also allows you to capture artifacts from the
build which is what we do to actually run real tests against a built
object (ala OSSTest). Travis + GitHub at this point allow you to specify
that you need sudo and then from there you can script up Docker commands
by hand which we can certainly do but there is no parallelism and its a
lot more we need to maintain in our scripts.

> 
> GitLab can be used hosted (for free projects) or host it locally on a VM
> 
> Doug: would probably require taking the github yaml and rewrite as gitlab 
> yam. There are a number of advantages on OS support and docker integration, 
> which makes set-up a lot easier compared to github.
> 
> Andy: if we were going to do this, the project should probably get a support 
> license
> 
> Summary:
> - Gather community input on GitHub vs. GitLab
> - GitLab code is public, and thus should address 

I think this was suppose to end with "address concerns about software
licensing of the software we use"

> 
> There was also a bit of discussion about the review work-flow functionality 
> in GitLab: just to clarify, we were not discussing any changes in this area. 
> Merely to use GitLab as framework for Travis / Build testing and addressing 
> some of the past issues that were raised around GitHub code availability.

I'm not proposing using any of the merge request or issue tracker bits.
Just the CI portion for doing builds and allowing people to push a
branch and have it built on a variety of compilers to lessen the chance
that "it worked on my machine" situations strike.


[1] https://gitlab.com/gitlab-org/gitlab-ce
[2] https://about.gitlab.com/gitlab-com/

-- 
Doug Goldstein

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel