Re: [Xen-devel] xenbits and https redirect (was Re: [xen-unstable-smoke test] 99610: regressions - FAIL)

[George Dunlap <george.dunlap@xxxxxxxxxx>]

On Mon, Jul 25, 2016 at 4:24 PM, Ian Jackson <ian.jackson@xxxxxxxxxxxxx> wrote:
> osstest service owner writes ("[xen-unstable-smoke test] 99610: regressions - 
> FAIL"):
>> flight 99610 xen-unstable-smoke real [real]
>> http://logs.test-lab.xenproject.org/osstest/logs/99610/
>>
>> Regressions :-(
>>
>> Tests which did not succeed and are blocking,
>> including tests which could not be run:
>>  build-amd64                   5 xen-build    fail REGR. vs. 97725
>
> fatal: unable to access
> 'https://xenbits.xen.org/git-http/mini-os.git/': Failed to connect to
> xenbits.xen.org port 443: Connection refused
>
> This is because:
>
> 1. xen.git specifies the url http://xenbits.xen.org/git-http/mini-os.git/
>
> 2. xenbits has/had a redirect to https, as part of our transition to
>    using https everywhere.
>
> 3. So the xen.git build tries to access
>    https://xenbits.xen.org/git-http/mini-os.git/
>
> 4. The http_proxy variable set by osstest to try to make http requests
>    by build systems go via the cacheing proxy is not effective for
>    https
>
> 5. The colo firewall intentionally prevents general accesses to the
>    global internet by builds (or indeed tests) run by osstest.
>    One purpose is to prevent uncached accesses, which are a
>    performance and reliability problem.
>
> This is hard to fix because:
>
> 6. Using the https_proxy variable would not help.  It would cause
>    libcurl (which is what git is using) to use CONNECT.  That is
>    not what we need: we need the client to trust the proxy, so
>    that we can get cacheing.
>
> 7. squid would support proxying GET https:// requests but there is
>    AFAICT no way for curl to do this (even in fairly recent curl).
>    I found this thread on the subject:
>      https://curl.haxx.se/mail/archive-2015-12/0009.html
>
> I think in the medium term the right answer is for osstest to use its
> git cacheing proxy, rather than trying to cache the http[s] protocol,
> by specifying a .gitconfig involving insteadOf.
>
> For now I have asked Credativ to disable the https redirect on
> xenbits.
>
> I would be interested to know whether this redirect caused other
> problems.  Should we reenable it when osstest is improved ?
>
> When should we change the http:// references in xen.git to https:// ?

Once we're confident that the https urls work properly, is there any
reason to wait?

Or to put it differently, why not do it right now?

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel