Re: [Xen-devel] [PATCH OSSTEST v3 00/19] add distro domU testing flight

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Mon, 2014-12-01 at 12:56 +0000, Ian Campbell wrote:
> The exception to that good news is that sg-report-flight
> --that-flight=NNN MMM doesn't seem to be considering failures in MMM
> regressions vs NNN. e.g. test-amd64-i386-i386-daily-netboot-pvgrub
> should appear as a regression from 31961 to 31964 but isn't. I'm going
> to have a poke around that.

This turned out to be as easy as passing --blessings=play to
sg-report-flight (the default is ["real", "real-bisect"]), which
produced the output below (previously everything was "Tests which did
not succeed, but are not blocking")

I have a feeling this is deliberate and that only real+bisects are
supposed to be considered here on purpose.

If not then I think the right answer is just to add this to sgr_args in
the relevant case:
        --blessings=${DAILY_BRANCH_TESTED_BLESSING:-$OSSTEST_BLESSING}

Ian.

------

31964: regressions - FAIL

flight 31964 distros play [play]
http://www.chiark.greenend.org.uk/~xensrcts/logs/31964/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-i386-current-netinst-pygrub  5 xen-boot   fail REGR. vs. 31961
 test-amd64-i386-i386-daily-netboot-pvgrub 7 debian-di-install fail REGR. vs. 
31961
 test-amd64-i386-i386-jessie-netboot-pvgrub  5 xen-boot    fail REGR. vs. 31961

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-i386-weekly-netinst-pygrub 7 debian-di-install fail like 31961

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-i386-sid-netboot-pvgrub  7 debian-di-install  fail never pass
 test-amd64-amd64-amd64-jessie-netboot-pvgrub 7 debian-di-install fail never 
pass
 test-armhf-armhf-armhf-sid-netboot-pygrub  7 debian-di-install fail never pass
 test-amd64-i386-i386-sid-netboot-pygrub  7 debian-di-install   fail never pass
 test-armhf-armhf-armhf-jessie-netboot-pygrub 7 debian-di-install fail never 
pass
 test-armhf-armhf-armhf-daily-netboot-pygrub 7 debian-di-install fail never pass
 test-amd64-amd64-i386-jessie-netboot-pygrub 7 debian-di-install fail never pass
 test-amd64-amd64-amd64-sid-netboot-pygrub  7 debian-di-install fail never pass
 test-amd64-i386-amd64-jessie-netboot-pygrub 7 debian-di-install fail never pass
 test-amd64-i386-amd64-sid-netboot-pygrub  7 debian-di-install  fail never pass

baseline version:
 flight               31961

jobs:
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-amd64-daily-netboot-pvgrub                  pass    
 test-amd64-i386-i386-daily-netboot-pvgrub                    fail    
 test-amd64-amd64-amd64-jessie-netboot-pvgrub                 fail    
 test-amd64-i386-i386-jessie-netboot-pvgrub                   fail    
 test-amd64-amd64-i386-sid-netboot-pvgrub                     fail    
 test-amd64-amd64-amd64-wheezy-netboot-pvgrub                 pass    
 test-amd64-i386-i386-wheezy-netboot-pvgrub                   pass    
 test-amd64-i386-amd64-daily-netboot-pygrub                   pass    
 test-armhf-armhf-armhf-daily-netboot-pygrub                  fail    
 test-amd64-amd64-i386-daily-netboot-pygrub                   pass    
 test-amd64-i386-amd64-jessie-netboot-pygrub                  fail    
 test-armhf-armhf-armhf-jessie-netboot-pygrub                 fail    
 test-amd64-amd64-i386-jessie-netboot-pygrub                  fail    
 test-amd64-amd64-amd64-sid-netboot-pygrub                    fail    
 test-amd64-i386-amd64-sid-netboot-pygrub                     fail    
 test-armhf-armhf-armhf-sid-netboot-pygrub                    fail    
 test-amd64-i386-i386-sid-netboot-pygrub                      fail    
 test-amd64-amd64-amd64-squeeze-netboot-pygrub                pass    
 test-amd64-i386-amd64-squeeze-netboot-pygrub                 pass    
 test-amd64-amd64-i386-squeeze-netboot-pygrub                 pass    
 test-amd64-i386-i386-squeeze-netboot-pygrub                  pass    
 test-amd64-i386-amd64-wheezy-netboot-pygrub                  pass    
 test-amd64-amd64-i386-wheezy-netboot-pygrub                  pass    
 test-amd64-amd64-amd64-current-netinst-pygrub                pass    
 test-amd64-i386-amd64-current-netinst-pygrub                 pass    
 test-amd64-amd64-i386-current-netinst-pygrub                 pass    
 test-amd64-i386-i386-current-netinst-pygrub                  fail    
 test-amd64-amd64-amd64-weekly-netinst-pygrub                 pass    
 test-amd64-i386-amd64-weekly-netinst-pygrub                  pass    
 test-amd64-amd64-i386-weekly-netinst-pygrub                  pass    
 test-amd64-i386-i386-weekly-netinst-pygrub                   fail    


------------------------------------------------------------
sg-report-flight on kazak.uk.xensource.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images

Logs, config files, etc. are available at
    http://www.chiark.greenend.org.uk/~xensrcts/logs

Test harness code can be found at
    http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary





_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel