[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] [PATCH OSSTEST v4] Arrange to test migration from the previous Xen version



There are several steps to this:

- Identify $prevxenbranch, that is the branch which precedes
  $xenbranch.
- Create appropriate build jobs.
- Add support in ts-xen-install for overriding {xen,}buildjob on a
  per-ident basis
- Add a new receipt test-pair-oneway which only migrates from
  src_host to dst_host and not the reverse
- Create appropriate test jobs, overridding the default builds for
  src_host.

Currently we only do this for xen* branches and using xl, but in the
future we may wish to add to the libvirt branch too.

In make-flight if REVISION_PREVXEN is not supplied (e.g. called from
standalone-reset or by hand etc) then we create the build-$arch-prev jobs
with no revision_xen, same as build-$arch

It would be nice to try and reuse the builds from the last flight
which tested the $prevxenbranch baseline. I've not dont that here.

+build-amd64-prev                                      arch                     
   amd64
+build-amd64-prev                                      build_lvextend_max       
   50
+build-amd64-prev                                      enable_ovmf              
   true
+build-amd64-prev                                      enable_xend              
   false
+build-amd64-prev                                      enable_xsm               
   false
+build-amd64-prev                                      host_hostflags           
   share-build-wheezy-amd64,arch-amd64,suite-wheezy,purpose-build
+build-amd64-prev                                      revision_ovmf
+build-amd64-prev                                      revision_qemu
+build-amd64-prev                                      revision_qemuu           
   c4a962ec0c61aa9b860a3635c8424472e6c2cc2c
+build-amd64-prev                                      revision_seabios
+build-amd64-prev                                      revision_xen             
   666b80f239c566283cb1b3435180d99a329d0156
+build-amd64-prev                                      tree_ovmf
+build-amd64-prev                                      tree_qemu                
   git://xenbits.xen.org/staging/qemu-xen-unstable.git
+build-amd64-prev                                      tree_qemuu               
   git://xenbits.xen.org/staging/qemu-upstream-unstable.git
+build-amd64-prev                                      tree_seabios
+build-amd64-prev                                      tree_xen                 
   git://xenbits.xen.org/xen.git

+build-i386-prev                                       arch                     
   i386
+build-i386-prev                                       build_lvextend_max       
   50
+build-i386-prev                                       enable_ovmf              
   true
+build-i386-prev                                       enable_xend              
   false
+build-i386-prev                                       enable_xsm               
   false
+build-i386-prev                                       host_hostflags           
   share-build-wheezy-i386,arch-i386,suite-wheezy,purpose-build
+build-i386-prev                                       revision_ovmf
+build-i386-prev                                       revision_qemu
+build-i386-prev                                       revision_qemuu           
   c4a962ec0c61aa9b860a3635c8424472e6c2cc2c
+build-i386-prev                                       revision_seabios
+build-i386-prev                                       revision_xen             
   666b80f239c566283cb1b3435180d99a329d0156
+build-i386-prev                                       tree_ovmf
+build-i386-prev                                       tree_qemu                
   git://xenbits.xen.org/staging/qemu-xen-unstable.git
+build-i386-prev                                       tree_qemuu               
   git://xenbits.xen.org/staging/qemu-upstream-unstable.git
+build-i386-prev                                       tree_seabios
+build-i386-prev                                       tree_xen                 
   git://xenbits.xen.org/xen.git

+test-amd64-amd64-upgrade                              all_hostflags            
   arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test,equiv-1
+test-amd64-amd64-upgrade                              arch                     
   amd64
+test-amd64-amd64-upgrade                              buildjob                 
   build-amd64
+test-amd64-amd64-upgrade                              debian_arch              
   amd64
+test-amd64-amd64-upgrade                              debian_kernkind          
   pvops
+test-amd64-amd64-upgrade                              kernbuildjob             
   build-amd64-pvops
+test-amd64-amd64-upgrade                              kernkind                 
   pvops
+test-amd64-amd64-upgrade                              src_host_buildjob        
   build-amd64-prev
+test-amd64-amd64-upgrade                              src_host_xenbuildjob     
   build-amd64-prev
+test-amd64-amd64-upgrade                              toolstack                
   xl
+test-amd64-amd64-upgrade                              xenbuildjob              
   build-amd64

+test-amd64-i386-upgrade                               all_hostflags            
   arch-i386,arch-xen-amd64,suite-wheezy,purpose-test,equiv-1
+test-amd64-i386-upgrade                               arch                     
   i386
+test-amd64-i386-upgrade                               buildjob                 
   build-i386
+test-amd64-i386-upgrade                               debian_arch              
   i386
+test-amd64-i386-upgrade                               debian_kernkind          
   pvops
+test-amd64-i386-upgrade                               kernbuildjob             
   build-i386-pvops
+test-amd64-i386-upgrade                               kernkind                 
   pvops
+test-amd64-i386-upgrade                               src_host_buildjob        
   build-i386-prev
+test-amd64-i386-upgrade                               src_host_xenbuildjob     
   build-amd64-prev
+test-amd64-i386-upgrade                               toolstack                
   xl
+test-amd64-i386-upgrade                               xenbuildjob              
   build-amd64

NB the regular build jobs are, as expected, unchanged (and different to the
ones above):

 build-amd64                                           revision_xen             
   9256f66c1606cd9339412bff7fbc7bd9e8beb28c
 build-i386                                            revision_xen             
   9256f66c1606cd9339412bff7fbc7bd9e8beb28c

Signed-off-by: Ian Campbell <ian.campbell@xxxxxxxxxx>
---
v4:
 - Create build-$arch-prev with no revision_xen runvar if REVISION_XEN
   not supplied.
 - Use "xl" not "$onetoolstack" which is now gone.
v3:
 - Handle a lack of REVISION_PREVXEN in make-flight
 - Do not use (undefined) $prevbfi, instead just use $bfi, use of
   $prevbfi is contingent on the TODO about reusing other builds
   (which is alonside a dummy variable actually called $prevxenbfi, so
   it wasn't even a correct reference to that)
v2:
 - Don't do this for Xen 4.0 and earlier
 - Update some comments and change some XXX to TODO
 - Match branch against xen-*
---
 cr-daily-branch |  4 ++++
 cri-common      | 20 ++++++++++++++++++++
 make-flight     | 16 ++++++++++++++++
 mfi-common      | 35 +++++++++++++++++++++++++++++++++++
 sg-run-job      | 14 ++++++++++++--
 ts-xen-install  |  3 ++-
 6 files changed, 89 insertions(+), 3 deletions(-)

diff --git a/cr-daily-branch b/cr-daily-branch
index 1fcfd9d..7402d3f 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -117,6 +117,10 @@ if [ "x$REVISION_XEN" = x ]; then
         REVISION_XEN="`fetch_version $xenbranch`"
         export REVISION_XEN
 fi
+if [ "x$REVISION_PREVXEN" = x -a "x$prevxenbranch" != "x" ]; then
+       REVISION_PREVXEN="`./ap-fetch-version-baseline $prevxenbranch`"
+       export REVISION_PREVXEN
+fi
 if [ "x$REVISION_QEMU_UPSTREAM" = x ]; then
        case "$xenbranch" in
         xen-3.*|xen-4.[01]-*)
diff --git a/cri-common b/cri-common
index 5c58482..94696ab 100644
--- a/cri-common
+++ b/cri-common
@@ -60,6 +60,24 @@ repo_tree_rev_fetch_git () {
        git rev-parse $localtag^0
 }
 
+select_prevxenbranch () {
+       local b
+       local p
+       for b in $(./mg-list-all-branches) ; do # already sorted by version
+               case "$b" in
+               xen*)
+                       if [ "x$b" = "x$xenbranch" ] ; then
+                               break
+                       else
+                               p=$b
+                       fi
+                       ;;
+               *)      ;;
+               esac
+       done
+       prevxenbranch=$p
+}
+
 select_xenbranch () {
        case "$branch" in
        xen-*)                  tree=xen;       xenbranch=$branch ;;
@@ -83,6 +101,8 @@ select_xenbranch () {
        if [ x$qemuubranch = x ]; then
                qemuubranch="qemu-upstream-${xenbranch#xen-}"
        fi
+
+       select_prevxenbranch
 }
 
 select_branch () {
diff --git a/make-flight b/make-flight
index cfb6540..00f797e 100755
--- a/make-flight
+++ b/make-flight
@@ -541,6 +541,22 @@ test_matrix_do_one () {
             all_hostflags=$most_hostflags,equiv-1
   done
 
+  if branch_wants_migrupgrade_tests ; then
+      # x86 only for now
+      if [ x$xenarch = xamd64 ] ; then
+      job_create_test test-$xenarch$kern-$dom0arch-migrupgrade 
test-pair-oneway \
+                      xl $xenarch $dom0arch \
+                      !host !host_hostflags \
+                      src_host_buildjob=${bfi}build-$dom0arch-prev \
+                      src_host_xenbuildjob=${bfi}build-$xenarch-prev \
+                      $debian_runvars \
+                      all_hostflags=$most_hostflags,equiv-1
+      fi
+      # TODO it would be nice to test i386->amd64 tools migration too (for
+      # Xen >= 4.6), but currently $r{arch} is used in several places,
+      # which would need looking at.
+  fi
+
   if [ x$test_pvh = xy -a $xenarch = amd64 -a $dom0arch = amd64 ]; then
 
     for cpuvendor in amd intel; do
diff --git a/mfi-common b/mfi-common
index c79dd11..991baf5 100644
--- a/mfi-common
+++ b/mfi-common
@@ -41,6 +41,15 @@ branch_wants_rumpkernel_tests () {
   esac
 }
 
+branch_wants_migrupgrade_tests () {
+  case "$branch" in
+    xen-3.*-testing) return 1 ;;
+    xen-4.0-testing) return 1 ;;
+    xen-*) return 0;;
+    *) return 1 ;;
+  esac
+}
+
 xenbranch_xsm_variants () {
     case "$xenbranch" in
     xen-3.*-testing) echo "false";;
@@ -161,6 +170,14 @@ create_build_jobs () {
     *) enable_ovmf=true;
     esac
 
+    want_prevxen=n
+    if branch_wants_migrupgrade_tests ; then
+        # Only x86 for now
+        if [ x$arch = xamd64 -o x$arch = xi386 ] ; then
+            want_prevxen=y
+        fi
+    fi
+
     eval "
         arch_runvars=\"\$ARCH_RUNVARS_$arch\"
     "
@@ -191,6 +208,24 @@ create_build_jobs () {
                 revision_ovmf=$REVISION_OVMF
     done
 
+    if [ x$want_prevxen = xy ] ; then
+        # TODO could find latest pass on that branch and attempt to reuse.
+        #bfiprevxen=...
+        #
+        # To avoid the need for TREE_PREVQEMU, TREE_PREVOVMF etc we
+        # only pass tree_xen and revision_xen here and therefore
+        # pickup the versions implicitly selected by the
+        # $REVISION_PREVXEN.
+        ./cs-job-create $flight build-$arch-prev build                       \
+                    arch=$arch enable_xend=false enable_ovmf=$enable_ovmf    \
+                    enable_xsm=false                                         \
+            tree_xen=$TREE_XEN                                               \
+                    $RUNVARS $BUILD_RUNVARS $BUILD_XEN_RUNVARS $arch_runvars \
+                    $suite_runvars                                           \
+                    host_hostflags=$build_hostflags                          \
+                    revision_xen=$REVISION_PREVXEN
+    fi
+
     if [ $build_extraxend = "true" ] ; then
     job_create_build build-$arch-xend build                                  \
                 arch=$arch enable_xend=true enable_ovmf=$enable_ovmf         \
diff --git a/sg-run-job b/sg-run-job
index c34ad5b..0b0449b 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -294,13 +294,16 @@ proc run-job/test-debianhvm {} {
     test-guest debianhvm
 }
 
-proc need-hosts/test-pair {} { return {src_host dst_host} }
-proc run-job/test-pair {} {
+proc setup-test-pair {} {
     run-ts . =              ts-debian-install      dst_host
     run-ts . =              ts-debian-fixup        dst_host          + debian
     run-ts . =              ts-guests-nbd-mirror + dst_host src_host + debian
     per-host-ts . =(*)     {ts-leak-check basis}
     run-ts . =              ts-guest-start       + src_host          + debian
+}
+proc need-hosts/test-pair {} { return {src_host dst_host} }
+proc run-job/test-pair {} {
+    setup-test-pair
     run-ts . =              ts-guest-migrate       src_host dst_host + debian
     run-ts . =              ts-guest-migrate       dst_host src_host + debian
     run-ts . =              ts-guest-stop          src_host          + debian
@@ -310,6 +313,13 @@ proc run-job/test-pair {} {
 #    run-ts . remus-failover ts-remus-check         src_host dst_host + debian
 }
 
+proc need-hosts/test-pair-oneway {} { return {src_host dst_host} }
+proc run-job/test-pair-oneway {} {
+    setup-test-pair
+    run-ts . =              ts-guest-migrate       src_host dst_host + debian
+    run-ts . =              ts-guest-stop          dst_host          + debian
+}
+
 proc test-guest-migr {g} {
     set to_reap [spawn-ts . = ts-migrate-support-check + host $g 1]
     set can_migrate [reap-ts $to_reap]
diff --git a/ts-xen-install b/ts-xen-install
index 0f53382..e9e4ada 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -75,7 +75,8 @@ sub extract () {
 
     foreach my $part (@parts) {
         target_extract_jobdistpath($ho, $part, "path_${part}dist",
-                                  $r{"${part}buildjob"}, \%distpath);
+               $r{"$ho->{Ident}_${part}buildjob"} // $r{"${part}buildjob"},
+               \%distpath);
     }
     if (target_file_exists($ho, "/usr/lib64/efi/xen.efi")) {
        target_cmd_root($ho,<<END);
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.