[Xen-devel] [OSSTEST PATCH] make-flight: Trim the matrix of disk format flights

[Ian Jackson <ian.jackson@xxxxxxxxxxxxx>]

We don't need to test every combination of toolstack, architecture,
and disk format.  We don't expect many architecture-specific bugs in
the per-disk-format code in the toolstack layers.

We _do_ want to test every combination of toolstack and disk format
(since the format configuration machinery is toolstack specific) and a
reasonable selection of architectures for each disk format (since
arch-specific bugs in actual underlying disk drivers are a
possibility).

The implementation strategy is for do_pv_debian_tests to select a
particular architecture for each combination of toolstack and format.
(Because the architecture is actually in an outer loop, we recalculate
that selection multiple times, and skip inner iterations for the other
architectures.  This is all in bash code so the wasted computation is
not particularly important.)

We have a safety catch which spots if any architecture is entirely
untested in any of these combinations; this would happen if a new
architecture is introduced elsewhere and not added to the list.  We do
not have a safety catch which spots when a (toolstack,format)
combination becomes untested due to deletion of an architecture.
(That would be more fiddly to implement without restructuring.)

We list armhf twice because we would like to do at least as many ARM
as x86 tests (particularly given our current workload and capacity).

The result is that the set of generated jobs is adjusted as follows:

   keep  test-amd64-i386-xl-raw
     -   test-amd64-i386-xl-vhd
     -   test-amd64-i386-xl-qcow2
     -   test-amd64-i386-libvirt-raw
     -   test-amd64-i386-libvirt-vhd
     -   test-amd64-i386-libvirt-qcow2
     -   test-amd64-amd64-xl-raw
     -   test-amd64-amd64-xl-vhd
   keep  test-amd64-amd64-xl-qcow2
     -   test-amd64-amd64-libvirt-raw
   keep  test-amd64-amd64-libvirt-vhd
     -   test-amd64-amd64-libvirt-qcow2
     -   test-armhf-armhf-xl-raw
   keep  test-armhf-armhf-xl-vhd
     -   test-armhf-armhf-xl-qcow2
   keep  test-armhf-armhf-libvirt-raw
     -   test-armhf-armhf-libvirt-vhd
   keep  test-armhf-armhf-libvirt-qcow2

(Where `-' means `drop this job'.)

Or to look at it another way:

                         i386   amd64   armhf

                 raw     keep     -       -
                 vhd       -      -     keep
                 qcow2     -    keep      -

         libvirt raw       -      -     keep
         libvirt vhd       -    keep      -
         libvirt qcow2     -      -     keep

I have diffed standalone-generate-dump-flight-runvars output before
and after and it looks plausible.

Signed-off-by: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
---
 make-flight |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/make-flight b/make-flight
index 70c909d..8aab28d 100755
--- a/make-flight
+++ b/make-flight
@@ -460,10 +460,29 @@ do_pv_debian_tests () {
     do_pv_debian_test_one libvirt '' libvirt '' enable_xsm=$xsm
   done
 
+  # We compute a desired architecture for each combination of ts and fmt
+  # Within each fmt we rotate through the list of arches
+  # The starting list rotates once per ts, so that we try to
+  # exercise each fmt on each arch family.
+  local fmtarches_outer="i386 armhf amd64 armhf"
+  local endfmt="do_pv_debian_tests-missing-ts-fmt-for-dom0arch="
+
   for ts in xl libvirt ; do
 
+    local fmtarches=$fmtarches_outer
+    fmtarches_outer="${fmtarches_outer#* } ${fmtarches_outer%% *}"
+
     for fmt in raw vhd qcow2 ; do
 
+      local fmtarch="${fmtarches%% *}"
+      fmtarches="${fmtarches#* } $fmtarch"
+
+      if [ "x$fmtarch" != "x$dom0arch" ]; then
+        continue
+      else
+        endfmt=": "
+      fi
+
       fmt_runvar="debian_diskfmt=$fmt"
 
       do_pv_debian_test_one $ts-$fmt '-di' $ts '' \
@@ -475,6 +494,8 @@ do_pv_debian_tests () {
 
     done
   done
+
+  $endfmt$dom0arch
 }
 
 test_matrix_do_one () {
-- 
1.7.10.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel