[Xen-devel] [OSSTEST PATCH 3/3] builds: Run i386 builds on amd64 kernels

[Ian Jackson <ian.jackson@xxxxxxxxxxxxx>]

Most hardware that supports i386 supports amd64 too.  When doing
builds we do need the right userland, but we don't actually care what
the kernel is doing.  With Linux 32-on-64 is good for that.

Especially, there is a kernel regression (evident in the Debian
stretch kernel, but not present in jessie's) where 32-bit Linux
mismanages the memory on hosts with moderately large amounts of
RAM (ie, significantly more RAM than can be addressed at once),
resulting in what amounts to a near-stall of the paging system.  Since
the paging system is used for filesystem writes too, the effect is
that commands run by builds can take totally unreasonable amounts of
time.  Ie, this version of Linux is broken when i386 PAE is needed.

In practice this is causing significant trouble in the Xen Project CI.

This kernel bug probably won't affect our test jobs because
(i) we use our own kernels, so we would probably detect this
regression when switching kernel branches etc. (ii) test jobs
run with a dom0_mem setting which avoids the preconditions for the
particular bug.

CC: Juergen Gross <jgross@xxxxxxxx>
CC: Boris Ostrovsky <boris.ostrovsky@xxxxxxxxxx>
CC: Wei Liu <wei.liu2@xxxxxxxxxx>
CC: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
CC: Paul Durrant <paul.durrant@xxxxxxxxxx>
Signed-off-by: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
---
 Osstest/Debian.pm | 35 +++++++++++++++++++++++++++++++++++
 ts-host-install   |  8 ++++++++
 2 files changed, 43 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 600f18b1..0f10a9c3 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1334,6 +1334,41 @@ END
        preseed_hook_command($ho, 'late_command', $sfx, $cmds);
     });
 
+    my $kern_arch = $xopts{LinuxKernArchMap}{$r{arch}};
+    if ($kern_arch &&
+       $ho->{Flags}{"arch-$r{arch}"} && !$ho->{Flags}{"arch-$kern_arch"}) {
+       # We check the flag for $r{arch} because for various reasons
+       # it may not be set.  For example, in standalone mode it is
+       # currently not normally necessary to set any arch hostflags.
+       # In that situation we bet that the kern arch can be used;
+       # this is a good bet because currently this is used for
+       # running amd64 kernels with i386 userland and hosts that can
+       # do only i386 are fairly rare, especially server hosts.
+       # If this causes a problem, setting the i386 host flag will
+       # (somewhat counterintuitively) cause this check to miss
+       # the amd64 host flag
+       logm("NOT using $kern_arch kernel (flags say not supported by host)");
+       $kern_arch = undef;
+    }
+    if ($kern_arch) {
+       logm("Using $kern_arch kernel for $r{arch} userland");
+       preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+in-target sh -xec '
+    # the $r{arch} kernels end up being the default so shuffle them
+    # where update-grub will not find them
+    for f in /boot/vmlinu*; do
+        dpkg-divert --rename --divert "/boot/~disable~\${f##*/}" "\$f"
+    done
+    dpkg --add-architecture $kern_arch
+    apt-get update
+    apt-get install -f -y linux-image-$kern_arch
+    update-grub
+'
+END
+    }
+
     if ( $ho->{Flags}{'need-uboot-bootscr'} ) {
        my @bootargs = uboot_common_kernel_bootargs($ho);
 
diff --git a/ts-host-install b/ts-host-install
index 7094559f..35601a42 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -31,6 +31,7 @@ my $debconf_priority;
 my $poweron_test_only;
 
 our %xopts;
+our %linux_kern_arch_map;
 
 while (@ARGV and $ARGV[0] =~ m/^-/) {
     $_= shift @ARGV;
@@ -42,6 +43,12 @@ while (@ARGV and $ARGV[0] =~ m/^-/) {
     } elsif  (m/^--rescue$/) {
        $xopts{RescueMode}= 1;
     } elsif  (m/^--build$/) {
+       # When installing to do builds rather than tests, we can use an
+       # amd64 kernel on i386 hosts, and this is generally faster.
+       # Also, it avoids a kernel regression in Debian stretch -
+       # a regression which Linux upstream do not consider a bug.  Eg
+       #  https://lkml.org/lkml/2017/8/2/394
+       $linux_kern_arch_map{i386} = 'amd64';
     } else {
        die "$_ $!";
     }
@@ -61,6 +68,7 @@ sub install () {
         ($ho, '',
          DiskDevice => $ho->{DiskDevice},
          Properties => $ho->{Properties},
+        LinuxKernArchMap => \%linux_kern_arch_map,
          ExtraPreseed => <<END );
 d-i netcfg/get_hostname string $ho->{Name}
 d-i netcfg/get_ipaddress string $ho->{Ip}
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel