[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[OSSTEST PATCH 33/60] sg-report-job-history: Use fork-based parallelism


  • To: <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • From: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
  • Date: Fri, 14 Aug 2020 18:21:38 +0100
  • Authentication-results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
  • Cc: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
  • Delivery-date: Fri, 14 Aug 2020 17:22:40 +0000
  • Ironport-sdr: YROxpqmpJnw+hqp3Viw9ketkdbRBZUlJk2QHsBCtgt++Wq0+86m8LUPrRx6JoHJoDcXgaVZcV3 yRR24GZIeL6spqBiCIh86SY80w+orkcRLa2RsmPYNNAjaVoLJc50qx5VSBrEHMeOKhyU1yWA+I lNJVQUeCcdX4YvIK43+iTLu1oQkH32yrElpH+ZC4JPI0zr+vKp3enUwrMbi3xanjd6Z54HcJG+ UUmqtlT8DoyNysOTvgsk1j9wzMx5MtznEIr4lw1JqwDg9mZ+9KsZ3HtDdLZlf4oUVl0lEXW7Qi Hyc=
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

For now, one child per job (for all branches).  This is already a
speedup.

Signed-off-by: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
---
 sg-report-job-history | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/sg-report-job-history b/sg-report-job-history
index 424053f1..384a7670 100755
--- a/sg-report-job-history
+++ b/sg-report-job-history
@@ -26,11 +26,13 @@ use IO::Handle;
 use HTML::Entities;
 
 use Osstest::Executive;
+use Osstest::HistoryReport;
 
 our (@blessings,@branches);
 our $limit= 100;
 our $htmlout;
 our $flight;
+our $maxjobs=10;
 our $job;
 
 open DEBUG, ">/dev/null";
@@ -38,7 +40,7 @@ open DEBUG, ">/dev/null";
 while (@ARGV && $ARGV[0] =~ m/^-/) {
     $_= shift @ARGV;
     last if m/^--?$/;
-    if (m/^--(job|flight)\=(.*)$/) {
+    if (m/^--(job|flight|maxjobs)\=(.*)$/) {
         $$1= $2;
     } elsif (m/^--(limit)\=([1-9]\d*)$/) {
         $$1= $2;
@@ -300,6 +302,9 @@ sub processjob ($) {
     processjobbranch($j,$_) foreach @branches;
 }
 
-db_begin_work($dbh_tests, []);
-db_readonly_report();
-foreach my $j (@jobs) { processjob($j); }
+parallel_by_fork('sg-report-job-history', $maxjobs, \@jobs, sub {
+    my ($j) = @_;
+    db_retry($dbh_tests, [], sub {
+        processjob($j);
+    });
+});
-- 
2.11.0




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.