[Xen-devel] [OSSTEST PATCH 5/9] db retry, bisect: Cache build reuse investigations

[Ian Jackson <ian.jackson@xxxxxxxxxxxxx>]

If we previously searched for builds to reuse, trust our previous
answers.  We will only have seen data from committed transactions and
we will only have looked at jobs in completed flights, which won't
have changed.

So any previously reuseable build is still reuseable.  (Unless its
stash check failed, in which case we might want to search again if we
do a db retry.)  We don't care much about missing any
recently-finished jobs - there is a much bigger and unavoidable race
there anyway, where multiple bisections on different branches may
choose to pointlessly rebuild the same thing at the same time.

Not doing this search over and over again is important because it is a
very wide ranging search, which will often cause database transaction
serialisation errors.  Without some caching here, we may never
converge.

In principle we could do this another way: we could make a readonly
transaction which did all the searching.  But that's a more awkward
way to organise the code because our search uses a temporary table
which we then construct the flight from.

Signed-off-by: Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>
---
 cs-bisection-step | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/cs-bisection-step b/cs-bisection-step
index 1d1962a..819e519 100755
--- a/cs-bisection-step
+++ b/cs-bisection-step
@@ -1121,6 +1121,7 @@ END
 }
 
 our %jobs_created;
+our %builds_investigated; # $builds_investigated{$popjob} = 0, or {..row..}
 
 sub preparejob ($$$);
 sub preparejob ($$$) {
@@ -1194,7 +1195,11 @@ END
     my $usejob;
 
     if ($cache_option and $cacheok and $recipe =~ m/^build/ and !@$subjobs) {
-        my $reusejob= $dbh_tests->selectrow_hashref(<<END,{}, $popjob,$popjob);
+       my $reusejob= $builds_investigated{$popjob};
+       if (!defined $reusejob) {
+           print STDERR "Searching for $popflight.$popjob to reuse...\n";
+           $reusejob =
+               $dbh_tests->selectrow_hashref(<<END,{}, $popjob,$popjob);
             SELECT  *
             FROM    flights JOIN jobs j USING (flight)
             WHERE   j.job=?
@@ -1216,6 +1221,9 @@ END
             ORDER BY flights.started desc
             LIMIT 1
 END
+           $reusejob //= 0; # defined but falseish
+           $builds_investigated{$popjob}= $reusejob;
+       }
 
         if ($reusejob) {
            my $wantusejob= "$reusejob->{flight}.$reusejob->{job}";
@@ -1226,6 +1234,7 @@ END
            } else {
                print STDERR "Can NOT reuse $recipe $wantusejob:".
                    " $stashcheck: $!\n";
+               undef $builds_investigated{$popjob};
            }
         }
     }
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel