[Xen-devel] [OSSTEST PATCH 00/13] Speed up and restore host history

[Ian Jackson <ian.jackson@xxxxxxxxxxxxx>]

Earlier this week we discovered that sg-report-host-history was running
extremely slowly.  We applied an emergency fix 0fa72b13f5af
  sg-report-host-history: Reduce limit from 2000 to 200

The main problem is that sg-report-host-history runs once for each
flight, and must generate a relevant history view of the recent
history for each host - including much history that is already in the
old version of the html file.

The slow part is asking the database about information about each job,
including its final step, allocation step, etc.  (The main query which
digs out relevant jobs is also rather time consuming it runs all in
one go and takes only a minute or two.)

In this series we introduce a mechanism which caches much of the
historical analysis.

It is not straightforward to reuse old html data as-is because we
would have to do a merge sort with the new data and that would involve
rewriting the alternating background colour (!)

So instead, we stuff the information we got from the database into
comments in the HTML, which we can then scan on future runs.

The overall result is a factor of 10 speedup in my tests, for the
original history limit of 2000.  That is now fast enough we can put
it back.

(I was not able to reproduce the exceptional case I saw earlier in the
week, where it was apparently taking hours.  I suspect that there
comes a tipping point where db transactions end up being restarted.)

The patches are broken down into small pieces so that I could think
about them clearly and do self-review.

Ian Jackson (13):
  sg-report-host-history: Improve debugging output
  sg-report-host-history: New --no-install option for testing
  sg-report-host-history: Move `computeflightsrange' after hosts
  sg-report-host-history: Actually honour $minflight
  sg-report-host-history: Get job status from mainquery
  sg-report-host-history: Add $cachekey argument to jobquery
  sg-report-host-history: Store per-job query results in %$jr
  sg-report-host-history: Write cache entries
  sg-report-host-history: Write cache entries for tail, too
  sg-report-host-history: Read cache entries
  sg-report-host-history: Move job runvars query later
  sg-report-host-history: Cache runvar queries (power information)
  Revert "sg-report-host-history: Reduce limit from 2000 to 200"

 sg-report-host-history | 189 +++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 160 insertions(+), 29 deletions(-)

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel