[OSSTEST PATCH 00/60] Speed up sg-report-job-history

[Ian Jackson <ian.jackson@xxxxxxxxxxxxx>]

I think this is the final section of my work to speed up osstest's
reporting db queries etc.

After this is pushed and working, I will look and see what the
interval is now, between one flight finishing and the next starting,
to see ig there is more to be done.

Ian Jackson (60):
  history reporting (nfc): Do not key cache on hostname any more
  history reporting (nfc): Add some test runes to the notes
  history reporting: Delete two debug prints
  history reporting (nfc): Refactor to generalise, cache_read_existing
  history reporting (nfc): Make cache_write_entry into a top-level sub
  history reporting (nfc): Rename jobquery to cacheable_query
  history reporting (nfc): Break out cache_row_lookup_prep
  history reporting (nfc): Rename "existing" to "previous"
  history reporting (nfc): Provide cacheable_fn
  history reporting (nfc): Bind by name in cacheable_query
  history reporting (nfc): Rename $cachehits to $rows_hit
  history reporting (nfc): Record query-specific stats
  history reporting (nfc): Move cacheable_* further up the file
  history reporting (nfc): Use cacheable_fn for power methods
  history reporting (nfc): Introduce cache_set_key_cols
  history reporting (nfc): Record more row-specific stats
  history reporting: Cache stats reporting: Centralise and rework
  history reporting (nfc): Remove now-obsolete stats variables
  history reporting (nfc): Introduce empty HistoryReport module
  history reporting (nfc): Move cache code into HistoryReport module
  history reporting (nfc): Rename some module variables, remove "cache"
  history reporting: Skip undefined keys
  history reporting (nfc): Documentation for the new module
  history reporting: Cache data limit now in History module
  history reporting: Print debug for cache misses
  history reporting: Improve an error message slightly
  sg-report-host-history: Write cache entry for unfinished jobs
  parallel by fork: Break out into HistoryReport
  parallel by fork: Disconnect $dbh_tests as well as undefing it
  parallel by fork: Fix a variable name to $task
  sg-report-job-history: Prep for fork: Move $buildsq query
  sg-report-job-history: Prep for fork: Move $revisionsq query
  sg-report-job-history: Use fork-based parallelism
  sg-report-job-history: Use one child per report
  sg-report-job-history (nfc): Have main program decide HTML filename
  sg-report-job-history: Always write HTML output
  sg-report-job-history (nfc): Remove needless conditional
  history reporting (nfc): Add another test rune to the notes
  history reporting (nfc): Make cacheable_fn work without cache
  sg-report-job-history: Refactor "ALL" handling
  sg-report-job-history (nfc): Make $ri->{Hosts} a hash
  sg-report-job-history (nfc): Add new hostvarcols calculation
  sg-report-job-history (nfc): Query hosts runvars in one go
  sg-report-job-history (nfc): Drop $hostsq query
  history reporting (nfc): Provide cache_set_task_print
  sg-report-job-history: Introduce use of cache, for hosts query
  history reporting (nfc): Break out url_quote
  history reporting (nfc): Break out url_unquote
  history reporting (nfc): Break out $url_ok_chars
  history reporting (nfc): url-quoting: quote = too
  history reporting (nfc): Quote keys too
  sg-report-job-history: Cache the per-flight revisions
  sg-report-job-history (nfc): Refactor osstestrevs code
  sg-report-job-history: Cache osstestrevs
  history reporting: Break out minflight_by_time
  sg-report-job-history (nfc): Abolish $fromstuff
  sg-report-job-history: Cache report_run_getinfo
  sg-report-host-history: Cache report_run_getinfo
  sg-report-job-history: Provide --time-limit
  sg-report-job-history: Increase default limit

 Osstest/HistoryReport.pm | 301 +++++++++++++++++++++++++++++++++++++++++++++
 runes                    |  10 ++
 sg-report-host-history   | 216 +++++++-------------------------
 sg-report-job-history    | 313 +++++++++++++++++++++++++----------------------
 4 files changed, 520 insertions(+), 320 deletions(-)
 create mode 100644 Osstest/HistoryReport.pm

-- 
2.11.0