[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[xen-unstable-smoke bisection] complete build-arm64-xsm



branch xen-unstable-smoke
xenbranch xen-unstable-smoke
job build-arm64-xsm
testid xen-build/dist-test

Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  25164571fc11ed3010c5885a98a68fac3b891d33
  Bug not present: 0cd791c499bdc698d14a24050ec56d60b45732e0
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/145004/


  commit 25164571fc11ed3010c5885a98a68fac3b891d33
  Merge: 0cd791c499 5083e0ff93
  Author: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
  Date:   Thu Dec 19 20:16:43 2019 -0500
  
      Merge branch 'livepatch.aws.v6' into staging
      
      * livepatch.aws.v6:
        livepatch: Add metadata runtime retrieval mechanism
        livepatch: Handle arbitrary size names with the list operation
        livepatch: Add support for modules .modinfo section metadata
        livepatch: Add support for inline asm livepatching expectations
        livepatch: Add per-function applied/reverted state tracking marker
        livepatch: Do not enforce ELF_LIVEPATCH_FUNC section presence
        livepatch: Add support for apply|revert action replacement hooks
        livepatch: Implement pre-|post- apply|revert hooks
        livepatch: Export payload structure via livepatch_payload.h
        livepatch: Allow to override inter-modules buildid dependency
        livepatch: Always check hypervisor build ID upon livepatch upload
      
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
  
  commit 5083e0ff939d149860db40e0da54ea2048749471
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:08:00 2019 +0000
  
      livepatch: Add metadata runtime retrieval mechanism
      
      Extend the livepatch list operation to fetch also payloads' metadata.
      This is achieved by extending the sysctl list interface with 2 extra
      guest handles:
      * metadata     - an array of arbitrary size strings
      * metadata_len - an array of metadata strings' lengths (uin32_t each)
      
      Payloads' metadata is a string of arbitrary size and does not have an
      upper bound limit. It may also vary in size between payloads.
      
      In order to let the userland allocate enough space for the incoming
      data add a metadata total size field to the list sysctl operation and
      fill it with total size of all payloads' metadata.
      
      Extend the libxc to handle the metadata back-to-back data transfers
      as well as metadata length array data transfers.
      
      The xen-livepatch userland tool is extended to always display the
      metadata for each received module. The metadata is received with the
      following format: key=value\0key=value\0...key=value\0. The format is
      modified to the following one: key=value;key=value;...key=value.
      The new format allows to easily parse the metadata for a given module
      by a machine.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Reviewed-by: Norbert Manthey <nmanthey@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit b145b4a39c1324186b1b43313a9fefc19b7aa43f
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:59 2019 +0000
  
      livepatch: Handle arbitrary size names with the list operation
      
      The payloads' name strings can be of arbitrary size (typically small
      with an upper bound of XEN_LIVEPATCH_NAME_SIZE).
      Current implementation of the list operation interface allows to copy
      names in the XEN_LIVEPATCH_NAME_SIZE chunks regardless of its actual
      size and enforces space allocation requirements on userland tools.
      
      To unify and simplify the interface, handle the name strings of
      arbitrary size by copying them in adhering chunks to the userland.
      In order to let the userland allocate enough space for the incoming
      data add an auxiliary interface xc_livepatch_list_get_sizes() that
      provides the current number of payload entries and the total size of
      all name strings. This is achieved by extending the sysctl list
      interface with an extra fields: name_total_size.
      
      The xc_livepatch_list_get_sizes() issues the livepatch sysctl list
      operation with the nr field set to 0. In this mode the operation
      returns the number of payload entries and calculates the total sizes
      for all payloads' names.
      When the sysctl operation is issued with a non-zero nr field (for
      instance with a value obtained earlier with the prior call to the
      xc_livepatch_list_get_sizes()) the new field name_total_size provides
      the total size of actually copied data.
      
      Extend the libxc to handle the name back-to-back data transfers.
      
      The xen-livepatch tool is modified to start the list operation with a
      call to the xc_livepatch_list_get_sizes() to obtain the actual number
      of payloads as well as the necessary space for names.
      The tool now always requests the actual number of entries and leaves
      the preemption handling to the libxc routine. The libxc still returns
      'done' and 'left' parameters with the same semantic allowing the tool
      to detect anomalies and react to them. At the moment it is expected
      that the tool receives the exact number of entries as requested.
      The xen-livepatch tool has been also modified to handle the name
      back-to-back transfers correctly.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Bjoern Doebel <doebel@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 4848297ad42135ee8e7e1e6e14b3855ceaf3eb08
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:58 2019 +0000
  
      livepatch: Add support for modules .modinfo section metadata
      
      Having detailed livepatch metadata helps to properly identify module's
      origin and version. It also allows to keep track of the history of
      livepatch loads in the system (at least within dmesg buffer size
      limits).
      
      The livepatch metadata are embedded in a form of .modinfo section.
      Each such section contains data of the following format:
      key=value\0key=value\0...key=value\0
      
      The .modinfo section may be generated and appended to the resulting
      livepatch ELF file optionally as an extra step of a higher level
      livepatch build system.
      
      The metadata section pointer and the section length is stored in the
      livepatch payload structure and is used to display the content upon
      livepatch apply operation.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Bjoern Doebel <doebel@xxxxxxxxx>
      Reviewed-by: Leonard Foerster <foersleo@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Reviewed-by: Norbert Manthey <nmanthey@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 8e24c887887a95cb2dda0017569ed19b65670152
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:57 2019 +0000
  
      livepatch: Add support for inline asm livepatching expectations
      
      This is the initial implementation of the expectations enhancement
      to improve inline asm livepatching.
      
      Expectations are designed as optional feature, since the main use of
      them is planned for inline asm livepatching. The flag enabled allows
      to control the expectation state.
      Each expectation has data and len fields that describe the data
      that is expected to be found at a given patching (old_addr) location.
      The len must not exceed the data array size. The data array size
      follows the size of the opaque array, since the opaque array holds
      the original data and therefore must match what is specified in the
      expectation (if enabled).
      
      The payload structure is modified as each expectation structure is
      part of the livepatch_func structure and hence extends the payload.
      
      Each expectation is checked prior to the apply action (i.e. as late
      as possible to check against the most current state of the code).
      
      For the replace action a new payload's expectations are checked AFTER
      all applied payloads are successfully reverted, but BEFORE new payload
      is applied. That breaks the replace action's atomicity and in case of
      an expectation check failure would leave a system with all payloads
      reverted. That is obviously insecure. Use it with caution and act
      upon replace errors!
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Reviewed-by: Norbert Manthey <nmanthey@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 6047104c3ccc50205464a9b6a90daa85d21a4798
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:56 2019 +0000
  
      livepatch: Add per-function applied/reverted state tracking marker
      
      Livepatch only tracks an entire payload applied/reverted state. But,
      with an option to supply the apply_payload() and/or revert_payload()
      functions as optional hooks, it becomes possible to intermix the
      execution of the original apply_payload()/revert_payload() functions
      with their dynamically supplied counterparts.
      It is important then to track the current state of every function
      being patched and prevent situations of unintentional double-apply
      or unapplied revert.
      
      To support that, it is necessary to extend public interface of the
      livepatch. The struct livepatch_func gets additional field holding
      the applied/reverted state marker.
      
      To reflect the livepatch payload ABI change, bump the version flag
      LIVEPATCH_PAYLOAD_VERSION up to 2.
      
      [And also update the top of the design document]
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Bjoern Doebel <doebel@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Acked-by: Julien Grall <julien.grall@xxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 76b3d4098a92a323a43bc250c67c721c1eed0acb
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:55 2019 +0000
  
      livepatch: Do not enforce ELF_LIVEPATCH_FUNC section presence
      
      With default implementation the ELF_LIVEPATCH_FUNC section containing
      all functions to be replaced or added must be part of the livepatch
      payload, otherwise the payload is rejected (with -EINVAL).
      
      However, with the extended hooks implementation, a livepatch may be
      constructed of only hooks to perform certain actions without any code
      to be added or replaced.
      Therefore, do not always expect the functions section and allow it to
      be missing, provided there is at least one section containing hooks
      present. The functions section, when present in a payload, must be a
      single, non-empty section.
      
      Check also all extended hooks sections if they are a single, non-empty
      sections each.
      
      At least one of the functions or hooks section must be present in a
      valid payload.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Bjoern Doebel <doebel@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit ef87efee9d38b61624f25c1a056d386a70ba99aa
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:54 2019 +0000
  
      livepatch: Add support for apply|revert action replacement hooks
      
      By default, in the quiescing zone, a livepatch payload is applied with
      apply_payload() and reverted with revert_payload() functions. Both of
      the functions receive the payload struct pointer as a parameter. The
      functions are also a place where standard 'load' and 'unload' module
      hooks are executed.
      
      To increase livepatching system's agility and provide more flexible
      long-term livepatch solution, allow to overwrite the default apply
      and revert action functions with hook-like supplied alternatives.
      The alternative functions are optional and the default functions are
      used by default.
      
      Since the alternative functions have direct access to the livepatch
      payload structure, they can better control context of the 'load' and
      'unload' hooks execution as well as exact instructions replacement
      workflows. They can be also easily extended to support extra features
      in the future.
      
      To simplify the alternative function generation move code responsible
      for payload and livepatch region registration outside of the function.
      That way it is guaranteed that the registration step occurs even for
      newly supplied functions.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Petre Eftime <epetre@xxxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxxx>
      Reviewed-by: Norbert Manthey <nmanthey@xxxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Bjoern Doebel <doebel@xxxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 8313c864fa95074c2176f19af711b7e13bf20504
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:53 2019 +0000
  
      livepatch: Implement pre-|post- apply|revert hooks
      
      This is an implementation of 4 new livepatch module vetoing hooks,
      that can be optionally supplied along with modules.
      Hooks that currently exists in the livepatch mechanism aren't agile
      enough and have various limitations:
      * run only from within a quiescing zone
      * cannot conditionally prevent applying or reverting
      * do not have access to the module context
      To address these limitations the following has been implemented:
      1) pre-apply hook
        runs before the apply action is scheduled for execution. Its main
        purpose is to prevent from applying a livepatch when certain
        expected conditions aren't met or when mutating actions implemented
        in the hook fail or cannot be executed.
      
      2) post-apply hook
        runs after the apply action has been executed and quiescing zone
        exited. Its main purpose is to provide an ability to follow-up on
        actions performed by the pre- hook, when module application was
        successful or undo certain preparation steps of the pre- hook in
        case of a failure. The success/failure error code is provided to
        the post- hooks via the rc field of the payload structure.
      
      3) pre-revert hook
        runs before the revert action is scheduled for execution. Its main
        purpose is to prevent from reverting a livepatch when certain
        expected conditions aren't met or when mutating actions implemented
        in the hook fail or cannot be executed.
      
      4) post-revert hook
        runs after the revert action has been executed and quiescing zone
        exited. Its main purpose is to perform cleanup of all previously
        executed mutating actions in order to restore the original system
        state from before the current module application.
        The success/failure error code is provided to the post- hooks via
        the rc field of the payload structure.
      
      The replace action performs atomically the following actions:
      - revert all applied modules
      - apply a single replacement module.
      With the vetoing hooks in place various inter-hook dependencies may
      arise. Also, during the revert part of the operation certain vetoing
      hooks may detect failing conditions that previously were satisfied.
      That could in turn lead to situation when the revert part must be
      rolled back with all the pre- and post- hooks re-applied, which again
      can't be guaranteed to always succeed.
      The simplest response to this complication is to disallow the replace
      action completely on modules with vetoing hooks.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Petre Eftime <epetre@xxxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Reviewed-by: Norbert Manthey <nmanthey@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 3bafe4a06051de1d7abfffe77c8b9cb58594f39f
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:52 2019 +0000
  
      livepatch: Export payload structure via livepatch_payload.h
      
      The payload structure will be used by the new hooks implementation and
      therefore its definition has to be exported via the livepatch_payload
      header.
      The new hooks will make use of the payload structure fields and the
      hooks' pointers will also be defined in the payload structure, so
      the structure along with all field definitions needs to be available
      to the code being patched in.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Eslam Elnikety <elnikety@xxxxxxxxx>
      Reviewed-by: Leonard Foerster <foersleo@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
  
  commit b274989b610d37f0775e93c08343d30ec267a80f
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:51 2019 +0000
  
      livepatch: Allow to override inter-modules buildid dependency
      
      By default Livepatch enforces the following buildid-based dependency
      chain between livepatch modules:
        1) first module depends on given hypervisor buildid
        2) every consecutive module depends on previous module's buildid
      This way proper livepatch stack order is maintained and enforced.
      While it is important for production livepatches it limits agility and
      blocks usage of testing or debug livepatches. These kinds of livepatch
      modules are typically expected to be loaded at any time irrespective
      of current state of the modules stack.
      
      To enable testing and debug livepatches allow user dynamically ignore
      the inter-modules dependency. In this case only hypervisor buildid
      match is verified and enforced.
      
      To allow userland pass additional paremeters for livepatch actions
      add support for action flags.
      Each of the apply, revert, unload and revert action gets additional
      32-bit parameter 'flags' where extra flags can be applied in a mask
      form.
      Initially only one flag '--nodeps' is added for the apply action.
      This flag modifies the default buildid dependency check as described
      above.
      The global sysctl interface input flag parameter is defined with a
      single corresponding flag macro:
        LIVEPATCH_ACTION_APPLY_NODEPS (1 << 0)
      
      The userland xen-livepatch tool is modified to support the '--nodeps'
      flag for apply and load commands. A general mechanism for specifying
      more flags in the future for apply and other action is however added.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Eslam Elnikety <elnikety@xxxxxxxxx>
      Reviewed-by: Petre Eftime <epetre@xxxxxxxxxx>
      Reviewed-by: Leonard Foerster <foersleo@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Reviewed-by: Norbert Manthey <nmanthey@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 879615f5db1d0a86afd99a67d284a8df6fd85be4
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:50 2019 +0000
  
      livepatch: Always check hypervisor build ID upon livepatch upload
      
      This change is part of a independant stacked livepatch modules
      feature. This feature allows to bypass dependencies between modules
      upon loading, but still verifies Xen build ID matching.
      
      In order to prevent (up)loading any livepatches built for different
      hypervisor version as indicated by the Xen Build ID, add checking for
      the payload's vs Xen's build id match.
      
      To achieve that embed into every livepatch another section with a
      dedicated hypervisor build id in it. After the payload is loaded and
      the .livepatch.xen_depends section becomes available, perform the
      check and reject the payload if there is no match.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Bjoern Doebel <doebel@xxxxxxxxx>
      Reviewed-by: Eslam Elnikety <elnikety@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-unstable-smoke/build-arm64-xsm.xen-build--dist-test.html
Revision IDs in each graph node refer, respectively, to the Trees above.

----------------------------------------
Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/xen-unstable-smoke/build-arm64-xsm.xen-build--dist-test
 --summary-out=tmp/145004.bisection-summary --basis-template=144983 
--blessings=real,real-bisect xen-unstable-smoke build-arm64-xsm 
xen-build/dist-test
Searching for failure / basis pass:
 144991 fail [host=laxton1] / 144983 ok.
Failure / basis pass flights: 144991 / 144983
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest 933ebad2470a169504799a1d95b8e410bd9847ef 
25164571fc11ed3010c5885a98a68fac3b891d33
Basis pass 933ebad2470a169504799a1d95b8e410bd9847ef 
0cd791c499bdc698d14a24050ec56d60b45732e0
Generating revisions with ./adhoc-revtuple-generator  
git://xenbits.xen.org/qemu-xen.git#933ebad2470a169504799a1d95b8e410bd9847ef-933ebad2470a169504799a1d95b8e410bd9847ef
 
git://xenbits.xen.org/xen.git#0cd791c499bdc698d14a24050ec56d60b45732e0-25164571fc11ed3010c5885a98a68fac3b891d33
Loaded 5001 nodes in revision graph
Searching for test results:
 144983 pass 933ebad2470a169504799a1d95b8e410bd9847ef 
0cd791c499bdc698d14a24050ec56d60b45732e0
 144991 fail 933ebad2470a169504799a1d95b8e410bd9847ef 
25164571fc11ed3010c5885a98a68fac3b891d33
 144997 pass 933ebad2470a169504799a1d95b8e410bd9847ef 
0cd791c499bdc698d14a24050ec56d60b45732e0
 145001 fail 933ebad2470a169504799a1d95b8e410bd9847ef 
25164571fc11ed3010c5885a98a68fac3b891d33
 145003 pass 933ebad2470a169504799a1d95b8e410bd9847ef 
0cd791c499bdc698d14a24050ec56d60b45732e0
 145004 fail 933ebad2470a169504799a1d95b8e410bd9847ef 
25164571fc11ed3010c5885a98a68fac3b891d33
Searching for interesting versions
 Result found: flight 144983 (pass), for basis pass
 Result found: flight 144991 (fail), for basis failure
 Repro found: flight 144997 (pass), for basis pass
 Repro found: flight 145001 (fail), for basis failure
 0 revisions at 933ebad2470a169504799a1d95b8e410bd9847ef 
0cd791c499bdc698d14a24050ec56d60b45732e0
No revisions left to test, checking graph state.
 Result found: flight 144983 (pass), for last pass
 Result found: flight 144991 (fail), for first failure
 Repro found: flight 144997 (pass), for last pass
 Repro found: flight 145001 (fail), for first failure
 Repro found: flight 145003 (pass), for last pass
 Repro found: flight 145004 (fail), for first failure

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  25164571fc11ed3010c5885a98a68fac3b891d33
  Bug not present: 0cd791c499bdc698d14a24050ec56d60b45732e0
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/145004/


  commit 25164571fc11ed3010c5885a98a68fac3b891d33
  Merge: 0cd791c499 5083e0ff93
  Author: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
  Date:   Thu Dec 19 20:16:43 2019 -0500
  
      Merge branch 'livepatch.aws.v6' into staging
      
      * livepatch.aws.v6:
        livepatch: Add metadata runtime retrieval mechanism
        livepatch: Handle arbitrary size names with the list operation
        livepatch: Add support for modules .modinfo section metadata
        livepatch: Add support for inline asm livepatching expectations
        livepatch: Add per-function applied/reverted state tracking marker
        livepatch: Do not enforce ELF_LIVEPATCH_FUNC section presence
        livepatch: Add support for apply|revert action replacement hooks
        livepatch: Implement pre-|post- apply|revert hooks
        livepatch: Export payload structure via livepatch_payload.h
        livepatch: Allow to override inter-modules buildid dependency
        livepatch: Always check hypervisor build ID upon livepatch upload
      
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
  
  commit 5083e0ff939d149860db40e0da54ea2048749471
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:08:00 2019 +0000
  
      livepatch: Add metadata runtime retrieval mechanism
      
      Extend the livepatch list operation to fetch also payloads' metadata.
      This is achieved by extending the sysctl list interface with 2 extra
      guest handles:
      * metadata     - an array of arbitrary size strings
      * metadata_len - an array of metadata strings' lengths (uin32_t each)
      
      Payloads' metadata is a string of arbitrary size and does not have an
      upper bound limit. It may also vary in size between payloads.
      
      In order to let the userland allocate enough space for the incoming
      data add a metadata total size field to the list sysctl operation and
      fill it with total size of all payloads' metadata.
      
      Extend the libxc to handle the metadata back-to-back data transfers
      as well as metadata length array data transfers.
      
      The xen-livepatch userland tool is extended to always display the
      metadata for each received module. The metadata is received with the
      following format: key=value\0key=value\0...key=value\0. The format is
      modified to the following one: key=value;key=value;...key=value.
      The new format allows to easily parse the metadata for a given module
      by a machine.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Reviewed-by: Norbert Manthey <nmanthey@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit b145b4a39c1324186b1b43313a9fefc19b7aa43f
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:59 2019 +0000
  
      livepatch: Handle arbitrary size names with the list operation
      
      The payloads' name strings can be of arbitrary size (typically small
      with an upper bound of XEN_LIVEPATCH_NAME_SIZE).
      Current implementation of the list operation interface allows to copy
      names in the XEN_LIVEPATCH_NAME_SIZE chunks regardless of its actual
      size and enforces space allocation requirements on userland tools.
      
      To unify and simplify the interface, handle the name strings of
      arbitrary size by copying them in adhering chunks to the userland.
      In order to let the userland allocate enough space for the incoming
      data add an auxiliary interface xc_livepatch_list_get_sizes() that
      provides the current number of payload entries and the total size of
      all name strings. This is achieved by extending the sysctl list
      interface with an extra fields: name_total_size.
      
      The xc_livepatch_list_get_sizes() issues the livepatch sysctl list
      operation with the nr field set to 0. In this mode the operation
      returns the number of payload entries and calculates the total sizes
      for all payloads' names.
      When the sysctl operation is issued with a non-zero nr field (for
      instance with a value obtained earlier with the prior call to the
      xc_livepatch_list_get_sizes()) the new field name_total_size provides
      the total size of actually copied data.
      
      Extend the libxc to handle the name back-to-back data transfers.
      
      The xen-livepatch tool is modified to start the list operation with a
      call to the xc_livepatch_list_get_sizes() to obtain the actual number
      of payloads as well as the necessary space for names.
      The tool now always requests the actual number of entries and leaves
      the preemption handling to the libxc routine. The libxc still returns
      'done' and 'left' parameters with the same semantic allowing the tool
      to detect anomalies and react to them. At the moment it is expected
      that the tool receives the exact number of entries as requested.
      The xen-livepatch tool has been also modified to handle the name
      back-to-back transfers correctly.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Bjoern Doebel <doebel@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 4848297ad42135ee8e7e1e6e14b3855ceaf3eb08
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:58 2019 +0000
  
      livepatch: Add support for modules .modinfo section metadata
      
      Having detailed livepatch metadata helps to properly identify module's
      origin and version. It also allows to keep track of the history of
      livepatch loads in the system (at least within dmesg buffer size
      limits).
      
      The livepatch metadata are embedded in a form of .modinfo section.
      Each such section contains data of the following format:
      key=value\0key=value\0...key=value\0
      
      The .modinfo section may be generated and appended to the resulting
      livepatch ELF file optionally as an extra step of a higher level
      livepatch build system.
      
      The metadata section pointer and the section length is stored in the
      livepatch payload structure and is used to display the content upon
      livepatch apply operation.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Bjoern Doebel <doebel@xxxxxxxxx>
      Reviewed-by: Leonard Foerster <foersleo@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Reviewed-by: Norbert Manthey <nmanthey@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 8e24c887887a95cb2dda0017569ed19b65670152
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:57 2019 +0000
  
      livepatch: Add support for inline asm livepatching expectations
      
      This is the initial implementation of the expectations enhancement
      to improve inline asm livepatching.
      
      Expectations are designed as optional feature, since the main use of
      them is planned for inline asm livepatching. The flag enabled allows
      to control the expectation state.
      Each expectation has data and len fields that describe the data
      that is expected to be found at a given patching (old_addr) location.
      The len must not exceed the data array size. The data array size
      follows the size of the opaque array, since the opaque array holds
      the original data and therefore must match what is specified in the
      expectation (if enabled).
      
      The payload structure is modified as each expectation structure is
      part of the livepatch_func structure and hence extends the payload.
      
      Each expectation is checked prior to the apply action (i.e. as late
      as possible to check against the most current state of the code).
      
      For the replace action a new payload's expectations are checked AFTER
      all applied payloads are successfully reverted, but BEFORE new payload
      is applied. That breaks the replace action's atomicity and in case of
      an expectation check failure would leave a system with all payloads
      reverted. That is obviously insecure. Use it with caution and act
      upon replace errors!
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Reviewed-by: Norbert Manthey <nmanthey@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 6047104c3ccc50205464a9b6a90daa85d21a4798
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:56 2019 +0000
  
      livepatch: Add per-function applied/reverted state tracking marker
      
      Livepatch only tracks an entire payload applied/reverted state. But,
      with an option to supply the apply_payload() and/or revert_payload()
      functions as optional hooks, it becomes possible to intermix the
      execution of the original apply_payload()/revert_payload() functions
      with their dynamically supplied counterparts.
      It is important then to track the current state of every function
      being patched and prevent situations of unintentional double-apply
      or unapplied revert.
      
      To support that, it is necessary to extend public interface of the
      livepatch. The struct livepatch_func gets additional field holding
      the applied/reverted state marker.
      
      To reflect the livepatch payload ABI change, bump the version flag
      LIVEPATCH_PAYLOAD_VERSION up to 2.
      
      [And also update the top of the design document]
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Bjoern Doebel <doebel@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Acked-by: Julien Grall <julien.grall@xxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 76b3d4098a92a323a43bc250c67c721c1eed0acb
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:55 2019 +0000
  
      livepatch: Do not enforce ELF_LIVEPATCH_FUNC section presence
      
      With default implementation the ELF_LIVEPATCH_FUNC section containing
      all functions to be replaced or added must be part of the livepatch
      payload, otherwise the payload is rejected (with -EINVAL).
      
      However, with the extended hooks implementation, a livepatch may be
      constructed of only hooks to perform certain actions without any code
      to be added or replaced.
      Therefore, do not always expect the functions section and allow it to
      be missing, provided there is at least one section containing hooks
      present. The functions section, when present in a payload, must be a
      single, non-empty section.
      
      Check also all extended hooks sections if they are a single, non-empty
      sections each.
      
      At least one of the functions or hooks section must be present in a
      valid payload.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Bjoern Doebel <doebel@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit ef87efee9d38b61624f25c1a056d386a70ba99aa
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:54 2019 +0000
  
      livepatch: Add support for apply|revert action replacement hooks
      
      By default, in the quiescing zone, a livepatch payload is applied with
      apply_payload() and reverted with revert_payload() functions. Both of
      the functions receive the payload struct pointer as a parameter. The
      functions are also a place where standard 'load' and 'unload' module
      hooks are executed.
      
      To increase livepatching system's agility and provide more flexible
      long-term livepatch solution, allow to overwrite the default apply
      and revert action functions with hook-like supplied alternatives.
      The alternative functions are optional and the default functions are
      used by default.
      
      Since the alternative functions have direct access to the livepatch
      payload structure, they can better control context of the 'load' and
      'unload' hooks execution as well as exact instructions replacement
      workflows. They can be also easily extended to support extra features
      in the future.
      
      To simplify the alternative function generation move code responsible
      for payload and livepatch region registration outside of the function.
      That way it is guaranteed that the registration step occurs even for
      newly supplied functions.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Petre Eftime <epetre@xxxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxxx>
      Reviewed-by: Norbert Manthey <nmanthey@xxxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Bjoern Doebel <doebel@xxxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 8313c864fa95074c2176f19af711b7e13bf20504
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:53 2019 +0000
  
      livepatch: Implement pre-|post- apply|revert hooks
      
      This is an implementation of 4 new livepatch module vetoing hooks,
      that can be optionally supplied along with modules.
      Hooks that currently exists in the livepatch mechanism aren't agile
      enough and have various limitations:
      * run only from within a quiescing zone
      * cannot conditionally prevent applying or reverting
      * do not have access to the module context
      To address these limitations the following has been implemented:
      1) pre-apply hook
        runs before the apply action is scheduled for execution. Its main
        purpose is to prevent from applying a livepatch when certain
        expected conditions aren't met or when mutating actions implemented
        in the hook fail or cannot be executed.
      
      2) post-apply hook
        runs after the apply action has been executed and quiescing zone
        exited. Its main purpose is to provide an ability to follow-up on
        actions performed by the pre- hook, when module application was
        successful or undo certain preparation steps of the pre- hook in
        case of a failure. The success/failure error code is provided to
        the post- hooks via the rc field of the payload structure.
      
      3) pre-revert hook
        runs before the revert action is scheduled for execution. Its main
        purpose is to prevent from reverting a livepatch when certain
        expected conditions aren't met or when mutating actions implemented
        in the hook fail or cannot be executed.
      
      4) post-revert hook
        runs after the revert action has been executed and quiescing zone
        exited. Its main purpose is to perform cleanup of all previously
        executed mutating actions in order to restore the original system
        state from before the current module application.
        The success/failure error code is provided to the post- hooks via
        the rc field of the payload structure.
      
      The replace action performs atomically the following actions:
      - revert all applied modules
      - apply a single replacement module.
      With the vetoing hooks in place various inter-hook dependencies may
      arise. Also, during the revert part of the operation certain vetoing
      hooks may detect failing conditions that previously were satisfied.
      That could in turn lead to situation when the revert part must be
      rolled back with all the pre- and post- hooks re-applied, which again
      can't be guaranteed to always succeed.
      The simplest response to this complication is to disallow the replace
      action completely on modules with vetoing hooks.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Petre Eftime <epetre@xxxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Reviewed-by: Norbert Manthey <nmanthey@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 3bafe4a06051de1d7abfffe77c8b9cb58594f39f
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:52 2019 +0000
  
      livepatch: Export payload structure via livepatch_payload.h
      
      The payload structure will be used by the new hooks implementation and
      therefore its definition has to be exported via the livepatch_payload
      header.
      The new hooks will make use of the payload structure fields and the
      hooks' pointers will also be defined in the payload structure, so
      the structure along with all field definitions needs to be available
      to the code being patched in.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Eslam Elnikety <elnikety@xxxxxxxxx>
      Reviewed-by: Leonard Foerster <foersleo@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
  
  commit b274989b610d37f0775e93c08343d30ec267a80f
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:51 2019 +0000
  
      livepatch: Allow to override inter-modules buildid dependency
      
      By default Livepatch enforces the following buildid-based dependency
      chain between livepatch modules:
        1) first module depends on given hypervisor buildid
        2) every consecutive module depends on previous module's buildid
      This way proper livepatch stack order is maintained and enforced.
      While it is important for production livepatches it limits agility and
      blocks usage of testing or debug livepatches. These kinds of livepatch
      modules are typically expected to be loaded at any time irrespective
      of current state of the modules stack.
      
      To enable testing and debug livepatches allow user dynamically ignore
      the inter-modules dependency. In this case only hypervisor buildid
      match is verified and enforced.
      
      To allow userland pass additional paremeters for livepatch actions
      add support for action flags.
      Each of the apply, revert, unload and revert action gets additional
      32-bit parameter 'flags' where extra flags can be applied in a mask
      form.
      Initially only one flag '--nodeps' is added for the apply action.
      This flag modifies the default buildid dependency check as described
      above.
      The global sysctl interface input flag parameter is defined with a
      single corresponding flag macro:
        LIVEPATCH_ACTION_APPLY_NODEPS (1 << 0)
      
      The userland xen-livepatch tool is modified to support the '--nodeps'
      flag for apply and load commands. A general mechanism for specifying
      more flags in the future for apply and other action is however added.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Eslam Elnikety <elnikety@xxxxxxxxx>
      Reviewed-by: Petre Eftime <epetre@xxxxxxxxxx>
      Reviewed-by: Leonard Foerster <foersleo@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Reviewed-by: Norbert Manthey <nmanthey@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>
  
  commit 879615f5db1d0a86afd99a67d284a8df6fd85be4
  Author: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
  Date:   Tue Nov 26 10:07:50 2019 +0000
  
      livepatch: Always check hypervisor build ID upon livepatch upload
      
      This change is part of a independant stacked livepatch modules
      feature. This feature allows to bypass dependencies between modules
      upon loading, but still verifies Xen build ID matching.
      
      In order to prevent (up)loading any livepatches built for different
      hypervisor version as indicated by the Xen Build ID, add checking for
      the payload's vs Xen's build id match.
      
      To achieve that embed into every livepatch another section with a
      dedicated hypervisor build id in it. After the payload is loaded and
      the .livepatch.xen_depends section becomes available, perform the
      check and reject the payload if there is no match.
      
      Signed-off-by: Pawel Wieczorkiewicz <wipawel@xxxxxxxxx>
      Reviewed-by: Andra-Irina Paraschiv <andraprs@xxxxxxxxxx>
      Reviewed-by: Bjoern Doebel <doebel@xxxxxxxxx>
      Reviewed-by: Eslam Elnikety <elnikety@xxxxxxxxx>
      Reviewed-by: Martin Pohlack <mpohlack@xxxxxxxxx>
      Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
      Reviewed-by: Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>

Revision graph left in 
/home/logs/results/bisect/xen-unstable-smoke/build-arm64-xsm.xen-build--dist-test.{dot,ps,png,html,svg}.
----------------------------------------
145004: tolerable all pass

flight 145004 xen-unstable-smoke real-bisect [real]
http://logs.test-lab.xenproject.org/osstest/logs/145004/

Failures :-/ but no regressions.

Tests which did not succeed,
including tests which could not be run:
 build-arm64-xsm               7 xen-build/dist-test     fail baseline untested


jobs:
 build-arm64-xsm                                              pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


_______________________________________________
osstest-output mailing list
osstest-output@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/osstest-output

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.