[xen master] xen/livepatch: Make check_for_livepatch_work() faster in the common case

[patchbot@xxxxxxx]

commit c30021be302aa2a644ef5544097fb8bb763ec140
Author:     Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
AuthorDate: Fri Dec 22 21:06:16 2023 +0000
Commit:     Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
CommitDate: Thu Jan 18 21:18:21 2024 +0000

    xen/livepatch: Make check_for_livepatch_work() faster in the common case
    
    When livepatching is enabled, this function is used all the time.  Really do
    check the fastpath first, and annotate it likely() as this is the right 
answer
    100% of the time (to many significant figures).  This cuts out 3 pointer
    dereferences in the "nothing to do path".
    
    However, GCC still needs some help to persuade it not to set the full stack
    frame (6 spilled registers, 3 slots of locals) even on the fastpath.
    
    Create a new check_for_livepatch_work() with the fastpath only, and make the
    "new" do_livepatch_work() noinline.  This causes the fastpath to need no 
stack
    frame, making it faster still.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
    Reviewed-by: Jan Beulich <jbeulich@xxxxxxxx>
---
 xen/common/livepatch.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
index 1209fea256..2c4b843827 100644
--- a/xen/common/livepatch.c
+++ b/xen/common/livepatch.c
@@ -1693,7 +1693,7 @@ static int livepatch_spin(atomic_t *counter, s_time_t 
timeout,
  * The main function which manages the work of quiescing the system and
  * patching code.
  */
-void check_for_livepatch_work(void)
+static void noinline do_livepatch_work(void)
 {
 #define ACTION(x) [LIVEPATCH_ACTION_##x] = #x
     static const char *const names[] = {
@@ -1711,10 +1711,6 @@ void check_for_livepatch_work(void)
          !is_idle_domain(current->sched_unit->domain) )
         return;
 
-    /* Fast path: no work to do. */
-    if ( !per_cpu(work_to_do, cpu ) )
-        return;
-
     smp_rmb();
     /* In case we aborted, other CPUs can skip right away. */
     if ( !livepatch_work.do_work )
@@ -1864,6 +1860,17 @@ void check_for_livepatch_work(void)
     }
 }
 
+void check_for_livepatch_work(void)
+{
+    unsigned int cpu = smp_processor_id();
+
+    /* Fast path: no work to do. */
+    if ( likely(!per_cpu(work_to_do, cpu)) )
+        return;
+
+    do_livepatch_work();
+}
+
 /*
  * Only allow dependent payload is applied on top of the correct
  * build-id.
--
generated by git-patchbot for /home/xen/git/xen.git#master