[xen master] x86/mm: re-implement get_page_light() using an atomic increment

[patchbot@xxxxxxx]

commit c40bc0576dcc5acd4d7e22ef628eb4642f568533
Author:     Roger Pau Monné <roger.pau@xxxxxxxxxx>
AuthorDate: Fri Mar 1 13:42:27 2024 +0100
Commit:     Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
CommitDate: Fri Mar 1 20:14:19 2024 +0000

    x86/mm: re-implement get_page_light() using an atomic increment
    
    The current usage of a cmpxchg loop to increase the value of page count is 
not
    optimal on amd64, as there's already an instruction to do an atomic add to a
    64bit integer.
    
    Switch the code in get_page_light() to use an atomic increment, as that 
avoids
    a loop construct.  This slightly changes the order of the checks, as current
    code will crash before modifying the page count_info if the conditions are 
not
    correct, while with the proposed change the crash will happen immediately
    after having carried the counter increase.  Since we are crashing anyway, I
    don't believe the re-ordering to have any meaningful impact.
    
    Note that the page must already have a non-zero reference count which 
prevents
    the flags from changing, and the previous usage of the cmpxchg loop didn't
    guarantee that the rest of the fields in count_info didn't change while
    updating the reference count.
    
    Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
    Reviewed-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
---
 xen/arch/x86/mm.c | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 4d6d7bfe4f..2aff6d4b53 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -2580,16 +2580,10 @@ bool get_page(struct page_info *page, const struct 
domain *domain)
  */
 static void get_page_light(struct page_info *page)
 {
-    unsigned long x, nx, y = page->count_info;
+    unsigned long old_pgc = arch_fetch_and_add(&page->count_info, 1);
 
-    do {
-        x  = y;
-        nx = x + 1;
-        BUG_ON(!(x & PGC_count_mask)); /* Not allocated? */
-        BUG_ON(!(nx & PGC_count_mask)); /* Overflow? */
-        y = cmpxchg(&page->count_info, x, nx);
-    }
-    while ( unlikely(y != x) );
+    BUG_ON(!(old_pgc & PGC_count_mask)); /* Not allocated? */
+    BUG_ON(!((old_pgc + 1) & PGC_count_mask)); /* Overflow? */
 }
 
 static int validate_page(struct page_info *page, unsigned long type,
--
generated by git-patchbot for /home/xen/git/xen.git#master