[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[XEN][PATCH v5 8/8] xen/memory: copy add_to_physmap batch entries in bulk



From: Haseeb Ashraf <haseeb.ashraf@xxxxxxxxxxx>

xenmem_add_to_physmap_batch() copies two inputs and one result for
every entry. On Arm every access to guest memory translates the
guest address with an AT instruction, which traps to the outer
hypervisor when Xen runs under nested virtualization.

Stage up to 128 input entries and copy the results in bulk. Entries
are still handled in order, an INVALID_GFN stops the loop after the
entries before it, and the hypercall can still be preempted between
two entries and continues with the first one that was not handled.
At 128 entries the three buffers use 2.5 KiB of stack (8 + 8 + 4 = 20
bytes per entry), about 8% of the 32 KiB Arm stack. This bounds stack
use while amortizing the two input copies and the result copy over each
chunk; it is not a measured throughput threshold.

Input copies precede the processing of each chunk and result copies
follow it. EFAULT is not transactional: mappings may already have
changed, and the order of partial copies and overlapping buffers is
not the same as with the element at a time implementation.

Signed-off-by: Haseeb Ashraf <haseeb.ashraf@xxxxxxxxxxx>
---
 xen/common/memory.c | 63 ++++++++++++++++++++++++++++++++++-----------
 1 file changed, 48 insertions(+), 15 deletions(-)

diff --git a/xen/common/memory.c b/xen/common/memory.c
index b5256bf6ec..c493766abd 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -1077,6 +1077,14 @@ int xenmem_add_to_physmap(struct domain *d, struct 
xen_add_to_physmap *xatp,
     return rc;
 }
 
+/*
+ * Number of entries copied from and to the guest at a time. Every access to
+ * guest memory needs a guest virtual to machine address translation, which
+ * is very slow when Xen itself runs on top of another hypervisor, so it is
+ * not done once per entry.
+ */
+#define XENMEM_ATPB_CHUNK 128U
+
 static int xenmem_add_to_physmap_batch(struct domain *d,
                                        struct xen_add_to_physmap_batch *xatpb,
                                        unsigned int extent)
@@ -1120,27 +1128,52 @@ static int xenmem_add_to_physmap_batch(struct domain *d,
 
     while ( xatpb->size > extent )
     {
-        xen_ulong_t idx;
-        xen_pfn_t gpfn;
-        int rc;
-
-        if ( unlikely(__copy_from_guest_offset(&idx, xatpb->idxs,
-                                               extent, 1)) ||
-             unlikely(__copy_from_guest_offset(&gpfn, xatpb->gpfns,
-                                               extent, 1)) )
+        xen_ulong_t idxs[XENMEM_ATPB_CHUNK];
+        xen_pfn_t gpfns[XENMEM_ATPB_CHUNK];
+        int errs[XENMEM_ATPB_CHUNK];
+        unsigned int n = min_t(unsigned int, xatpb->size - extent,
+                               XENMEM_ATPB_CHUNK);
+        unsigned int i;
+        bool preempt = false;
+        int rc = 0;
+
+        if ( unlikely(__copy_from_guest_offset(idxs, xatpb->idxs,
+                                               extent, n)) ||
+             unlikely(__copy_from_guest_offset(gpfns, xatpb->gpfns,
+                                               extent, n)) )
             return -EFAULT;
 
-        if ( gfn_eq(_gfn(gpfn), INVALID_GFN) )
-            return -EINVAL;
+        for ( i = 0; i < n; i++ )
+        {
+            if ( gfn_eq(_gfn(gpfns[i]), INVALID_GFN) )
+            {
+                rc = -EINVAL;
+                break;
+            }
 
-        rc = xenmem_add_to_physmap_one(d, xatpb->space, extra,
-                                       idx, _gfn(gpfn));
+            errs[i] = xenmem_add_to_physmap_one(d, xatpb->space, extra,
+                                                idxs[i], _gfn(gpfns[i]));
 
-        if ( unlikely(__copy_to_guest_offset(xatpb->errs, extent, &rc, 1)) )
+            /* Check for continuation if it's not the last iteration. */
+            if ( xatpb->size > extent + i + 1 && hypercall_preempt_check() )
+            {
+                preempt = true;
+                i++;
+                break;
+            }
+        }
+
+        /* i entries have a result. */
+        if ( (i != 0U) &&
+             unlikely(__copy_to_guest_offset(xatpb->errs, extent, errs, i)) )
             return -EFAULT;
 
-        /* Check for continuation if it's not the last iteration. */
-        if ( xatpb->size > ++extent && hypercall_preempt_check() )
+        extent += i;
+
+        if ( rc != 0 )
+            return rc;
+
+        if ( preempt )
             return extent;
     }
 



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.