[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[PATCH DNA 5/6] tools/xenstored: restore support for mapping ring as foreign memory


  • To: <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • From: Roger Pau Monne <roger.pau@xxxxxxxxxx>
  • Date: Fri, 17 Sep 2021 17:46:24 +0200
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ODViCmJYnKpIZT6ta/bzfRMYz5jZaA8JqNXfGiUOkig=; b=jPzNrGxDY1wy9FTQdtSiC1MqrQMqWyrrvdYuY3/xsQOvXuQU/b07v8I3RTq92Is6m4am0eAPl3br7JVrpCisBvJ5U7ycqmChW6CdQAFDC0l8FKY528N5il3T1wyaJ/3vR2UF2h7cpIDvjrEBOKyaXXI4lcp5OU9ufEIsp6TT1uUlSGNGrGQz22Bvn2S11DVpeoaNgoig6zFx9N3Ebmu3wwE9v//jadcGrjmZDowzuwI5F6qdzKngOCKFhrEdndV1I9ZP7oIw6nG5QzkBD/R6aIf98BF2Mg9Dexjf8slLvnXiXkIYAHHEPBRS8Yf1D68ShRFqOw0zVkNlgZXKRFnCMg==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gJP/gv9BZcBvmv3vrYTwxXY4zUImWbh5MbXJz1n0j9jdXM2hcCbVHCdl+0laV8EieWioyw0RCSKuA+Q0H5e32NLVqN0TlRnt8VYj7p+zaoCaK9OUexC2kGa62aMXtQ2miPnBk2fpRdEUCI6qThc/kfjfyjuwadzQMwYpm5yMmSqOZe4vrRY3ghvL5fcm/L4hr2QHnWNo3uCxigj18eP9nTFuDMwMoYLqk938avRkNPO5ki17EbGmSpyH1XzcS9o2g8AJzVqqvRO52X5AD0agSy15IC3gFCLwYODAabOomk+odtXDj3FpQGaygxHTwKgr6EIHiHsuQYWkrmbdzLgM7w==
  • Authentication-results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
  • Cc: Roger Pau Monne <roger.pau@xxxxxxxxxx>, Ian Jackson <iwj@xxxxxxxxxxxxxx>, Wei Liu <wl@xxxxxxx>, Juergen Gross <jgross@xxxxxxxx>, Julien Grall <julien@xxxxxxx>
  • Delivery-date: Fri, 17 Sep 2021 15:47:15 +0000
  • Ironport-data: A9a23:3Pnbz65wYTCpQmsHDM4yFAxRtOzAchMFZxGqfqrLsTDasY5as4F+v jFMX22PP//fNjOkc4tyaovi8khT7JPTzN43HldvrHs9Hi5G8cbLO4+Ufxz6V8+wwmwvb67FA +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo29Ew27BVPivW0 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z6 /Ngn5CcRzcQGKzpibRMaUJZAi1YIvgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr 7pCcmlLN03dwbLtqF64YrAEasALNs7kMZlZonh95TrYEewnUdbIRKCiCdpwgWxs35gfQKm2i 8wxTwNUXRTiPj12ABQ4EckVxcqToUKkSmgNwL6SjfVuuDWCpOBr65DvP8DSYceiXthOkwCTo Weu12bkBhAXMvSPxDzD9Wij7sfQmQvrVYRUE6e3ntZ6jVvWymENBRk+UVqgveL/mkO4Q8hYK UEf5mwpt6da3E20TPHtUhugunmGsxUAHd1KHIUHBBqlk/SOpVzDXy5dE2AHOIdOWNIKqSIC9 Xmktd/FCwxWlrywWCLMzOyVvD+UJn1ARYMdXhPoXTfp8vG6/tpq3kmeH445eEKmpoaqQmCrm lhmuAB73u9K3JBRj81X6Hia22rEm3TfcuIiCuw7tEqe5wVlbcaOY4Wy4DA3Bt4Rcd7EEjFtU JUC8vVyDdzi77nWz0Rho81XRdlFAspp1xWG2jZS82EJrWjFxpJaVdk4DMtCyKJV3iEsImSBX aMukVkJuM870IWCNPcqC25ONyja5fe5Tom0PhwlRvFPfoJwZGe6ENJGPBXLt10BZHMEyPllU b/CKJ7EJS9DVcxPkWrnL89AgORD7n1vmgvuqWXTkk3PPUy2PyXOF9/o8TKmM4gE0U9ziF+Eq 4oAb5TalUo3vS+XSnC/zLP/5GsidBATLZv3t9ZWZqiEJA9nE3smEPjf3fUqfIkNokifvr6gE qiVVhAKxVzhq2fALAnWOHlvZKm2BcR0rG4hPDxqNlGtgiBxbYGq5aYZVp02Ybh4q7Azka8qF 6EIK5eaH/BCajXb4DBBP5Pzm5NvKUawjgWUMiv7PDVmJ8x8RxbE88PPdxf08HVcFTK+sMYz+ uXy1g7STZcZaR5lCcLaNKCmw1+r5CBPk+NuRUrYZNJUfRy0ooRtLiXwiN4xIt0NdkqflmfLi V7ODE5B9+fXooIz/N3Yvoy+rt+kQ7lkA05XP2jH9rLqZyPUyXWunN1bW+GScDGDCG6toPe+Z f9Yxu3XOeEcmAoYqJJ1FrtmwP5s59broLMGnA1oEG+SMgauA7JkZHKHwdNOputGwboA4Vm6X UeG+997P7SVOZy6TA5NdVR9NunTh+sJnjTy7OguJBSo7SB6y7OLTEFOMkTekydaNrZ0bNsoz OpJVBT6MOBjZs7G6uq7sx0=
  • Ironport-hdrordr: A9a23:GHULGqq7icN6S65es1HxuPEaV5u6L9V00zEX/kB9WHVpm5Oj+P xGzc526farslsssREb+OxpOMG7MBfhHPlOkPUs1NaZLXTbUQ6TQr2KgrGSpQEIdxeOlNK1kJ 0QCJSWa+eAfGSS7/yKmTVQeuxIqLLskNHK9JfjJjVWPHtXgslbnnlE422gYypLrWd9dP8E/M 323Ls5m9PsQwVbUu2LQl0+G8TTrdzCk5zrJTYAGh4c8QGLyRel8qTzHRS01goXF2on+8ZszU H11yjCoomzufCyzRHRk0fV8pRtgdPkjv9OHtaFhMQ5IijlziyoeINicbufuy1dmpDi1H8a1P 335zswNcV67H3cOkmzvBvWwgHllA0j7nfzoGXoyUfLkIjcfnYXGsBBjYVWfl/y8Ew7puxx16 pNwiawq4dXJQmoplW82/H4EzVR0makq3srluAey1ZFV5EFVbNXpYsDuGtIDZY7Gj7g4oxPKp guMCjl3ocVTbqmVQGdgoE2q+bcGkjbXy32DHTqg/blkAS/xxtCvgwlLM92pAZIyHtycegD2w xoWp4Y4I2mdfVmH56VMt1xN/dfOla9Mi4kD1jiVGgPbJt3Q04li6SHq4ndt9vaMqDh8vMJ6e P8uRVjxDcPR34=
  • Ironport-sdr: EmJ3mxN3nx6zzStBk3fTRuwk7pnO7KwmLP0B2u8QxQXnQ51V592DGCTH2sRhkYhlmxJhZR1KBn omw1YMRBKrFaM3R8PUeLPqCM8uJNZOV6a02ZOffodFAisS1GHCq8ns6Jo6HGvKMUX3HHgTPr9d PZHuDzcsl7MvIH3qOIgiIQhdT4ie2FTKOtMHEO1XHn7cyFQhpARIL90EI6mEOCG05XTLvqNZ7z ve8bumARwld8WapcgTsCzhmeLinps8Uo9F4BoXdzUpHEsKQMHwm91Mpot3vaWJzaKnlYENLTce W8QWfSe0NT14qHLrw1j34oLc
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

Restore the previous way of mapping the xenstore ring using foreign
memory. Use xenforeignmemory instead of libxc in order to avoid adding
another dependency on a unstable interface.

This in turn requires storing the gfn into xs_state_connection for
resume purposes, which breaks the current format.

Note this is a preparatory patch in order to support the usage of
xenstore on domains without grant table. This not only allows xenstore
usage, but also makes things like "@introduceDomain" events work when
using such guests, otherwise the mapping done in introduce_domain
fails and the "@introduceDomain" event won't be signaled.

Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
---
 tools/xenstore/Makefile                 |  4 +-
 tools/xenstore/include/xenstore_state.h |  1 +
 tools/xenstore/xenstored_domain.c       | 69 +++++++++++++++++++------
 3 files changed, 56 insertions(+), 18 deletions(-)

diff --git a/tools/xenstore/Makefile b/tools/xenstore/Makefile
index 292b478fa1..9a9f7be5cb 100644
--- a/tools/xenstore/Makefile
+++ b/tools/xenstore/Makefile
@@ -67,10 +67,10 @@ $(XENSTORED_OBJS): CFLAGS += $(SYSTEMD_CFLAGS)
 xenstored: LDFLAGS += $(SYSTEMD_LIBS)
 endif
 
-$(XENSTORED_OBJS): CFLAGS += $(CFLAGS_libxengnttab)
+$(XENSTORED_OBJS): CFLAGS += $(CFLAGS_libxengnttab) 
$(CFLAGS_libxenforeignmemory)
 
 xenstored: $(XENSTORED_OBJS)
-       $(CC) $^ $(LDFLAGS) $(LDLIBS_libxenevtchn) $(LDLIBS_libxengnttab) 
$(LDLIBS_libxenctrl) $(LDLIBS_xenstored) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS)
+       $(CC) $^ $(LDFLAGS) $(LDLIBS_libxenevtchn) $(LDLIBS_libxengnttab) 
$(LDLIBS_libxenforeignmemory) $(LDLIBS_libxenctrl) $(LDLIBS_xenstored) 
$(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS)
 
 xenstored.a: $(XENSTORED_OBJS)
        $(AR) cr $@ $^
diff --git a/tools/xenstore/include/xenstore_state.h 
b/tools/xenstore/include/xenstore_state.h
index ae0d053c8f..8dcc8d9d8b 100644
--- a/tools/xenstore/include/xenstore_state.h
+++ b/tools/xenstore/include/xenstore_state.h
@@ -80,6 +80,7 @@ struct xs_state_connection {
             uint16_t domid;  /* Domain-Id. */
             uint16_t tdomid; /* Id of target domain or DOMID_INVALID. */
             uint32_t evtchn; /* Event channel port. */
+            uint64_t gfn;    /* Store GFN. */
         } ring;
         int32_t socket_fd;   /* File descriptor for socket connections. */
     } spec;
diff --git a/tools/xenstore/xenstored_domain.c 
b/tools/xenstore/xenstored_domain.c
index 8930303773..f3563e47aa 100644
--- a/tools/xenstore/xenstored_domain.c
+++ b/tools/xenstore/xenstored_domain.c
@@ -33,10 +33,12 @@
 
 #include <xenevtchn.h>
 #include <xenctrl.h>
+#include <xenforeignmemory.h>
 #include <xen/grant_table.h>
 
 static xc_interface *xc_handle;
 xengnttab_handle *xgt_handle;
+static xenforeignmemory_handle *xfm_handle;
 static evtchn_port_t virq_port;
 
 xenevtchn_handle *xce_handle = NULL;
@@ -66,12 +68,18 @@ struct domain
        /* Generation count at domain introduction time. */
        uint64_t generation;
 
+       /* Store GFN (if using a ring connection). */
+       xen_pfn_t gfn;
+
        /* Have we noticed that this domain is shutdown? */
        bool shutdown;
 
        /* Has domain been officially introduced? */
        bool introduced;
 
+       /* Is the ring memory foreign mapped? */
+       bool foreign_mapped;
+
        /* number of entry from this domain in the store */
        int nbentry;
 
@@ -196,16 +204,29 @@ static const struct interface_funcs domain_funcs = {
        .can_read = domain_can_read,
 };
 
-static void *map_interface(domid_t domid)
+static void *map_interface(domid_t domid, xen_pfn_t gfn, bool *foreign_mapped)
 {
-       return xengnttab_map_grant_ref(xgt_handle, domid,
-                                      GNTTAB_RESERVED_XENSTORE,
-                                      PROT_READ|PROT_WRITE);
+       void *map = xengnttab_map_grant_ref(xgt_handle, domid,
+                                           GNTTAB_RESERVED_XENSTORE,
+                                           PROT_READ|PROT_WRITE);
+
+       if (!map)
+       {
+               map = xenforeignmemory_map(xfm_handle, domid,
+                                          PROT_READ|PROT_WRITE, 1,
+                                          &gfn, NULL);
+               *foreign_mapped = !!map;
+       }
+
+       return map;
 }
 
-static void unmap_interface(void *interface)
+static void unmap_interface(void *interface, bool foreign_mapped)
 {
-       xengnttab_unmap(xgt_handle, interface, 1);
+       if (foreign_mapped)
+               xenforeignmemory_unmap(xfm_handle, interface, 1);
+       else
+               xengnttab_unmap(xgt_handle, interface, 1);
 }
 
 static int destroy_domain(void *_domain)
@@ -228,7 +249,8 @@ static int destroy_domain(void *_domain)
                if (domain->domid == 0)
                        unmap_xenbus(domain->interface);
                else
-                       unmap_interface(domain->interface);
+                       unmap_interface(domain->interface,
+                                       domain->foreign_mapped);
        }
 
        fire_watches(NULL, domain, "@releaseDomain", NULL, false, NULL);
@@ -363,12 +385,15 @@ static struct domain *find_or_alloc_domain(const void 
*ctx, unsigned int domid)
        return domain ? : alloc_domain(ctx, domid);
 }
 
-static int new_domain(struct domain *domain, int port, bool restore)
+static int new_domain(struct domain *domain, int port, xen_pfn_t gfn,
+                     bool foreign_mapped, bool restore)
 {
        int rc;
 
        domain->port = 0;
        domain->shutdown = false;
+       domain->gfn = gfn;
+       domain->foreign_mapped = foreign_mapped;
        domain->path = talloc_domain_path(domain, domain->domid);
        if (!domain->path) {
                errno = ENOMEM;
@@ -436,7 +461,8 @@ static void domain_conn_reset(struct domain *domain)
 
 static struct domain *introduce_domain(const void *ctx,
                                       unsigned int domid,
-                                      evtchn_port_t port, bool restore)
+                                      evtchn_port_t port, xen_pfn_t gfn,
+                                      bool restore)
 {
        struct domain *domain;
        int rc;
@@ -448,17 +474,21 @@ static struct domain *introduce_domain(const void *ctx,
                return NULL;
 
        if (!domain->introduced) {
+               bool foreign_mapped = false;
+
                interface = is_master_domain ? xenbus_map()
-                                            : map_interface(domid);
+                                            : map_interface(domid, gfn,
+                                                            &foreign_mapped);
                if (!interface && !restore)
                        return NULL;
-               if (new_domain(domain, port, restore)) {
+               if (new_domain(domain, port, gfn, foreign_mapped, restore)) {
                        rc = errno;
                        if (interface) {
                                if (is_master_domain)
                                        unmap_xenbus(interface);
                                else
-                                       unmap_interface(interface);
+                                       unmap_interface(interface,
+                                                       foreign_mapped);
                        }
                        errno = rc;
                        return NULL;
@@ -489,19 +519,20 @@ int do_introduce(struct connection *conn, struct 
buffered_data *in)
        char *vec[3];
        unsigned int domid;
        evtchn_port_t port;
+       xen_pfn_t gfn;
 
        if (get_strings(in, vec, ARRAY_SIZE(vec)) < ARRAY_SIZE(vec))
                return EINVAL;
 
        domid = atoi(vec[0]);
-       /* Ignore the gfn, we don't need it. */
+       gfn = atol(vec[1]);
        port = atoi(vec[2]);
 
        /* Sanity check args. */
        if (port <= 0)
                return EINVAL;
 
-       domain = introduce_domain(in, domid, port, false);
+       domain = introduce_domain(in, domid, port, gfn, false);
        if (!domain)
                return errno;
 
@@ -718,7 +749,7 @@ void dom0_init(void)
        if (port == -1)
                barf_perror("Failed to initialize dom0 port");
 
-       dom0 = introduce_domain(NULL, xenbus_master_domid(), port, false);
+       dom0 = introduce_domain(NULL, xenbus_master_domid(), port, 0, false);
        if (!dom0)
                barf_perror("Failed to initialize dom0");
 
@@ -758,6 +789,10 @@ void domain_init(int evtfd)
         */
        xengnttab_set_max_grants(xgt_handle, DOMID_FIRST_RESERVED);
 
+       xfm_handle = xenforeignmemory_open(NULL, 0);
+       if (!xfm_handle)
+               barf_perror("Failed to create handle for foreign memory");
+
        if (evtfd < 0)
                xce_handle = xenevtchn_open(NULL, XENEVTCHN_NO_CLOEXEC);
        else
@@ -1189,6 +1224,7 @@ const char *dump_state_connections(FILE *fp)
                        sc.spec.ring.tdomid = c->target ? c->target->id
                                                : DOMID_INVALID;
                        sc.spec.ring.evtchn = c->domain->port;
+                       sc.spec.ring.gfn = c->domain->gfn;
                } else {
                        sc.conn_type = XS_STATE_CONN_TYPE_SOCKET;
                        sc.spec.socket_fd = c->fd;
@@ -1290,7 +1326,8 @@ void read_state_connection(const void *ctx, const void 
*state)
 #endif
        } else {
                domain = introduce_domain(ctx, sc->spec.ring.domid,
-                                         sc->spec.ring.evtchn, true);
+                                         sc->spec.ring.evtchn,
+                                         sc->spec.ring.gfn, true);
                if (!domain)
                        barf("domain allocation error");
 
-- 
2.33.0




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.