[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH v11 5/6] x86/ioreq server: Asynchronously reset outstanding p2m_ioreq_server entries.





On 4/6/2017 10:25 PM, George Dunlap wrote:
On 06/04/17 14:19, Yu Zhang wrote:
After an ioreq server has unmapped, the remaining p2m_ioreq_server
entries need to be reset back to p2m_ram_rw. This patch does this
asynchronously with the current p2m_change_entry_type_global()
interface.

New field entry_count is introduced in struct p2m_domain, to record
the number of p2m_ioreq_server p2m page table entries. One nature of
these entries is that they only point to 4K sized page frames, because
all p2m_ioreq_server entries are originated from p2m_ram_rw ones in
p2m_change_type_one(). We do not need to worry about the counting for
2M/1G sized pages.

This patch disallows mapping of an ioreq server, when there's still
p2m_ioreq_server entry left, in case another mapping occurs right after
the current one being unmapped, releases its lock, with p2m table not
synced yet.

This patch also disallows live migration, when there's remaining
p2m_ioreq_server entry in p2m table. The core reason is our current
implementation of p2m_change_entry_type_global() lacks information
to resync p2m_ioreq_server entries correctly if global_logdirty is
on.

Signed-off-by: Yu Zhang <yu.c.zhang@xxxxxxxxxxxxxxx>
Reviewed-by: Paul Durrant <paul.durrant@xxxxxxxxxx>
---
Cc: Paul Durrant <paul.durrant@xxxxxxxxxx>
Cc: Jan Beulich <jbeulich@xxxxxxxx>
Cc: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
Cc: George Dunlap <george.dunlap@xxxxxxxxxxxxx>
Cc: Jun Nakajima <jun.nakajima@xxxxxxxxx>
Cc: Kevin Tian <kevin.tian@xxxxxxxxx>

changes in v6:
   - According to comments from Jan & George: move the count from
     p2m_change_type_one() to {ept,p2m_pt}_set_entry.
   - According to comments from George: comments change.

changes in v5:
   - According to comments from Jan: use unsigned long for entry_count;
   - According to comments from Jan: refuse mapping requirement when
     there's p2m_ioreq_server remained in p2m table.
   - Added "Reviewed-by: Paul Durrant <paul.durrant@xxxxxxxxxx>"

changes in v4:
   - According to comments from Jan: use ASSERT() instead of 'if'
     condition in p2m_change_type_one().
   - According to comments from Jan: commit message changes to mention
     the p2m_ioreq_server are all based on 4K sized pages.

changes in v3:
   - Move the synchronously resetting logic into patch 5.
   - According to comments from Jan: introduce p2m_check_changeable()
     to clarify the p2m type change code.
   - According to comments from George: use locks in the same order
     to avoid deadlock, call p2m_change_entry_type_global() after unmap
     of the ioreq server is finished.

changes in v2:
   - Move the calculation of ioreq server page entry_cout into
     p2m_change_type_one() so that we do not need a seperate lock.
     Note: entry_count is also calculated in resolve_misconfig()/
     do_recalc(), fortunately callers of both routines have p2m
     lock protected already.
   - Simplify logic in hvmop_set_mem_type().
   - Introduce routine p2m_finish_type_change() to walk the p2m
     table and do the p2m reset.
---
  xen/arch/x86/hvm/ioreq.c  |  8 ++++++++
  xen/arch/x86/mm/hap/hap.c |  9 +++++++++
  xen/arch/x86/mm/p2m-ept.c | 20 +++++++++++++++++++-
  xen/arch/x86/mm/p2m-pt.c  | 28 ++++++++++++++++++++++++++--
  xen/arch/x86/mm/p2m.c     |  9 +++++++++
  xen/include/asm-x86/p2m.h |  9 ++++++++-
  6 files changed, 79 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 5bf3b6d..07a6c26 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -955,6 +955,14 @@ int hvm_map_mem_type_to_ioreq_server(struct domain *d, 
ioservid_t id,
spin_unlock_recursive(&d->arch.hvm_domain.ioreq_server.lock); + if ( rc == 0 && flags == 0 )
+    {
+        struct p2m_domain *p2m = p2m_get_hostp2m(d);
+
+        if ( read_atomic(&p2m->ioreq.entry_count) )
+            p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
+    }
+
      return rc;
  }
diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
index a57b385..4b591fe 100644
--- a/xen/arch/x86/mm/hap/hap.c
+++ b/xen/arch/x86/mm/hap/hap.c
@@ -187,6 +187,15 @@ out:
   */
  static int hap_enable_log_dirty(struct domain *d, bool_t log_global)
  {
+    struct p2m_domain *p2m = p2m_get_hostp2m(d);
+
+    /*
+     * Refuse to turn on global log-dirty mode if
+     * there are outstanding p2m_ioreq_server pages.
+     */
+    if ( log_global && read_atomic(&p2m->ioreq.entry_count) )
+        return -EBUSY;
+
      /* turn on PG_log_dirty bit in paging mode */
      paging_lock(d);
      d->arch.paging.mode |= PG_log_dirty;
diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index cc1eb21..c66607a 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -544,6 +544,12 @@ static int resolve_misconfig(struct p2m_domain *p2m, 
unsigned long gfn)
                      e.ipat = ipat;
                      if ( e.recalc && p2m_is_changeable(e.sa_p2mt) )
                      {
+                         if ( e.sa_p2mt == p2m_ioreq_server )
+                         {
+                             ASSERT(p2m->ioreq.entry_count > 0);
+                             p2m->ioreq.entry_count--;
+                         }
+
                           e.sa_p2mt = p2m_is_logdirty_range(p2m, gfn + i, gfn 
+ i)
                                       ? p2m_ram_logdirty : p2m_ram_rw;
                           ept_p2m_type_to_flags(p2m, &e, e.sa_p2mt, e.access);
@@ -816,6 +822,18 @@ ept_set_entry(struct p2m_domain *p2m, unsigned long gfn, 
mfn_t mfn,
          new_entry.suppress_ve = is_epte_valid(&old_entry) ?
                                      old_entry.suppress_ve : 1;
+ /*
+     * p2m_ioreq_server is only used for 4K pages, so
+     * we only need to do the count for leaf entries.
+     */
+    if ( unlikely(ept_entry->sa_p2mt == p2m_ioreq_server) &&
+         ept_entry->sa_p2mt != p2mt &&
+         i == 0 )
+    {
+        ASSERT(p2m->ioreq.entry_count > 0);
+        p2m->ioreq.entry_count--;
+    }
Now that you've taken the accounting out of p2m_change_type_one(), I
don't see anywhere that you increase the ioreq.entry_count.  With the
ASSERT() above, won't this crash as soon as you try to detach the ioreq
server?

I think the normal way of doing the accounting here would be to have two
separate if statements; something like this:

if ( p2mt == p2m_ioreq_server )
   p2m->ioreq.entry_count++;

if ( ept_entry->sa_p2mt == p2m_ioreq_server )
   p2m->ioreq.entry_count--;

If the types are the same, then you increase it then decrease it, but
that's fine.  Same with p2m-pt.c.

Oh, right. I made a stupid mistake.

I've looked through the other calls to atomic_write_ept_entry() and
p2m->write_p2m_entry() in those files, and I think you've caught all the
places where the entry count needs to be updated; so with that fixed it
should be good to go (AFAICT).

Thanks. Will send out a new version later.

Yu
  -George




_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.