[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Weird altp2m behaviour when switching early to a new view


  • To: Razvan Cojocaru <rcojocaru@xxxxxxxxxxxxxxx>, George Dunlap <dunlapg@xxxxxxxxx>
  • From: George Dunlap <george.dunlap@xxxxxxxxxx>
  • Date: Tue, 17 Apr 2018 15:58:29 +0100
  • Autocrypt: addr=george.dunlap@xxxxxxxxxx; prefer-encrypt=mutual; keydata= xsFNBFPqG+MBEACwPYTQpHepyshcufo0dVmqxDo917iWPslB8lauFxVf4WZtGvQSsKStHJSj 92Qkxp4CH2DwudI8qpVbnWCXsZxodDWac9c3PordLwz5/XL41LevEoM3NWRm5TNgJ3ckPA+J K5OfSK04QtmwSHFP3G/SXDJpGs+oDJgASta2AOl9vPV+t3xG6xyfa2NMGn9wmEvvVMD44Z7R W3RhZPn/NEZ5gaJhIUMgTChGwwWDOX0YPY19vcy5fT4bTIxvoZsLOkLSGoZb/jHIzkAAznug Q7PPeZJ1kXpbW9EHHaUHiCD9C87dMyty0N3TmWfp0VvBCaw32yFtM9jUgB7UVneoZUMUKeHA fgIXhJ7I7JFmw3J0PjGLxCLHf2Q5JOD8jeEXpdxugqF7B/fWYYmyIgwKutiGZeoPhl9c/7RE Bf6f9Qv4AtQoJwtLw6+5pDXsTD5q/GwhPjt7ohF7aQZTMMHhZuS52/izKhDzIufl6uiqUBge 0lqG+/ViLKwCkxHDREuSUTtfjRc9/AoAt2V2HOfgKORSCjFC1eI0+8UMxlfdq2z1AAchinU0 eSkRpX2An3CPEjgGFmu2Je4a/R/Kd6nGU8AFaE8ta0oq5BSFDRYdcKchw4TSxetkG6iUtqOO ZFS7VAdF00eqFJNQpi6IUQryhnrOByw+zSobqlOPUO7XC5fjnwARAQABzSRHZW9yZ2UgVy4g RHVubGFwIDxkdW5sYXBnQHVtaWNoLmVkdT7CwYAEEwEKACoCGwMFCwkIBwMFFQoJCAsFFgID AQACHgECF4ACGQEFAlpk2IEFCQo9I54ACgkQpjY8MQWQtG1A1BAAnc0oX3+M/jyv4j/ESJTO U2JhuWUWV6NFuzU10pUmMqpgQtiVEVU2QbCvTcZS1U/S6bqAUoiWQreDMSSgGH3a3BmRNi8n HKtarJqyK81aERM2HrjYkC1ZlRYG+jS8oWzzQrCQiTwn3eFLJrHjqowTbwahoiMw/nJ+OrZO /VXLfNeaxA5GF6emwgbpshwaUtESQ/MC5hFAFmUBZKAxp9CXG2ZhTP6ROV4fwhpnHaz8z+BT NQz8YwA4gkmFJbDUA9I0Cm9D/EZscrCGMeaVvcyldbMhWS+aH8nbqv6brhgbJEQS22eKCZDD J/ng5ea25QnS0fqu3bMrH39tDqeh7rVnt8Yu/YgOwc3XmgzmAhIDyzSinYEWJ1FkOVpIbGl9 uR6seRsfJmUK84KCScjkBhMKTOixWgNEQ/zTcLUsfTh6KQdLTn083Q5aFxWOIal2hiy9UyqR VQydowXy4Xx58rqvZjuYzdGDdAUlZ+D2O3Jp28ez5SikA/ZaaoGI9S1VWvQsQdzNfD2D+xfL qfd9yv7gko9eTJzv5zFr2MedtRb/nCrMTnvLkwNX4abB5+19JGneeRU4jy7yDYAhUXcI/waS /hHioT9MOjMh+DoLCgeZJYaOcgQdORY/IclLiLq4yFnG+4Ocft8igp79dbYYHkAkmC9te/2x Kq9nEd0Hg288EO/OwE0EVFq6vQEIAO2idItaUEplEemV2Q9mBA8YmtgckdLmaE0uzdDWL9To 1PL+qdNe7tBXKOfkKI7v32fe0nB4aecRlQJOZMWQRQ0+KLyXdJyHkq9221sHzcxsdcGs7X3c 17ep9zASq+wIYqAdZvr7pN9a3nVHZ4W7bzezuNDAvn4EpOf/o0RsWNyDlT6KECs1DuzOdRqD oOMJfYmtx9hMzqBoTdr6U20/KgnC/dmWWcJAUZXaAFp+3NYRCkk7k939VaUpoY519CeLrymd Vdke66KCiWBQXMkgtMGvGk5gLQLy4H3KXvpXoDrYKgysy7jeOccxI8owoiOdtbfM8TTDyWPR Ygjzb9LApA8AEQEAAcLBZQQYAQoADwUCVFq6vQIbDAUJAeEzgAAKCRCmNjwxBZC0bWknD/97 Tkh3PMAcvMZINmJefBdYYspmwTWZSR9USsy68oWzDsXKNDNTqBC781lR/7PSqhqaSOmSnty3 FNblaBYKfMV3OOWgrP0H8Voqp4IgH3yOOkQLVITIwulqbbxQtmCsJ3xkhZm6CA0EKbc9VM/j FX3aCAfOJf52vlY1gXjYOvVjrdrRrBXEjs8E5f6EsrQKDrWCKNx/9qRfmtsQeKHTsgpINkpZ s11ClX/sM/RCR9/BgB/K08QQZYsWD6lgZh1KxLXRzKRunba0L+jpcRsoQFUMj/ofrfnHAdl0 q2upzISM/wR8aer+kekMo+y00schmYJYu5JAAzbjQQuhCAg0UTBGPaNwteL2l3c9Ps8on1nl mq9TnbYwGLAxJzXSb3BATgz7dygpsBBNS5WhUNQgIJvcZJbLggEIqjZGs8o7/+dt4klwxCYL FVlsWYSwEjX0UYHVLMS/F7FcXbCMUeoN/4krmRyv7YICE/VDQSDPcSKedzWvQM8T+5uY5pFJ NiIaa6asFndP50GiKbFtD6xAM+rbnwT7Io+iPtvD/3ddMXQs58IVMzgNA/hcdOX/qlx6Jqk/ hYQQsl4HoQsx/GyrNiwiPErTx32QNeXxoGYm6kwxt7F5qK7AN5tyYNkEyoxYrv8bl9VjAve8 hpECyf4O1mOGC/dIuBCDk8gxL5Pbo3jl98LBZQQYAQoADwIbDAUCVlNqsQUJA9njdAAKCRCm NjwxBZC0bbJMEACigmtpL2lzS47DXydApr1X8SYCHIPc39OjvmErjP05lKUZjmesmhlM5eKO gPb/fzeJ0wXB4J8OyseIJ0D/XwyLLQeM8d/HUFFMBWr+HE7jIukAUXeQ6GRwR+MBYGK/KmR9 JHbMAUz8f3G087Ma12BfpNWayndlFwR3rvdV4lvlyx6cl0EaFhbzPu/N07HG5MTk0evtphgZ 7wuG1oAtO+DGA6orHEicor6nBAQNZzPyjqo40dBxTs+amx7UndMRPSL1dD57eJwbbvBeNa8I w8wT7oNy2/C21VWmSy5XzMzcUTgmjmQz6DSNJPz2dMK4Y/LtcVFTfSZTmlBIkfoc9Vay2EB9 3z2EmjZwGT7n/DRu9QDtLbXyeVTBuLTaP3D+q5AyR1/5Z4T0LhwNvxeND5yO+YNAwqocZwL+ OcctpSZUBpAuU4Ju/9JKMX57GlnbjB8YGahoBJsQZx4CZyw0MXlkCk5cR0EPjY9iI2CEA5lO QueOSbo0hf1ZJwCx724lx0WSwL8ngd8wZTYMNc8GngaU61kmzfcuCklhokTxQdK7Efme5ccv A1txzgGewx9mDhPgNcJweasBnyL0N3wya2RMAzm04gCio8y4FKQepwQpKCNKAYZIU4juAPxn nb6cbBGiMGO1NDuxG+qvl1cMElnq+cuhSUlZdr2sE9JRfa0gucLBZQQYAQoADwIbDAUCWHQN VAUJBfqGFwAKCRCmNjwxBZC0bbgCD/oC6mWUrxQKWPDvFE9+fzm8UKqKP7aciz+gvWUN3o4i 4sRFNyvAEOW/QY2zwM1pN07BFZ3Z+8AVxpgR6h7RQzDJYSPZ5k5WWCJzJEQs2sPI5rfYJGK8 um7mlsSvf2xcLK/1Aj07BmWDjR6glDDRY+iMmSSdHe6Te6tiQPPS6Woj8AE3qf5lBsdvcEln nrkSwzNeVKRQQROUOskVw4WmCsNJjZtKmrVpgId3df/5HWG7Bi4nPwA8IFOt6O72lJlkORFy DF5P7ML7Pc5LbEFimzETPBxTJzVu1UoOQb/THB+qxhKMXXudSf/5sdMhwvOwItIcc5pib/v6 7gWK48bAzoOTgNYzmDCVC/roeLLU2SpEQIlIR0eAaWImgt8VEtre3Gch33e41DtbUli54DX0 dRdhqQaDM1T1q77VyDoZcs+SpGX9Ic9mxl+BN+6vtGIUVgaOG5pF85aQlRfCD6IlFQgiZtiR XeRpeIYG27RUw5kIljW+VxPMdBUvZpUXEazqjoPvBKybg0oKFfMXrMj4vHo6J0FD3ZEToGnP dANspUCZRewRozjp7ZWIu7QfGasfJNQ8c1IDiAFl3rV+dAGXXdmrDcX6w2q5lqoFz+8npK2I ehKCA94U+J/RLywUiaLuHnXt40WvQ98kHm7uTsy36iWqqawPqzmn8m5ruynVHmmcXsLBZQQY AQoADwIbDAUCWmTXMwUJB+tP9gAKCRCmNjwxBZC0bb+2D/9hjn1k5WcRHlu19WGuH6q0Kgm1 LRT7PnnSz904igHNElMB5a7wRjw5kdNwU3sRm2nnmHeOJH8kYj2Hn1QgX5SqQsysWTHWOEse GeoXydx9zZZkt3oQJM+9NV1VjK0bOXwqhiQyEUWz5/9l467FS/k4FJ5CHNRumvhLa0l2HEEu 5pxq463HQZHDt4YE/9Y74eXOnYCB4nrYxQD/GSXEZvWryEWreDoaFqzq1TKtzHhFgQG7yFUE epxLRUUtYsEpT6Rks2l4LCqG3hVD0URFIiTyuxJx3VC2Ta4LH3hxQtiaIpuXqq2D4z63h6vC x2wxfZc/WRHGbr4NAlB81l35Q/UHyMocVuYLj0llF0rwU4AjiKZ5qWNSEdvEpL43fTvZYxQh DCjQTKbb38omu5P4kOf1HT7s+kmQKRtiLBlqHzK17D4K/180ADw7a3gnmr5RumcZP3NGSSZA 6jP5vNqQpNu4gqrPFWNQKQcW8HBiYFgq6SoLQQWbRxJDHvTRYJ2ms7oCe870gh4D1wFFqTLe yXiVqjddENGNaP8ZlCDw6EU82N8Bn5LXKjR1GWo2UK3CjrkHpTt3YYZvrhS2MO2EYEcWjyu6 LALF/lS6z6LKeQZ+t9AdQUcILlrx9IxqXv6GvAoBLJY1jjGBq+/kRPrWXpoaQn7FXWGfMqU+ NkY9enyrlw==
  • Cc: "Tian, Kevin" <kevin.tian@xxxxxxxxx>, Tamas K Lengyel <tamas@xxxxxxxxxxxxx>, Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>, Tim Deegan <tim@xxxxxxx>, Jun Nakajima <jun.nakajima@xxxxxxxxx>, xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Tue, 17 Apr 2018 14:58:38 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Openpgp: preference=signencrypt

On 04/17/2018 03:21 PM, Razvan Cojocaru wrote:
> On 04/17/2018 04:53 PM, George Dunlap wrote:
>> On 04/17/2018 11:50 AM, Razvan Cojocaru wrote:
>>>  void p2m_init_altp2m_ept(struct domain *d, unsigned int i)
>>>  {
>>>      struct p2m_domain *p2m = d->arch.altp2m_p2m[i];
>>> +    struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
>>>      struct ept_data *ept;
>>>  
>>> +    p2m->max_mapped_pfn = hostp2m->max_mapped_pfn;
>>> +    p2m->default_access = hostp2m->default_access;
>>> +    p2m->domain = hostp2m->domain;
>>> +    p2m->logdirty_ranges = hostp2m->logdirty_ranges;
>>> +    p2m->global_logdirty = hostp2m->global_logdirty;
>>
>> This would certainly be one approach.  But then we'd need to keep a lot
>> more of these things in sync -- for instance, we'd have to have similar
>> code to enable and disable global_logdirty on all active altp2m entries.
>>
>> I also don't think the max_mapped_pfn should be copied here; the fact
>> that updates got filtered out before was a red herring I think.
> 
> I initially thought so too, and now I've commented out just that one
> line to remember why I couldn't remove, and the reason is this:
> 
> (XEN) Assertion 's <= e' failed at rangeset.c:121
> (XEN) ----[ Xen-4.11-unstable  x86_64  debug=y   Not tainted ]----
> (XEN) CPU:    0
> (XEN) RIP:    e008:[<ffff82d080228826>] rangeset_add_range+0x5/0x1e6
> (XEN) RFLAGS: 0000000000010206   CONTEXT: hypervisor (d0v0)
> (XEN) rax: 0000000000000000   rbx: ffff830b577f92e0   rcx: 00000000000f0000
> (XEN) rdx: 0000000000000000   rsi: 00000000000f0000   rdi: ffff830ad6a1ce50
> (XEN) rbp: ffff83007ce87c78   rsp: ffff83007ce87c20   r8:  0000000000000000
> (XEN) r9:  0000000000000000   r10: 000000000000006f   r11: 0000000000000028
> (XEN) r12: 0000000000000002   r13: 0000000000000000   r14: 0000000000000001
> (XEN) r15: ffff830ad6ddd000   cr0: 0000000080050033   cr4: 00000000003526e0
> (XEN) cr3: 0000000ad714f000   cr2: 0000000000c12000
> (XEN) fsb: 00007f794c7b2700   gsb: ffff880276c00000   gss: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen code around <ffff82d080228826> (rangeset_add_range+0x5/0x1e6):
> (XEN)  00 5d c3 48 39 d6 76 02 <0f> 0b 55 48 89 e5 41 56 41 55 41 54 53
> 49 89 d5
> (XEN) Xen stack trace from rsp=ffff83007ce87c20:
> (XEN)    ffff82d08032c644 0000000000000206 0000000000000010 0000000000000028
> (XEN)    00000000000f0000 ffff82c000217000 0000000000000000 ffff830ad6ddd000
> (XEN)    00000000000f0240 0000000000000000 0000000000000002 ffff83007ce87cc8
> (XEN)    ffff82d08032c7ac 0000000000000240 00000000000f0000 ffff82c000217000
> (XEN)    ffff830ad6ddd000 00000000000f0240 0000000000000048 ffff82c000217000
> (XEN)    00000000000f0000 ffff83007ce87d38 ffff82d080362ade ffff830c5bb20000
> (XEN)    ffff830ad6ddd650 0000000000000000 0000000000000000 00007f794c7bd004
> (XEN)    0000000000000048 ffff830c5bb20000 0000000000000000 ffff83007ce87e00
> (XEN)    ffffffffffffffea ffff82d0802e9c64 deadbeefdeadf00d ffff83007ce87de8
> (XEN)    ffff82d0802e92c1 ffff830c5bb20000 ffff83007d616000 0000000000000000
> (XEN)    ffff83007ce87dd8 0000000000000000 ffff83007ce87df8 ffff83007ce87df4
> (XEN)    ffff82e016a5de40 ffff83007d616000 0000000000000007 0000000000000240
> (XEN)    00000000000f0000 ffff83007ce87dc8 ffff830ad6ddd000 ffff83007ce87dc8
> (XEN)    0000000000000002 0000000000000001 00007f794c7bf004 ffff82d0802e9c64
> (XEN)    deadbeefdeadf00d ffff83007ce87e48 ffff82d0802e9ce1 ffff82d080374434
> (XEN)    0000000280370001 00007f794c7be004 0000000000000020 00007f794c7bd004
> (XEN)    0000000000000048 ffff82d080374434 ffff83007ce87ef8 ffff83007d616000
> (XEN)    0000000000000029 ffff83007ce87ee8 ffff82d08036d8a6 03ff82d080374434
> (XEN)    0000000000000001 0000000000000002 00007f794c7bf004 deadbeefdeadf00d
> (XEN)    deadbeefdeadf00d ffff82d080374434 ffff82d080374428 ffff82d080374434
> (XEN) Xen call trace:
> (XEN)    [<ffff82d080228826>] rangeset_add_range+0x5/0x1e6
> (XEN)    [<ffff82d08032c7ac>] p2m_change_type_range+0x66/0x7f
> (XEN)    [<ffff82d080362ade>] hap_track_dirty_vram+0x240/0x491
> (XEN)    [<ffff82d0802e92c1>] dm.c#dm_op+0x45c/0xd06
> (XEN)    [<ffff82d0802e9ce1>] do_dm_op+0x7d/0xb3
> (XEN)    [<ffff82d08036d8a6>] pv_hypercall+0x1f4/0x440
> (XEN)    [<ffff82d080374495>] lstar_enter+0x115/0x120
> (XEN)
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) Assertion 's <= e' failed at rangeset.c:121
> (XEN) ****************************************

Right, but this is basically a bug in p2m_change_type_range(), where it
handles end > max_mapped_pfn, but not start > max_mapped_pfn.  It should
check the latter just after grabbing the lock and bail if true.  (This
should probably be in a separate patch, as it's really a generic bug in
p2m_change_type_range().)

>> Another approach would be to maintain the logdirty_ranges and
>> global_logdirty only for the host p2m, but to misconfigure entries for
>> all the p2ms; and then on a misconfiguration, handle the
>> misconfiguration for the hostp2m and then do a lazy propagate for the
>> altp2m.  On the whole that's probably more error-prone than just doing a
>> for() loop, though, and not that much faster. :-)
> We can try that too.

If you want -- but as I said, arguably your approach is more robust;
just incomplete.

>> The other thing that seems to be missing from synchronization is that in
>> p2m-ept.c:ept_enable_pml() sets the p2m->ept.ad bit (which ends up being
>> part of the eptp).  The code seems to indicate that this is required for
>> PML (hardware-assisted logdirty), but I don't see anywhere this is set
>> or copied from the host p2m.
>>
>> It might be nice to have a more structured way of keeping all these
>> changes in sync, rather than relying on this open-coding everywhere.
> 
> Very true. It has also occured to me that some of these issues would be
> at least partially mitigated if altp2m was always on, but of course I
> can also see why that would be frowned upon at this time.

How would having it enabled all the time have helped in this situation?

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.