Re: [PATCH v10 00/13] switch to domheap for Xen page tables

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 21/04/2021 15:15, Hongyan Xia wrote:
> From: Hongyan Xia <hongyxia@xxxxxxxxxx>
>
> This series rewrites all the remaining functions and finally makes the
> switch from xenheap to domheap for Xen page tables, so that they no
> longer need to rely on the direct map, which is a big step towards
> removing the direct map.

Staging is broken.  Xen hits an assertion just after dom0 starts.

(XEN) Freed 616kB init memory
mapping kernel into physical memory
about to get started...
(XEN) Assertion 'hashent->refcnt' failed at domain_page.c:204
(XEN) ----[ Xen-4.16-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82d040316f80>] unmap_domain_page+0x2af/0x2e0
(XEN) RFLAGS: 0000000000010046   CONTEXT: hypervisor (d0v0)
(XEN) rax: 0000000000000000   rbx: ffff831c47bf9040   rcx: ffff831c47c1a000
(XEN) rdx: 0000000000000092   rsi: 0000000000000092   rdi: 0000000000000206
(XEN) rbp: ffff8300a5ca7c88   rsp: ffff8300a5ca7c78   r8:  0000000001c4f2fc
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000092018   r13: 0000000000800163   r14: fff0000000000000
(XEN) r15: 0000000000000001   cr0: 0000000080050033   cr4: 00000000003406e0
(XEN) cr3: 0000001c42008000   cr2: ffffc9000133d000
(XEN) fsb: 0000000000000000   gsb: ffff888266a00000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen code around <ffff82d040316f80> (unmap_domain_page+0x2af/0x2e0):
(XEN)  14 04 00 00 eb 19 0f 0b <0f> 0b 0f 0b ba 00 00 00 00 48 89 10 48
8b 81 d0
(XEN) Xen stack trace from rsp=ffff8300a5ca7c78:
(XEN)    ffff820040092018 0000000000000000 ffff8300a5ca7d58 ffff82d040327e20
(XEN)    a000000000000000 0000000000000000 ffff82d0405dbd40 008001e300000000
(XEN)    8000000000000000 8000000000000000 00000000000001e3 00000000000001e3
(XEN)    8000000000000000 0000000000000000 8000000000000163 0000000001440000
(XEN)    ffff82e0014b92e0 0000000301c1a000 0000000000000000 ffff820040090800
(XEN)    00000000026c10d8 0000000001c4f2fc 8010001c4240f067 ffff8300a5ca7df0
(XEN)    ffff82c00071c000 0000000000000001 0000000000001000 ffff8300a5ca7df8
(XEN)    ffff8300a5ca7dc8 ffff82d040232c08 ffff8300a5ca7db8 0000000140088078
(XEN)    ffff8300a5ca7df0 0080016300000001 ffffffff00000000 ffff82c00071c000
(XEN)    ffff82d0405b1300 ffff831c47bf9000 ffff82e04d821ae0 00000000026c10d7
(XEN)    ffff831c47c1a000 0000000000000100 ffff8300a5ca7dd8 ffff82d040232cdb
(XEN)    ffff8300a5ca7df8 ffff82d04031718b ffff8300a5ca7df8 00000000026c10d7
(XEN)    ffff8300a5ca7e38 ffff82d040209cb6 ffff831c47c1a018 0000000000000000
(XEN)    ffffffff82003e90 ffff831c47c1a018 ffff831c47bf9000 fffffffffffffff2
(XEN)    ffff8300a5ca7eb8 ffff82d04020a69a ffff82d04038a228 ffff82d04038a21c
(XEN)    00000000026c10d7 0000000000000100 ffff82d04038a228 ffff82d04038a21c
(XEN)    ffff82d04038a228 ffff82d04038a21c ffff82d04038a228 ffff8300a5ca7ef8
(XEN)    ffff831c47bf9000 0000000000000003 0000000000000000 0000000000000000
(XEN)    ffff8300a5ca7ee8 ffff82d040306e14 ffff82d04038a228 ffff831c47bf9000
(XEN)    0000000000000000 0000000000000000 00007cff5a3580e7 ffff82d04038a29d
(XEN) Xen call trace:
(XEN)    [<ffff82d040316f80>] R unmap_domain_page+0x2af/0x2e0
(XEN)    [<ffff82d040327e20>] F map_pages_to_xen+0x101a/0x1166
(XEN)    [<ffff82d040232c08>] F __vmap+0x332/0x3cd
(XEN)    [<ffff82d040232cdb>] F vmap+0x38/0x3a
(XEN)    [<ffff82d04031718b>] F map_domain_page_global+0x46/0x51
(XEN)    [<ffff82d040209cb6>] F map_vcpu_info+0x129/0x2c5
(XEN)    [<ffff82d04020a69a>] F do_vcpu_op+0x1eb/0x681
(XEN)    [<ffff82d040306e14>] F pv_hypercall+0x4e6/0x53d
(XEN)    [<ffff82d04038a29d>] F lstar_enter+0x12d/0x140
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion 'hashent->refcnt' failed at domain_page.c:204
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

I don't see an obvious candidate for the breakage.  Unless someone can
point one out quickly, I'll revert the lot to unblock staging.

~Andrew