Re: [Xen-devel] [PATCH v1] x86/hvm: Generic instruction re-execution mechanism for execute faults

[Razvan Cojocaru <rcojocaru@xxxxxxxxxxxxxxx>]

On 11/22/18 12:05 PM, Roger Pau Monné wrote:
> On Wed, Nov 21, 2018 at 08:55:48PM +0200, Razvan Cojocaru wrote:
>> On 11/16/18 7:04 PM, Roger Pau Monné wrote:
>>>> +            if ( a == v )
>>>> +                continue;
>>>> +
>>>> +            /* Pause, synced. */
>>>> +            while ( !a->arch.in_host )
>>> Why not use a->is_running as a way to know whether the vCPU is
>>> running?
>>>
>>> I think the logic of using vcpu_pause and expecting the running vcpu
>>> to take a vmexit and thus set in_host is wrong because a vcpu that
>>> wasn't running when vcpu_pause_nosync is called won't get scheduled
>>> anymore, thus not taking a vmexit and this function will lockup.
>>>
>>> I don't think you need the in_host boolean at all.
>>>
>>>> +                cpu_relax();
>>> Is this really better than using vcpu_pause?
>>>
>>> I assume this is done to avoid waiting on each vcpu, and instead doing
>>> it here likely means less wait time?
>>
>> The problem with plain vcpu_pause() is that we weren't able to use it,
>> for the same reason (which remains unclear as of yet) that we couldn't
>> use a->is_running: we get CPU stuck hypervisor crashes that way. Here's
>> one that uses the same logic, but loops on a->is_running instead of
>> !a->arch.in_host:
>>
>> (XEN) [ 3663.19(XEN) [ 3667.995061] Watchdog timer detects that CPU0 is
>> stuck!
>> (XEN) [ 3668.000694] ----[ Xen-4.7.5  x86_64  debug=y  Not tainted ]----
>> (XEN) [ 3668.007108] CPU:    0
>> (XEN) [ 3668.009882] RIP:    e008:[<ffff82d0801327d2>]
>> vcpu_sleep_sync+0x40/0x71
>> (XEN) [ 3668.016989] RFLAGS: 0000000000000202   CONTEXT: hypervisor (d0v0)
>> (XEN) [ 3668.023575] rax: 0000000000000001   rbx: ffff83007ccfc000
>> rcx: ffff83007ccfc128
>> (XEN) [ 3668.031548] rdx: 0000000000000000   rsi: 0000000000000246
>> rdi: ffff830c52984148
>> (XEN) [ 3668.039522] rbp: ffff83007cf2fcd8   rsp: ffff83007cf2fcc8   r8:
>>  0000000000000003
>> (XEN) [ 3668.047495] r9:  0000000000000000   r10: ffff82d080348460
>> r11: 0000000000000000
>> (XEN) [ 3668.055465] r12: ffff82d080132792   r13: ffff830b172b4000
>> r14: ffff82c000225000
>> (XEN) [ 3668.063439] r15: 00000000000f0000   cr0: 0000000080050033
>> cr4: 00000000003526e0
>> (XEN) [ 3668.071415] cr3: 0000000b4ba94000   cr2: 00007f6161714f70
>> (XEN) [ 3668.077308] fsb: 00007f9164f088c0   gsb: ffff880276c00000
>> gss: 0000000000000000
>> (XEN) [ 3668.085280] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss:
>> e010   cs: e008
>> (XEN) [ 3668.092731] Xen code around <ffff82d0801327d2>
>> (vcpu_sleep_sync+0x40/0x71):
>> (XEN) [ 3668.100186]  01 00 00 00 74 24 f3 90 <8b> 11 48 8b 43 10 8b 80
>> dc 01 00 00 09 d0 48 98
>> (XEN) [ 3668.108593] Xen stack trace from rsp=ffff83007cf2fcc8:
>> (XEN) [ 3668.114223]    0000000000000240 ffff83007ccfc000
>> ffff83007cf2fd08 ffff82d08010735b
>> (XEN) [ 3668.122282]    ffff82d0801358ad ffff830b172b4000
>> 0000000000000240 0000000000000048
>> (XEN) [ 3668.130346]    ffff83007cf2fd18 ffff82d08010879a
>> ffff83007cf2fd88 ffff82d080245e69
>> (XEN) [ 3668.138402]    ffff83007d615000 ffff830b172b4658
>> ffff83007cf2fd48 00000000000f0000
>> (XEN) [ 3668.146464]    00007f9164fb8004 0000000000000048
>> ffff830c52974000 0000000000000006
>> (XEN) [ 3668.154523]    ffffffffffffffff ffffffffffffffea
>> 00007f9164fb1004 0000000000000000
>> (XEN) [ 3668.162584]    ffff83007cf2fe48 ffff82d0801dd8f5
>> ffff82d080374d58 ffff82d08024b308
>> (XEN) [ 3668.170643]    ffff83007cf2fdc8 ffff83007cf2ffff
>> ffff83007cf2fdc8 ffff830b172b4000
>> (XEN) [ 3668.178704]    0000024000000001 00000000000f0000
>> 00007f9164fb8004 fffffffffffffffc
>> (XEN) [ 3668.186763]    0000000000000293 00007f91631f85d3
>> ffff82d080250834 ffff82d080250828
>> (XEN) [ 3668.194820]    ffff82d080250834 ffff82d080250828
>> ffff82d080250834 ffff83007cf2fef8
>> (XEN) [ 3668.202882]    0000000000000022 ffff82d0801dc037
>> deadbeefdeadf00d ffffffff8100144a
>> (XEN) [ 3668.210942]    ffff83007cf2fee8 ffff82d080172aca
>> 02ff82d080250834 0000000000000006
>> (XEN) [ 3668.219000]    00007f9164fb1004 deadbeefdeadf00d
>> deadbeefdeadf00d deadbeefdeadf00d
>> (XEN) [ 3668.227062]    ffff82d080250834 ffff82d080250828
>> ffff82d080250834 ffff82d080250828
>> (XEN) [ 3668.235121]    ffff82d080250834 ffff82d080250828
>> ffff82d080250834 ffff83007d615000
>> (XEN) [ 3668.243180]    0000000000000000 0000000000000000
>> 0000000000000000 0000000000000000
>> (XEN) [ 3668.251240]    00007cff830d00e7 ffff82d080250899
>> 00007ffef6baf1d0 0000000000305000
>> (XEN) [ 3668.259298]    ffff88022740b900 fffffffffffffff2
>> ffff88022b31fe98 ffff88026f3374d8
>> (XEN) [ 3668.267361]    0000000000000282 0000000000000000
>> ffff88007c995080 0000000000000000
>> (XEN) [ 3668.275417] Xen call trace:
>> (XEN) [ 3668.278714]    [<ffff82d0801327d2>] vcpu_sleep_sync+0x40/0x71
>> (XEN) [ 3668.284952]    [<ffff82d08010735b>]
>> domain.c#do_domain_pause+0x33/0x4f
>> (XEN) [ 3668.291973]    [<ffff82d08010879a>] domain_pause+0x25/0x27
>> (XEN) [ 3668.297952]    [<ffff82d080245e69>]
>> hap_track_dirty_vram+0x2c1/0x4a7
>> (XEN) [ 3668.304797]    [<ffff82d0801dd8f5>] do_hvm_op+0x18be/0x2b58
>> (XEN) [ 3668.310864]    [<ffff82d080172aca>] pv_hypercall+0x1e5/0x402
>> (XEN) [ 3668.317017]    [<ffff82d080250899>] entry.o#test_all_events+0/0x3d
>> (XEN) [ 3668.323689]
>> (XEN) [ 3668.325685]
>> (XEN) [ 3668.327678] ****************************************
>> (XEN) [ 3668.333138] Panic on CPU 0:
>> (XEN) [ 3668.336428] FATAL TRAP: vector = 2 (nmi)
>> (XEN) [ 3668.340850] [error_code=0000]
>> (XEN) [ 3668.344404] ****************************************
>> (XEN) [ 3668.349863]
>> (XEN) [ 3668.351854] Reboot in five seconds...
>> (XEN) [ 3668.356017] Dumping other CPUs
>> (XEN) [ 3668.359567] *** Dumping CPU1 host state: ***
>> (XEN) [ 3668.364337] ----[ Xen-4.7.5  x86_64  debug=y  Not tainted ]----
>> (XEN) [ 3668.370750] CPU:    1
>> (XEN) [ 3668.373522] RIP:    e008:[<ffff82d08016b5a6>]
>> domain.c#default_idle+0xa2/0xb5
>> (XEN) [ 3668.381149] RFLAGS: 0000000000000202   CONTEXT: hypervisor
>> (XEN) [ 3668.387128] rax: 0000000000000000   rbx: ffff830c529b7fff
>> rcx: 0000000000000048
>> (XEN) [ 3668.395101] rdx: 0000000000000000   rsi: ffff830c529b7fff
>> rdi: ffff830c529b7ef8
>> (XEN) [ 3668.403076] rbp: ffff830c529b7ed0   rsp: ffff830c529b7ed0   r8:
>>  ffff830c529fe4a8
>> (XEN) [ 3668.411048] r9:  ffff830c529bac20   r10: ffff830c529fe490
>> r11: ffff830c529ba148
>> (XEN) [ 3668.419019] r12: ffff830c529ba140   r13: ffff83007cf75000
>> r14: 000003540fd7cd6b
>> (XEN) [ 3668.426994] r15: ffffffffffffffff   cr0: 000000008005003b
>> cr4: 00000000003526e0
>> (XEN) [ 3668.434964] cr3: 000000007cf1d000   cr2: 0000000000000000
>> (XEN) [ 3668.440861] fsb: 0000000000000000   gsb: 0000000000000000
>> gss: 0000000000000000
>> (XEN) [ 3668.448832] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss:
>> 0000   cs: e008
>> (XEN) [ 3668.456285] Xen code around <ffff82d08016b5a6>
>> (domain.c#default_idle+0xa2/0xb5):
>> (XEN) [ 3668.464260]  00 00 00 0f 30 90 fb f4 <0f> b6 46 f5 80 a7 fd 00
>> 00 00 fe 0f 30 90 eb 01
>> (XEN) [ 3668.472663] Xen stack trace from rsp=ffff830c529b7ed0:
>> (XEN) [ 3668.478297]    ffff830c529b7ef0 ffff82d08016b628
>> ffff82d080134ffe ffff83007cf75000
>> (XEN) [ 3668.486358]    ffff830c529b7df0 0000000000000000
>> 0000000000000000 0000000000000000
>> (XEN) [ 3668.494417]    0000000000000000 00000000001c3a38
>> 0000000000000000 0000000000000000
>> (XEN) [ 3668.502478]    0000000000000000 0000000000000000
>> 0000000000000000 0000000000000000
>> (XEN) [ 3668.510538]    00000000000002ff 00000000001c00e9
>> 0000000000000000 0000000000000000
>> (XEN) [ 3668.518595]    0000beef0000beef 0000000000103f15
>> 000000bf0000beef 0000000000000046
>> (XEN) [ 3668.526656]    00000000001c3a38 000000000000beef
>> ffffea000d5bbeef ffffea000d5bbeef
>> (XEN) [ 3668.534715]    000000000000beef 000000000000beef
>> 017fffc000000001 ffff83007cf75000
>> (XEN) [ 3668.542775]    0000003bd2646380 00000000003526e0
>> 0000000000000000 0000000c5299e000
>> (XEN) [ 3668.550837]    0000070100000000 0000000000000000
>> (XEN) [ 3668.555948] Xen call trace:
>> (XEN) [ 3668.559242]    [<ffff82d08016b5a6>] domain.c#default_idle+0xa2/0xb5
>> (XEN) [ 3668.566000]    [<ffff82d08016b628>] domain.c#idle_loop+0x57/0x6e
>> (XEN) [ 3668.572502]
>> (XEN) [ 3668.574494] *** Dumping CPU2 host state: ***
>> (XEN) [ 3668.579261] ----[ Xen-4.7.5  x86_64  debug=y  Not tainted ]----
>> (XEN) [ 3668.585675] CPU:    2
>> (XEN) [ 3668.588449] RIP:    e008:[<ffff82d080127880>]
>> queue_read_lock_slowpath+0x27/0x4d
>> (XEN) [ 3668.596332] RFLAGS: 0000000000000286   CONTEXT: hypervisor (d1v1)
>> (XEN) [ 3668.602919] rax: 00000000000000ff   rbx: ffff830b1b2b6980
>> rcx: 0000000000000000
>> (XEN) [ 3668.610893] rdx: ffff830c52997fff   rsi: 0000000000000009
>> rdi: ffff830b1b2b698a
>> (XEN) [ 3668.618865] rbp: ffff830c52997a68   rsp: ffff830c52997a58   r8:
>>  0000000000000000
>> (XEN) [ 3668.626837] r9:  0000000000000003   r10: 0000000000000000
>> r11: 0000000000000000
>> (XEN) [ 3668.634812] r12: ffff830b1b2b6984   r13: ffff830c52997aa4
>> r14: ffff830c52997c34
>> (XEN) [ 3668.642786] r15: 00000000000001aa   cr0: 0000000080050033
>> cr4: 00000000003526e0
>> (XEN) [ 3668.650759] cr3: 0000000b105ef000   cr2: 00000190068c3000
>> (XEN) [ 3668.656650] fsb: 0000000000000000   gsb: 0000000000000000
>> gss: 0000004f58bd3000
>> (XEN) [ 3668.664624] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss:
>> 0000   cs: e008
>> (XEN) [ 3668.672077] Xen code around <ffff82d080127880>
>> (queue_read_lock_slowpath+0x27/0x4d):
>> (XEN) [ 3668.680309]  84 c0 74 08 f3 90 8b 03 <84> c0 75 f8 b8 00 01 00
>> 00 f0 0f c1 03 3c ff 75
>> (XEN) [ 3668.688717] Xen stack trace from rsp=ffff830c52997a58:
>> (XEN) [ 3668.694351]    ffff830b1b2b6980 ffff830c52997b54
>> ffff830c52997ad8 ffff82d08020c1df
>> (XEN) [ 3668.702411]    ffff830c52997b08 ffff82d080217db4
>> ffff830b172b4000 0000000352997c44
>> (XEN) [ 3668.710468]    000000000db12f43 0000000000000000
>> ffff830c00000000 00000000000001aa
>> (XEN) [ 3668.718529]    ffff830b1b2b6980 fffff801a1e18d03
>> ffff830c52997c34 ffff830078ba7000
>> (XEN) [ 3668.726591]    ffff830c52997b88 ffff82d080247208
>> ffff830b1b2b6980 ffff830c52997c44
>> (XEN) [ 3668.734648]    0000000000000000 fffff801a1e18d03
>> ffff830c52997b68 ffff82d08020bf20
>> (XEN) [ 3668.742707]    0000000000000000 0000000208a008e3
>> ffff830c52997b58 0000000400000000
>> (XEN) [ 3668.750768]    0000000000008000 0000000000000000
>> ffff830c52997be0 0000000000000000
>> (XEN) [ 3668.758826]    0000000000000000 ffff830078ba7000
>> ffff830c52997c34 fffff801a1e18d03
>> (XEN) [ 3668.766888]    ffff830b1b2b6980 ffff82d080311520
>> ffff830c52997b98 ffff82d080247475
>> (XEN) [ 3668.774945]    ffff830c52997be8 ffff82d080212751
>> 0000000000008000 ffffef07c38b76b0
>> (XEN) [ 3668.783006]    0000000000000010 fffff801a1e18d03
>> fffff801a1e18d03 0000000000000d03
>> (XEN) [ 3668.791067]    000fffff801a1e18 ffff830c52997ef8
>> ffff830c52997c78 ffff82d0801d66a0
>> (XEN) [ 3668.799128]    ffffef07c38b7708 ffff830c52997c44
>> ffff830c52997c34 0000000000000004
>> (XEN) [ 3668.807188]    ffff830c52997d38 0000001000000004
>> ffff830078ba7000 0000001100000010
>> (XEN) [ 3668.815244]    ffffea000d59beef ffffea000d59beef
>> 000000000000beef ffff830c52997d10
>> (XEN) [ 3668.823304]    ffff830078ba7000 0000000000000001
>> 0000000000000000 ffff830c52997ef8
>> (XEN) [ 3668.831363]    ffff830c52997c88 ffff82d0801d844d
>> ffff830c52997ce8 ffff82d0801d13da
>> (XEN) [ 3668.839423]    ffff830c52997d38 ffff82d0803107e0
>> 0000000000000000 fffff801a1e18d03
>> (XEN) [ 3668.847484]    ffff830c52997cd8 ffff830078ba7000
>> ffff830c52997d10 000000000000002c
>> (XEN) [ 3668.855544] Xen call trace:
>> (XEN) [ 3668.858838]    [<ffff82d080127880>]
>> queue_read_lock_slowpath+0x27/0x4d
>> (XEN) [ 3668.865857]    [<ffff82d08020c1df>]
>> get_page_from_gfn_p2m+0x14e/0x3b0
>> (XEN) [ 3668.872792]    [<ffff82d080247208>]
>> hap_p2m_ga_to_gfn_4_levels+0x48/0x299
>> (XEN) [ 3668.880071]    [<ffff82d080247475>]
>> hap_gva_to_gfn_4_levels+0x1c/0x1e
>> (XEN) [ 3668.887004]    [<ffff82d080212751>] paging_gva_to_gfn+0x10e/0x11d
>> (XEN) [ 3668.893590]    [<ffff82d0801d66a0>] hvm.c#__hvm_copy+0x98/0x37f
>> (XEN) [ 3668.900003]    [<ffff82d0801d844d>]
>> hvm_fetch_from_guest_virt_nofault+0x14/0x16
>> (XEN) [ 3668.907801]    [<ffff82d0801d13da>]
>> emulate.c#_hvm_emulate_one+0x118/0x2bc
>> (XEN) [ 3668.915168]    [<ffff82d0801d1674>] hvm_emulate_one+0x10/0x12
>> (XEN) [ 3668.921409]    [<ffff82d0801e08c2>] handle_mmio+0x52/0xc9
>> (XEN) [ 3668.927303]    [<ffff82d0802034a2>]
>> vmx_vmexit_handler+0x1e0e/0x1e45
>> (XEN) [ 3668.934149]    [<ffff82d08020820c>]
>> vmx_asm_vmexit_handler+0xec/0x250
>> (XEN) [ 3668.941079]
>> (XEN) [ 3668.943072] *** Dumping CPU2 guest state (d1v1): ***
>> (XEN) [ 3668.948533] ----[ Xen-4.7.5  x86_64  debug=y  Not tainted ]----
>> (XEN) [ 3668.954948] CPU:    2
>> (XEN) [ 3668.957719] RIP:    0010:[<fffff801a1e18d03>]
>> (XEN) [ 3668.962572] RFLAGS: 0000000000010046   CONTEXT: hvm guest (d1v1)
>> (XEN) [ 3668.969075] rax: fffff78880009000   rbx: 000000000004002f
>> rcx: fffff801a1e19300
>> (XEN) [ 3668.977045] rdx: ffffef07c38b76b8   rsi: ffffef07c38b7708
>> rdi: 0000000000000000
>> (XEN) [ 3668.985018] rbp: ffffef07c38b76b0   rsp: ffffef07c38b75f0   r8:
>>  ffffef07c38b7708
>> (XEN) [ 3668.992991] r9:  000000000000002f   r10: 0000000000000001
>> r11: 0000000000000001
>> (XEN) [ 3669.000966] r12: 0000000000000001   r13: 0000000000000000
>> r14: 0000000000000001
>> (XEN) [ 3669.008938] r15: 000000000000002f   cr0: 0000000080050031
>> cr4: 0000000000170678
>> (XEN) [ 3669.016913] cr3: 00000000001aa002   cr2: 00000190068c3000
>> (XEN) [ 3669.022806] fsb: 0000000000000000   gsb: ffffc9814c820000
>> gss: 0000000473bfe000
>> (XEN) [ 3669.030776] ds: 002b   es: 002b   fs: 0053   gs: 002b   ss:
>> 0000   cs: 0010
>> (XEN) [ 3669.038229]
>> (XEN) [ 3669.040223] *** Dumping CPU3 host state: ***
>> (XEN) [ 3669.044988] ----[ Xen-4.7.5  x86_64  debug=y  Not tainted ]----
>> (XEN) [ 3669.051403] CPU:    3
>> (XEN) [ 3669.054177] RIP:    e008:[<ffff82d08021006a>]
>> vmx_start_reexecute_instruction+0x107/0x68a
>> (XEN) [ 3669.062841] RFLAGS: 0000000000000202   CONTEXT: hypervisor (d1v0)
>> (XEN) [ 3669.069431] rax: ffff830078ba7000   rbx: ffff83007ccfc000
>> rcx: 0000000000000002
>> (XEN) [ 3669.077404] rdx: ffff830c5297ffff   rsi: 0000000000000246
>> rdi: ffff830c52998148
>> (XEN) [ 3669.085377] rbp: ffff830c5297fd18   rsp: ffff830c5297fcb8   r8:
>>  0000000000000002
>> (XEN) [ 3669.093349] r9:  0000000000000006   r10: 000000000003d976
>> r11: 0000000000000006
>> (XEN) [ 3669.101320] r12: 0000000000000000   r13: ffff82d08028a3e4
>> r14: 0000000000000000
>> (XEN) [ 3669.109296] r15: 0000000113f007f8   cr0: 0000000080050033
>> cr4: 00000000003526e0
>> (XEN) [ 3669.117269] cr3: 0000000b10380000   cr2: 0000000000000000
>> (XEN) [ 3669.123163] fsb: 0000000000000000   gsb: 0000000000000000
>> gss: fffff801a129e000
>> (XEN) [ 3669.131132] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss:
>> 0000   cs: e008
>> (XEN) [ 3669.138586] Xen code around <ffff82d08021006a>
>> (vmx_start_reexecute_instruction+0x107/0x68a):
>> (XEN) [ 3669.147598]  90 80 b8 0b 01 00 00 00 <75> f5 48 8b 40 18 48 85
>> c0 75 de e9 19 05 00 00
>> (XEN) [ 3669.156005] Xen stack trace from rsp=ffff830c5297fcb8:
>> (XEN) [ 3669.161640]    ffff830b17342800 0000000300000009
>> 0000000000000100 ffff830b17342000
>> (XEN) [ 3669.169697]    ffff830c00000000 ffff830b1b2b6980
>> ffff830b172b4000 ffff830b1b2b6980
>> (XEN) [ 3669.177761]    000000001b2b6801 0000000000000002
>> ffff83007ccfc000 000000000000003b
>> (XEN) [ 3669.185818]    ffff830c5297fda8 ffff82d080210b3e
>> 0000000000113f00 0000000000000000
>> (XEN) [ 3669.193877]    00007ff91cd34d60 0000000113f007f8
>> 0000000000000000 ffff830c5297fdf0
>> (XEN) [ 3669.201937]    0000000000113f00 0000000000000000
>> ffff83007ccfc000 0000000000000005
>> (XEN) [ 3669.209997]    ffff83007ccfc000 ffff830b172b4000
>> ffff83007ccfc000 ffff83007ccfc000
>> (XEN) [ 3669.218056]    0000000000113f00 0000000000000000
>> ffff830c5297fe38 ffff82d0801dee9e
>> (XEN) [ 3669.226116]    0000000000913f00 0000000000000000
>> 00007ff91cd34d60 ffff830b1b2b6980
>> (XEN) [ 3669.234177]    0000003b5297fe38 0000000113f007f8
>> 0000000000000296 0000000000000000
>> (XEN) [ 3669.242236]    ffff830b1b2b6980 0000000000000005
>> ffff82d0802081d1 ffff830c5297fef8
>> (XEN) [ 3669.250295]    ffff83007ccfc000 00000000000006ab
>> 000000000000001b 0000000113f007f8
>> (XEN) [ 3669.258354]    ffff830c5297fee8 ffff82d080202c00
>> ffff82d0802081d1 0000000000000080
>> (XEN) [ 3669.266417]    0000000000000000 0000000000000002
>> ffff830b172b4000 0000000000113f00
>> (XEN) [ 3669.274474]    00007ff91cd34d60 000000000000003b
>> ffff82d0802081d1 ffff82d0802081c5
>> (XEN) [ 3669.282537]    ffff82d0802081d1 ffff82d0802081c5
>> ffff82d0802081d1 ffff82d0802081c5
>> (XEN) [ 3669.290596]    ffff82d0802081d1 ffff83007ccfc000
>> 0000000000000000 0000000000000000
>> (XEN) [ 3669.298655]    0000000000000000 0000000000000000
>> 00007cf3ad6800e7 ffff82d08020820c
>> (XEN) [ 3669.306712]    00007ff91cd34d60 0000019285c42a50
>> 00000192858a8eb0 0000000000000000
>> (XEN) [ 3669.314772]    0000019285894438 41c64e6da3bd2845
>> 0000104000000000 00000fff239a69ac
>> (XEN) [ 3669.322832] Xen call trace:
>> (XEN) [ 3669.326128]    [<ffff82d08021006a>]
>> vmx_start_reexecute_instruction+0x107/0x68a
>> (XEN) [ 3669.333925]    [<ffff82d080210b3e>]
>> p2m_mem_access_check+0x551/0x64d
>> (XEN) [ 3669.340774]    [<ffff82d0801dee9e>]
>> hvm_hap_nested_page_fault+0x2f2/0x631
>> (XEN) [ 3669.348051]    [<ffff82d080202c00>]
>> vmx_vmexit_handler+0x156c/0x1e45
>> (XEN) [ 3669.354899]    [<ffff82d08020820c>]
>> vmx_asm_vmexit_handler+0xec/0x250
>> (XEN) [ 3669.361832]
>> (XEN) [ 3669.363827] *** Dumping CPU3 guest state (d1v0): ***
>> (XEN) [ 3669.369285] ----[ Xen-4.7.5  x86_64  debug=y  Not tainted ]----
>> (XEN) [ 3669.375700] CPU:    3
>> (XEN) [ 3669.378471] RIP:    0033:[<00007ff91cd34d60>]
>> (XEN) [ 3669.383323] RFLAGS: 0000000000010247   CONTEXT: hvm guest (d1v0)
>> (XEN) [ 3669.389824] rax: 00007ff91cd34d60   rbx: 41c64e6da3bd2845
>> rcx: 41c64e6da3bd2845
>> (XEN) [ 3669.397799] rdx: 0000000000000077   rsi: 0000000000000001
>> rdi: 0000019285877150
>> (XEN) [ 3669.405768] rbp: 0000019285894438   rsp: 0000008d6aa7f608   r8:
>>  0000000000000000
>> (XEN) [ 3669.413743] r9:  00000192858a8eb0   r10: 00000fff239a69ac
>> r11: 0000104000000000
>> (XEN) [ 3669.421716] r12: 0000000000000000   r13: 00000192858a8eb0
>> r14: 0000019285c42a50
>> (XEN) [ 3669.429690] r15: 00007ff91cd34d60   cr0: 0000000080050031
>> cr4: 0000000000170678
>> (XEN) [ 3669.437662] cr3: 0000000113f00002   cr2: 0000000000000000
>> (XEN) [ 3669.443555] fsb: 0000000000000000   gsb: 0000008d6a7cf000
>> gss: 0000002562d20000
>> (XEN) [ 3669.451529] ds: 002b   es: 002b   fs: 0053   gs: 002b   ss:
>> 002b   cs: 0033
>> (XEN) [ 3669.458980]
>> (XEN) [ 3669.463584] APIC error on CPU0: 40(00)
>>
>> Some scheduler magic appears to happen here where it is unclear why
>> is_running doesn't seem to end up being 0 as expected in our case. We'll
>> keep digging.
> 
> There seems to be some kind of deadlock between
> vmx_start_reexecute_instruction and hap_track_dirty_vram/handle_mmio.
> Are you holding a lock while trying to put the other vcpus to sleep?

d->arch.rexec_lock, but I don't see how that would matter in this case.


Thanks,
Razvan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel