[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] crash in csched_load_balance after xl vcpu-pin


  • To: Olaf Hering <olaf@xxxxxxxxx>, <xen-devel@xxxxxxxxxxxxx>
  • From: George Dunlap <george.dunlap@xxxxxxxxxx>
  • Date: Tue, 10 Apr 2018 16:29:40 +0100
  • Autocrypt: addr=george.dunlap@xxxxxxxxxx; prefer-encrypt=mutual; keydata= xsFNBFPqG+MBEACwPYTQpHepyshcufo0dVmqxDo917iWPslB8lauFxVf4WZtGvQSsKStHJSj 92Qkxp4CH2DwudI8qpVbnWCXsZxodDWac9c3PordLwz5/XL41LevEoM3NWRm5TNgJ3ckPA+J K5OfSK04QtmwSHFP3G/SXDJpGs+oDJgASta2AOl9vPV+t3xG6xyfa2NMGn9wmEvvVMD44Z7R W3RhZPn/NEZ5gaJhIUMgTChGwwWDOX0YPY19vcy5fT4bTIxvoZsLOkLSGoZb/jHIzkAAznug Q7PPeZJ1kXpbW9EHHaUHiCD9C87dMyty0N3TmWfp0VvBCaw32yFtM9jUgB7UVneoZUMUKeHA fgIXhJ7I7JFmw3J0PjGLxCLHf2Q5JOD8jeEXpdxugqF7B/fWYYmyIgwKutiGZeoPhl9c/7RE Bf6f9Qv4AtQoJwtLw6+5pDXsTD5q/GwhPjt7ohF7aQZTMMHhZuS52/izKhDzIufl6uiqUBge 0lqG+/ViLKwCkxHDREuSUTtfjRc9/AoAt2V2HOfgKORSCjFC1eI0+8UMxlfdq2z1AAchinU0 eSkRpX2An3CPEjgGFmu2Je4a/R/Kd6nGU8AFaE8ta0oq5BSFDRYdcKchw4TSxetkG6iUtqOO ZFS7VAdF00eqFJNQpi6IUQryhnrOByw+zSobqlOPUO7XC5fjnwARAQABzSRHZW9yZ2UgVy4g RHVubGFwIDxkdW5sYXBnQHVtaWNoLmVkdT7CwYAEEwEKACoCGwMFCwkIBwMFFQoJCAsFFgID AQACHgECF4ACGQEFAlpk2IEFCQo9I54ACgkQpjY8MQWQtG1A1BAAnc0oX3+M/jyv4j/ESJTO U2JhuWUWV6NFuzU10pUmMqpgQtiVEVU2QbCvTcZS1U/S6bqAUoiWQreDMSSgGH3a3BmRNi8n HKtarJqyK81aERM2HrjYkC1ZlRYG+jS8oWzzQrCQiTwn3eFLJrHjqowTbwahoiMw/nJ+OrZO /VXLfNeaxA5GF6emwgbpshwaUtESQ/MC5hFAFmUBZKAxp9CXG2ZhTP6ROV4fwhpnHaz8z+BT NQz8YwA4gkmFJbDUA9I0Cm9D/EZscrCGMeaVvcyldbMhWS+aH8nbqv6brhgbJEQS22eKCZDD J/ng5ea25QnS0fqu3bMrH39tDqeh7rVnt8Yu/YgOwc3XmgzmAhIDyzSinYEWJ1FkOVpIbGl9 uR6seRsfJmUK84KCScjkBhMKTOixWgNEQ/zTcLUsfTh6KQdLTn083Q5aFxWOIal2hiy9UyqR VQydowXy4Xx58rqvZjuYzdGDdAUlZ+D2O3Jp28ez5SikA/ZaaoGI9S1VWvQsQdzNfD2D+xfL qfd9yv7gko9eTJzv5zFr2MedtRb/nCrMTnvLkwNX4abB5+19JGneeRU4jy7yDYAhUXcI/waS /hHioT9MOjMh+DoLCgeZJYaOcgQdORY/IclLiLq4yFnG+4Ocft8igp79dbYYHkAkmC9te/2x Kq9nEd0Hg288EO/OwE0EVFq6vQEIAO2idItaUEplEemV2Q9mBA8YmtgckdLmaE0uzdDWL9To 1PL+qdNe7tBXKOfkKI7v32fe0nB4aecRlQJOZMWQRQ0+KLyXdJyHkq9221sHzcxsdcGs7X3c 17ep9zASq+wIYqAdZvr7pN9a3nVHZ4W7bzezuNDAvn4EpOf/o0RsWNyDlT6KECs1DuzOdRqD oOMJfYmtx9hMzqBoTdr6U20/KgnC/dmWWcJAUZXaAFp+3NYRCkk7k939VaUpoY519CeLrymd Vdke66KCiWBQXMkgtMGvGk5gLQLy4H3KXvpXoDrYKgysy7jeOccxI8owoiOdtbfM8TTDyWPR Ygjzb9LApA8AEQEAAcLBZQQYAQoADwUCVFq6vQIbDAUJAeEzgAAKCRCmNjwxBZC0bWknD/97 Tkh3PMAcvMZINmJefBdYYspmwTWZSR9USsy68oWzDsXKNDNTqBC781lR/7PSqhqaSOmSnty3 FNblaBYKfMV3OOWgrP0H8Voqp4IgH3yOOkQLVITIwulqbbxQtmCsJ3xkhZm6CA0EKbc9VM/j FX3aCAfOJf52vlY1gXjYOvVjrdrRrBXEjs8E5f6EsrQKDrWCKNx/9qRfmtsQeKHTsgpINkpZ s11ClX/sM/RCR9/BgB/K08QQZYsWD6lgZh1KxLXRzKRunba0L+jpcRsoQFUMj/ofrfnHAdl0 q2upzISM/wR8aer+kekMo+y00schmYJYu5JAAzbjQQuhCAg0UTBGPaNwteL2l3c9Ps8on1nl mq9TnbYwGLAxJzXSb3BATgz7dygpsBBNS5WhUNQgIJvcZJbLggEIqjZGs8o7/+dt4klwxCYL FVlsWYSwEjX0UYHVLMS/F7FcXbCMUeoN/4krmRyv7YICE/VDQSDPcSKedzWvQM8T+5uY5pFJ NiIaa6asFndP50GiKbFtD6xAM+rbnwT7Io+iPtvD/3ddMXQs58IVMzgNA/hcdOX/qlx6Jqk/ hYQQsl4HoQsx/GyrNiwiPErTx32QNeXxoGYm6kwxt7F5qK7AN5tyYNkEyoxYrv8bl9VjAve8 hpECyf4O1mOGC/dIuBCDk8gxL5Pbo3jl98LBZQQYAQoADwIbDAUCVlNqsQUJA9njdAAKCRCm NjwxBZC0bbJMEACigmtpL2lzS47DXydApr1X8SYCHIPc39OjvmErjP05lKUZjmesmhlM5eKO gPb/fzeJ0wXB4J8OyseIJ0D/XwyLLQeM8d/HUFFMBWr+HE7jIukAUXeQ6GRwR+MBYGK/KmR9 JHbMAUz8f3G087Ma12BfpNWayndlFwR3rvdV4lvlyx6cl0EaFhbzPu/N07HG5MTk0evtphgZ 7wuG1oAtO+DGA6orHEicor6nBAQNZzPyjqo40dBxTs+amx7UndMRPSL1dD57eJwbbvBeNa8I w8wT7oNy2/C21VWmSy5XzMzcUTgmjmQz6DSNJPz2dMK4Y/LtcVFTfSZTmlBIkfoc9Vay2EB9 3z2EmjZwGT7n/DRu9QDtLbXyeVTBuLTaP3D+q5AyR1/5Z4T0LhwNvxeND5yO+YNAwqocZwL+ OcctpSZUBpAuU4Ju/9JKMX57GlnbjB8YGahoBJsQZx4CZyw0MXlkCk5cR0EPjY9iI2CEA5lO QueOSbo0hf1ZJwCx724lx0WSwL8ngd8wZTYMNc8GngaU61kmzfcuCklhokTxQdK7Efme5ccv A1txzgGewx9mDhPgNcJweasBnyL0N3wya2RMAzm04gCio8y4FKQepwQpKCNKAYZIU4juAPxn nb6cbBGiMGO1NDuxG+qvl1cMElnq+cuhSUlZdr2sE9JRfa0gucLBZQQYAQoADwIbDAUCWHQN VAUJBfqGFwAKCRCmNjwxBZC0bbgCD/oC6mWUrxQKWPDvFE9+fzm8UKqKP7aciz+gvWUN3o4i 4sRFNyvAEOW/QY2zwM1pN07BFZ3Z+8AVxpgR6h7RQzDJYSPZ5k5WWCJzJEQs2sPI5rfYJGK8 um7mlsSvf2xcLK/1Aj07BmWDjR6glDDRY+iMmSSdHe6Te6tiQPPS6Woj8AE3qf5lBsdvcEln nrkSwzNeVKRQQROUOskVw4WmCsNJjZtKmrVpgId3df/5HWG7Bi4nPwA8IFOt6O72lJlkORFy DF5P7ML7Pc5LbEFimzETPBxTJzVu1UoOQb/THB+qxhKMXXudSf/5sdMhwvOwItIcc5pib/v6 7gWK48bAzoOTgNYzmDCVC/roeLLU2SpEQIlIR0eAaWImgt8VEtre3Gch33e41DtbUli54DX0 dRdhqQaDM1T1q77VyDoZcs+SpGX9Ic9mxl+BN+6vtGIUVgaOG5pF85aQlRfCD6IlFQgiZtiR XeRpeIYG27RUw5kIljW+VxPMdBUvZpUXEazqjoPvBKybg0oKFfMXrMj4vHo6J0FD3ZEToGnP dANspUCZRewRozjp7ZWIu7QfGasfJNQ8c1IDiAFl3rV+dAGXXdmrDcX6w2q5lqoFz+8npK2I ehKCA94U+J/RLywUiaLuHnXt40WvQ98kHm7uTsy36iWqqawPqzmn8m5ruynVHmmcXsLBZQQY AQoADwIbDAUCWmTXMwUJB+tP9gAKCRCmNjwxBZC0bb+2D/9hjn1k5WcRHlu19WGuH6q0Kgm1 LRT7PnnSz904igHNElMB5a7wRjw5kdNwU3sRm2nnmHeOJH8kYj2Hn1QgX5SqQsysWTHWOEse GeoXydx9zZZkt3oQJM+9NV1VjK0bOXwqhiQyEUWz5/9l467FS/k4FJ5CHNRumvhLa0l2HEEu 5pxq463HQZHDt4YE/9Y74eXOnYCB4nrYxQD/GSXEZvWryEWreDoaFqzq1TKtzHhFgQG7yFUE epxLRUUtYsEpT6Rks2l4LCqG3hVD0URFIiTyuxJx3VC2Ta4LH3hxQtiaIpuXqq2D4z63h6vC x2wxfZc/WRHGbr4NAlB81l35Q/UHyMocVuYLj0llF0rwU4AjiKZ5qWNSEdvEpL43fTvZYxQh DCjQTKbb38omu5P4kOf1HT7s+kmQKRtiLBlqHzK17D4K/180ADw7a3gnmr5RumcZP3NGSSZA 6jP5vNqQpNu4gqrPFWNQKQcW8HBiYFgq6SoLQQWbRxJDHvTRYJ2ms7oCe870gh4D1wFFqTLe yXiVqjddENGNaP8ZlCDw6EU82N8Bn5LXKjR1GWo2UK3CjrkHpTt3YYZvrhS2MO2EYEcWjyu6 LALF/lS6z6LKeQZ+t9AdQUcILlrx9IxqXv6GvAoBLJY1jjGBq+/kRPrWXpoaQn7FXWGfMqU+ NkY9enyrlw==
  • Cc: Dario Faggioli <dfaggioli@xxxxxxxx>
  • Delivery-date: Tue, 10 Apr 2018 15:29:48 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Openpgp: preference=signencrypt

On 04/10/2018 04:18 PM, Olaf Hering wrote:
> On Tue, Apr 10, Olaf Hering wrote:
> 
>> (XEN) Xen BUG at sched_credit.c:1694
> 
> Another variant:
> 
> This time the domUs had just vcpus=36 and 
> cpus=nodes:N,node:^0/cpus_soft=nodes:N,node:^0
> 
> (XEN) Xen BUG at sched_credit.c:280
> (XEN) ----[ Xen-4.11.20180407T144959.e62e140daa-2.bug1087289_411  x86_64  
> debug=n   Not tainted ]----
> (XEN) CPU:    54
> (XEN) RIP:    e008:[<ffff82d0803591b1>] 
> sched_credit.c#__runq_insert.part.13+0/0x2
> (XEN) RFLAGS: 0000000000010087   CONTEXT: hypervisor (d96v20)
> (XEN) rax: ffff82d08095f100   rbx: ffff830670506ea0   rcx: ffff830779f4ae80
> (XEN) rdx: 00000036f95d7080   rsi: 0000000000000000   rdi: ffff830670506ea0
> (XEN) rbp: ffff82d08094a480   rsp: ffff830e7ab2fd30   r8:  ffff830779f361a0
> (XEN) r9:  ffff82d080227cf0   r10: 0000000000000000   r11: 0000000000000000
> (XEN) r12: 0000033c2684bb20   r13: ffff830779f4ae80   r14: ffff830779f36180
> (XEN) r15: 0000033c269c6f66   cr0: 000000008005003b   cr4: 00000000001526e0
> (XEN) cr3: 000000067058e000   cr2: 00007f1299b17000
> (XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> (XEN) Xen code around <ffff82d0803591b1> 
> (sched_credit.c#__runq_insert.part.13):
> (XEN)  f1 ff 5a 5b 31 c0 5d c3 <0f> 0b 0f 0b 0f 0b 48 89 e2 48 8d 05 eb 5d 60 
> 00
> (XEN) Xen stack trace from rsp=ffff830e7ab2fd30:
> (XEN)    ffff82d080228845 ffff82e030ac7f80 00000036563fc000 00000000ffffffff
> (XEN)    00000000000000a3 00000000000000c0 ffff83077a6c59e0 ffff830e7ab2fe70
> (XEN)    ffff82d0802354b5 ffff82d0802fff50 0000000000000000 0000000001c9c380
> (XEN)    000000008027bcd8 ffff82d0802255d0 0000000000000036 0000033c269c6f66
> (XEN)    ffff8307798d4f30 0000000000000000 ffff830779f361a0 0000000000000036
> (XEN)    ffff82d0802386cc ffff830779f361a0 0000000000000046 ffff82d08023827b
> (XEN)    0000000000000096 0000000000000036 ffff830779f361c8 ffff82d08030f9ab
> (XEN)    0000000000000036 ffff83007ba30000 ffff830779f36188 0000033c269c6f66
> (XEN)    ffff830779f36180 ffff82d08094a480 ffff82d08023153d ffff82d000000000
> (XEN)    ffff830779f361a0 0000000000000000 ffff82d0802e13d5 ffff83007ba30000
> (XEN)    ffff83007ba30000 0000000000000000 ffff82d08030bef6 ffff82d08030f9ab
> (XEN)    00000000ffffffff ffffffffffffffff ffff830e7ab2ffff ffff82d080933c00
> (XEN)    0000000000000000 0000000000000000 ffff82d080234cb2 0000000000000000
> (XEN)    ffff83007ba30000 0000000000000000 0000000000000000 0000000000000000
> (XEN)    ffff82d08030fb6b 0000000000000000 0000000000000100 0000000000540000
> (XEN)    0000000000000001 ffff88011ff16c80 ffff8800e1e20000 0000000000000000
> (XEN)    ffff88011f000858 ffff88011f0006c8 0000000000000000 0000000000000000
> (XEN)    0000000000000001 0000000000000001 00000000000000ad 00000000000000a5
> (XEN)    000000fb00000000 ffffffff810c8da3 0000000000000000 0000000000000046
> (XEN)    ffff8800ea3af910 0000000000000000 0000000000000000 0000000000000000
> (XEN) Xen call trace:
> (XEN)    [<ffff82d0803591b1>] sched_credit.c#__runq_insert.part.13+0/0x2
> (XEN)    [<ffff82d080228845>] sched_credit.c#csched_schedule+0xb55/0xba0
> (XEN)    [<ffff82d0802354b5>] smp_call_function_interrupt+0x85/0xa0
> (XEN)    [<ffff82d0802fff50>] vmcs.c#__vmx_clear_vmcs+0/0xe0
> (XEN)    [<ffff82d0802255d0>] sched_credit.c#csched_vcpu_yield+0/0x10
> (XEN)    [<ffff82d0802386cc>] timer.c#remove_entry+0x7c/0x90
> (XEN)    [<ffff82d08023827b>] timer.c#add_entry+0x4b/0xb0
> (XEN)    [<ffff82d08030f9ab>] vmx_asm_vmexit_handler+0xab/0x240
> (XEN)    [<ffff82d08023153d>] schedule.c#schedule+0xdd/0x5d0
> (XEN)    [<ffff82d0802e13d5>] hvm_interrupt_blocked+0x15/0xd0
> (XEN)    [<ffff82d08030bef6>] nvmx_switch_guest+0x86/0x1a00
> (XEN)    [<ffff82d08030f9ab>] vmx_asm_vmexit_handler+0xab/0x240
> (XEN)    [<ffff82d080234cb2>] softirq.c#__do_softirq+0x62/0x90
> (XEN)    [<ffff82d08030fb6b>] vmx_asm_do_vmentry+0x2b/0x30
> (XEN) ****************************************
> (XEN) Panic on CPU 54:
> (XEN) Xen BUG at sched_credit.c:280
> (XEN) ****************************************
> (XEN) Reboot in five seconds...

Ooh:

    BUG_ON( __vcpu_on_runq(svc) );

So we're trying to insert a vcpu onto a runqueue, but someone's already
put it on a runqueue.  Which still doesn't quite make sense...

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.