[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] task xenwatch blocked for more than 120 seconds.



I am seeing this problem on two of our test machines. After seeing this
problem, I couldn't start any VM on dom0. Following details the problem:

Kernel: 

3.18.44-20.el6.x86_64

Xen:

xen-libs-4.6.3-5.el6.x86_64
xen-licenses-4.6.3-5.el6.x86_64
xen-hypervisor-4.6.3-5.el6.x86_64
xen-4.6.3-5.el6.x86_64
xen-runtime-4.6.3-5.el6.x86_64

Call Trace:

Mar  8 15:12:25 b1-aesme1 kernel:      Not tainted 3.18.44-20.el6.x86_64 #1
Mar  8 15:12:25 b1-aesme1 kernel: "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar  8 15:12:25 b1-aesme1 kernel: xenwatch        D ffff880035913340     0  
178      2 0x00000000
Mar  8 15:12:25 b1-aesme1 kernel: ffff88002f243d58 0000000000000246
ffff88002f281a20 ffff88002f240010
Mar  8 15:12:25 b1-aesme1 kernel: 0000000000013340 0000000000013340
ffff88002f96b090 ffff88002ff46050
Mar  8 15:12:25 b1-aesme1 kernel: 0000000000000001 0000000000000000
ffff88002f96b090 0000000000000016
Mar  8 15:12:25 b1-aesme1 kernel: Call Trace:
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff81676be9>] schedule+0x29/0x70
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff81074279>]
cpu_hotplug_begin+0x89/0x90
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff8166eb52>] _cpu_down+0x92/0x2a0
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff813baf70>] ?
xenbus_transaction_start+0x60/0x60
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff8166ed96>] cpu_down+0x36/0x50
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff813b2e67>]
vcpu_hotplug+0x87/0xc0
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff813b2ee1>]
handle_vcpu_hotplug_event+0x41/0x50
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff813bb013>]
xenwatch_thread+0xa3/0x140
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff810b2da0>] ?
woken_wake_function+0x20/0x20
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff813baf70>] ?
xenbus_transaction_start+0x60/0x60
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff8109190e>] kthread+0xce/0xf0
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff81091840>] ?
kthread_freezable_should_stop+0x70/0x70
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff8167a858>]
ret_from_fork+0x58/0x90
Mar  8 15:12:25 b1-aesme1 kernel: [<ffffffff81091840>] ?
kthread_freezable_should_stop+0x70/0x70
Mar  8 15:14:25 b1-aesme1 kernel: INFO: task xenwatch:178 blocked for more
than 120 seconds.
Mar  8 15:14:25 b1-aesme1 kernel:      Not tainted 3.18.44-20.sp1.el6.x86_64
#1

# xl info
host                   : b1-aesme1.dr.avaya.com
release                : 3.18.44-20.sp1.el6.x86_64
version                : #1 SMP Wed Jan 25 08:19:52 MST 2017
machine                : x86_64
nr_cpus                : 24
max_cpu_id             : 31
nr_nodes               : 2
cores_per_socket       : 6
threads_per_core       : 2
cpu_mhz                : 2933
hw_caps                :
bfebfbff:2c100800:00000000:00003f00:029ee3ff:00000000:00000001:00000000
virt_caps              : hvm
total_memory           : 49141
free_memory            : 47739
sharing_freed_memory   : 0
sharing_used_memory    : 0
outstanding_claims     : 0
free_cpus              : 0
xen_major              : 4
xen_minor              : 6
xen_extra              : .3-5.el6
xen_version            : 4.6.3-5.el6
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : Sun Dec 18 15:42:59 2016 +0000 git:0abbdb3-dirty
xen_commandline        : dom0_mem=max:864M com1=9600,8n1 console=vga,com1
cc_compiler            : gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-17)
cc_compile_by          : mockbuild
cc_compile_domain      : centos.org
cc_compile_date        : Fri Dec 23 17:29:39 UTC 2016
xend_config_format     : 4

# xl vcpu-list 0
Name                                ID  VCPU   CPU State   Time(s) Affinity
(Hard / Soft)
Domain-0                             0     0   10   -b-     266.8  0-11 /
all
Domain-0                             0     1    4   -b-      67.5  0-11 /
all
Domain-0                             0     2    3   -b-     208.6  0-11 /
all
Domain-0                             0     3    6   r--      65.3  0-11 /
all
Domain-0                             0     4    0   -b-     115.6  0-11 /
all
Domain-0                             0     5    8   -b-      79.7  0-11 /
all
Domain-0                             0     6    2   -b-     144.5  0-11 /
all
Domain-0                             0     7   11   -b-      72.3  0-11 /
all
Domain-0                             0     8    1   -b-     202.1  0-11 /
all
Domain-0                             0     9    5   -b-      66.0  0-11 /
all
Domain-0                             0    10    7   -b-     395.9  0-11 /
all
Domain-0                             0    11    9   -b-      67.7  0-11 /
all
Domain-0                             0    12    -   --p       1.1  all / all
Domain-0                             0    13    -   --p       0.7  all / all
Domain-0                             0    14    -   --p       1.2  all / all
Domain-0                             0    15    -   --p       0.8  all / all
Domain-0                             0    16    -   --p       1.1  all / all
Domain-0                             0    17    -   --p       1.5  all / all
Domain-0                             0    18    -   --p       1.0  all / all
Domain-0                             0    19    -   --p       0.9  all / all
Domain-0                             0    20    -   --p       1.1  all / all
Domain-0                             0    21    -   --p       1.3  all / all
Domain-0                             0    22   22   -b-     404.7  all / all
Domain-0                             0    23   13   -b-     378.1  all / all

# xl list
Name                                        ID   Mem VCPUs      State  
Time(s)
Domain-0                                     0   864    14     r-----   
2546.5


Let me know if you need more information from my side.  If upgrading to
kernel 4 isn't possible, what else I can do for this problem?

Thanks,

-hxiao



--
View this message in context: 
http://xen.1045712.n5.nabble.com/task-xenwatch-blocked-for-more-than-120-seconds-tp5735392p5735924.html
Sent from the Xen - User mailing list archive at Nabble.com.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
https://lists.xen.org/xen-users

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.