[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] Xen3.3 / Xen3.4 CPU soft lockups under pvops 2.6.31/2.6.32



On Fri, Apr 09, 2010 at 04:34:08PM +0200, Pim van Riezen wrote:
> 
> On Apr 9, 2010, at 16:21 , Pasi Kärkkäinen wrote:
> 
> > Can you please post the soft lockup messages again..
> 
> Here's a metric truckload of them. The lockups don't seem to happen 
> consistently in the same code path. It may also be worth mentioning that it 
> only seems to happen with guests that have more than 1 vcpu, but I'm not sure 
> of that distinction.
>

I guess you already posted these to xen-devel ? 

-- Pasi

 
> Apr  9 08:21:01 host kernel: BUG: soft lockup - CPU#1 stuck for 628s! 
> [gzip:27658]
> Apr  9 08:21:01 host kernel: Modules linked in:
> Apr  9 08:21:01 host kernel: CPU 1:
> Apr  9 08:21:01 host kernel: Modules linked in:
> Apr  9 08:21:01 host kernel: Pid: 27658, comm: gzip Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 08:21:01 host kernel: RIP: e033:[<0000000000401958>]  
> [<0000000000401958>] 0x401958
> Apr  9 08:21:01 host kernel: RSP: e02b:00007fff026d7e38  EFLAGS: 00000297
> Apr  9 08:21:01 host kernel: RAX: 0000000000000020 RBX: 0000000000000002 RCX: 
> 0000000000657ce8
> Apr  9 08:21:01 host kernel: RDX: 0000000000000048 RSI: 0000000000000002 RDI: 
> 000000000000da48
> Apr  9 08:21:01 host kernel: RBP: 0000000000006a6c R08: 0000000000000034 R09: 
> 000000000000da48
> Apr  9 08:21:01 host kernel: R10: 0000000000000048 R11: 0000000000000080 R12: 
> 0000000000658c06
> Apr  9 08:21:01 host kernel: R13: 0000000000658d08 R14: 0000000000658c06 R15: 
> 0000000000000080
> Apr  9 08:21:01 host kernel: FS:  00007fa6088946e0(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 08:21:01 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 08:21:01 host kernel: CR2: 00007f7addede020 CR3: 00000000267db000 CR4: 
> 0000000000002660
> Apr  9 08:21:01 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 08:21:02 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 08:21:02 host kernel: Call Trace:
> Apr  9 08:29:39 host kernel: BUG: soft lockup - CPU#1 stuck for 483s! 
> [swapper:0]
> Apr  9 08:29:39 host kernel: Modules linked in:
> Apr  9 08:29:39 host kernel: CPU 1:
> Apr  9 08:29:39 host kernel: Modules linked in:
> Apr  9 08:29:39 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 08:29:39 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 08:29:39 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 08:29:39 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 08:29:39 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 08:29:39 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 08:29:39 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 08:29:39 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 08:29:39 host kernel: FS:  00007fa913f42720(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 08:29:39 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 08:29:39 host kernel: CR2: 00007ff39862b1f0 CR3: 0000000052c5e000 CR4: 
> 0000000000002660
> Apr  9 08:29:39 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 08:29:39 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 08:29:39 host kernel: Call Trace:
> Apr  9 08:29:39 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 08:29:39 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 08:29:39 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 08:29:39 host kernel: BUG: soft lockup - CPU#0 stuck for 483s! 
> [swapper:0]
> Apr  9 08:29:39 host kernel: Modules linked in:
> Apr  9 08:29:39 host kernel: CPU 0:
> Apr  9 08:29:40 host kernel: Modules linked in:
> Apr  9 08:29:40 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 08:29:40 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 08:29:40 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 08:29:40 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 08:29:40 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 08:29:40 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 08:29:40 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 08:29:40 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 08:29:40 host kernel: FS:  00007f07704fe6e0(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 08:29:40 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 08:29:40 host kernel: CR2: 00007ff39e4a8000 CR3: 000000007ea99000 CR4: 
> 0000000000002660
> Apr  9 08:29:40 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 08:29:40 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 08:29:40 host kernel: Call Trace:
> Apr  9 08:29:40 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 08:29:40 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 08:29:40 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 08:29:40 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 08:34:34 host kernel: BUG: soft lockup - CPU#1 stuck for 275s! 
> [ssh:2324]
> Apr  9 08:34:34 host kernel: Modules linked in:
> Apr  9 08:34:34 host kernel: CPU 1:
> Apr  9 08:34:34 host kernel: Modules linked in:
> Apr  9 08:34:34 host kernel: Pid: 2324, comm: ssh Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 08:34:34 host kernel: RIP: e033:[<00007f1b0fcac76a>]  
> [<00007f1b0fcac76a>] 0x7f1b0fcac76a
> Apr  9 08:34:34 host kernel: RSP: e02b:00007fffa45a7480  EFLAGS: 00010202
> Apr  9 08:34:34 host kernel: RAX: 0000000000000000 RBX: 0000000000000000 RCX: 
> 0000000000000181
> Apr  9 08:34:34 host kernel: RDX: 00007f1b0fcdca28 RSI: 0000000000000013 RDI: 
> 000000000000000a
> Apr  9 08:34:34 host kernel: RBP: 0000000000000010 R08: 0000000000000004 R09: 
> 0101010101010101
> Apr  9 08:34:34 host kernel: R10: 737361702e707466 R11: 00007f1b0fcac690 R12: 
> 0000000000000010
> Apr  9 08:34:34 host kernel: R13: 00007f1b11f825e0 R14: 0000000000000001 R15: 
> 0000000000000004
> Apr  9 08:34:34 host kernel: FS:  00007f1b10528710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 08:34:34 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 08:34:34 host kernel: CR2: 00007fa913f54000 CR3: 000000006ad44000 CR4: 
> 0000000000002660
> Apr  9 08:34:34 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 08:34:34 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 08:34:34 host kernel: Call Trace:
> Apr  9 08:34:34 host kernel: BUG: soft lockup - CPU#0 stuck for 275s! 
> [swapper:0]
> Apr  9 08:34:34 host kernel: Modules linked in:
> Apr  9 08:34:34 host kernel: CPU 0:
> Apr  9 08:34:34 host kernel: Modules linked in:
> Apr  9 08:34:34 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 08:34:34 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 08:34:34 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 08:34:34 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 08:34:34 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 08:34:34 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 08:34:34 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 08:34:34 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 08:34:34 host kernel: FS:  00007fa913f42720(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 08:34:34 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 08:34:34 host kernel: CR2: 00007fa913f54000 CR3: 000000006acb1000 CR4: 
> 0000000000002660
> Apr  9 08:34:34 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 08:34:34 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 08:34:34 host kernel: Call Trace:
> Apr  9 08:34:34 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 08:34:34 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 08:34:34 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 08:34:34 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 08:39:49 host kernel: BUG: soft lockup - CPU#1 stuck for 293s! 
> [gzip:30581]
> Apr  9 08:39:49 host kernel: Modules linked in:
> Apr  9 08:39:49 host kernel: CPU 1:
> Apr  9 08:39:49 host kernel: Modules linked in:
> Apr  9 08:39:49 host kernel: Pid: 30581, comm: gzip Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 08:39:49 host kernel: RIP: e030:[<ffffffff811ede38>]  
> [<ffffffff811ede38>] fuse_request_send+0x60/0x22b
> Apr  9 08:39:49 host kernel: RSP: e02b:ffff88007952bc28  EFLAGS: 00000202
> Apr  9 08:39:49 host kernel: RAX: ffff880079652df0 RBX: ffff88007e07d2c0 RCX: 
> fffffffffffdfb5e
> Apr  9 08:39:49 host kernel: RDX: 0000000000020001 RSI: 000000000000001d RDI: 
> ffff88007eb1f710
> Apr  9 08:39:49 host kernel: RBP: ffff88007e07d2c0 R08: 000000037ede1c40 R09: 
> ffff8800593170d8
> Apr  9 08:39:49 host kernel: R10: 0000000000000000 R11: 000000000000000f R12: 
> ffff88007eb1f400
> Apr  9 08:39:49 host kernel: R13: ffff88007e07d2e0 R14: 0000000000001000 R15: 
> 0000000000001000
> Apr  9 08:39:49 host kernel: FS:  00007f7b12f606e0(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 08:39:49 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 08:39:49 host kernel: CR2: 00000000006d9138 CR3: 00000000264f7000 CR4: 
> 0000000000002660
> Apr  9 08:39:49 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 08:39:49 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 08:39:49 host kernel: Call Trace:
> Apr  9 08:39:49 host kernel:  [<ffffffff811ede34>] ? 
> fuse_request_send+0x5c/0x22b
> Apr  9 08:39:49 host kernel:  [<ffffffff811f1655>] ? 
> fuse_wait_on_page_writeback+0x31/0x168
> Apr  9 08:39:49 host kernel:  [<ffffffff810904b4>] ? 
> add_to_page_cache_lru+0x41/0x58
> Apr  9 08:39:49 host kernel:  [<ffffffff811f186a>] ? fuse_send_write+0xb8/0xc2
> Apr  9 08:39:49 host kernel:  [<ffffffff811f29d7>] ? 
> fuse_file_aio_write+0x497/0x62f
> Apr  9 08:39:49 host kernel:  [<ffffffff810bab7e>] ? do_sync_write+0xce/0x113
> Apr  9 08:39:49 host kernel:  [<ffffffff8100f6af>] ? __switch_to+0x1ad/0x297
> Apr  9 08:39:49 host kernel:  [<ffffffff8100e3b2>] ? check_events+0x12/0x20
> Apr  9 08:39:49 host kernel:  [<ffffffff810561b8>] ? 
> autoremove_wake_function+0x0/0x2e
> Apr  9 08:39:49 host kernel:  [<ffffffff8103b070>] ? 
> finish_task_switch+0x27/0x6e
> Apr  9 08:39:49 host kernel:  [<ffffffff814d6a77>] ? thread_return+0x3e/0x109
> Apr  9 08:39:49 host kernel:  [<ffffffff810bb4b2>] ? vfs_write+0xad/0x149
> Apr  9 08:39:49 host kernel:  [<ffffffff810bb60a>] ? sys_write+0x45/0x6e
> Apr  9 08:39:49 host kernel:  [<ffffffff81010ac2>] ? 
> system_call_fastpath+0x16/0x1b
> Apr  9 08:39:49 host kernel: BUG: soft lockup - CPU#0 stuck for 293s! 
> [sshfs:2332]
> Apr  9 08:39:49 host kernel: Modules linked in:
> Apr  9 08:39:49 host kernel: CPU 0:
> Apr  9 08:39:49 host kernel: Modules linked in:
> Apr  9 08:39:49 host kernel: Pid: 2332, comm: sshfs Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 08:39:49 host kernel: RIP: e030:[<ffffffff813b93bd>]  
> [<ffffffff813b93bd>] sk_reset_timer+0x1/0x1d
> Apr  9 08:39:49 host kernel: RSP: e02b:ffff8800022afbc0  EFLAGS: 00000206
> Apr  9 08:39:49 host kernel: RAX: 0000000101b312c6 RBX: 0000000000000080 RCX: 
> 0000000101b312ce
> Apr  9 08:39:49 host kernel: RDX: 0000000101b312ce RSI: ffff88007d1dc4d0 RDI: 
> ffff88007d1dc1c0
> Apr  9 08:39:49 host kernel: RBP: ffff88007d1dc1c0 R08: 000000007d1dc10f R09: 
> 0000000101b312ce
> Apr  9 08:39:49 host kernel: R10: ffff880068b8f1c0 R11: ffffffff8126167a R12: 
> ffff880068b8f1c0
> Apr  9 08:39:49 host kernel: R13: 0000000000000000 R14: 0000000000000020 R15: 
> ffff880068b8f1f8
> Apr  9 08:39:49 host kernel: FS:  00007f1b10528710(0063) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 08:39:49 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 08:39:49 host kernel: CR2: 00007f66a4ede000 CR3: 000000006acb1000 CR4: 
> 0000000000002660
> Apr  9 08:39:49 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 08:39:49 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 08:39:49 host kernel: Call Trace:
> Apr  9 08:39:49 host kernel:  <IRQ>  [<ffffffff81419140>] ? 
> tcp_rcv_established+0x587/0x87b
> Apr  9 08:39:49 host kernel:  [<ffffffff8141f7f1>] ? tcp_v4_do_rcv+0x2c/0x1c9
> Apr  9 08:39:49 host kernel:  [<ffffffff81046156>] ? local_bh_enable+0x22/0x8f
> Apr  9 08:39:49 host kernel:  [<ffffffff8141fe05>] ? tcp_v4_rcv+0x477/0x6f8
> Apr  9 08:39:49 host kernel:  [<ffffffff814064f7>] ? 
> ip_local_deliver_finish+0xf2/0x1c7
> Apr  9 08:39:49 host kernel:  [<ffffffff81406126>] ? ip_rcv_finish+0x326/0x340
> Apr  9 08:39:49 host kernel:  [<ffffffff814063c0>] ? ip_rcv+0x280/0x2c5
> Apr  9 08:39:49 host kernel:  [<ffffffff8135d39d>] ? xennet_poll+0xb06/0xc68
> Apr  9 08:39:49 host kernel:  [<ffffffff8100e39f>] ? 
> xen_restore_fl_direct_end+0x0/0x1
> Apr  9 08:39:49 host kernel:  [<ffffffff81334afc>] ? 
> do_blkif_request+0x341/0x350
> Apr  9 08:39:49 host kernel:  [<ffffffff8100e3b2>] ? check_events+0x12/0x20
> Apr  9 08:39:49 host kernel:  [<ffffffff813c469b>] ? net_rx_action+0x77/0x179
> Apr  9 08:39:49 host kernel:  [<ffffffff810462b4>] ? __do_softirq+0xdd/0x1a1
> Apr  9 08:39:49 host kernel:  [<ffffffff81011bec>] ? call_softirq+0x1c/0x30
> Apr  9 08:39:49 host kernel:  [<ffffffff81012edf>] ? do_softirq+0x3f/0x7c
> Apr  9 08:39:49 host kernel:  [<ffffffff812ff975>] ? 
> xen_evtchn_do_upcall+0x173/0x190
> Apr  9 08:39:49 host kernel:  [<ffffffff81011c3e>] ? 
> xen_do_hypervisor_callback+0x1e/0x30
> Apr  9 08:39:49 host kernel:  <EOI>  [<ffffffff8100e3b2>] ? 
> check_events+0x12/0x20
> Apr  9 08:39:49 host kernel:  [<ffffffff8100922a>] ? 
> hypercall_page+0x22a/0x1001
> Apr  9 08:39:49 host kernel:  [<ffffffff8100922a>] ? 
> hypercall_page+0x22a/0x1001
> Apr  9 08:39:49 host kernel:  [<ffffffff8100dd39>] ? 
> xen_force_evtchn_callback+0x9/0xa
> Apr  9 08:39:49 host kernel:  [<ffffffff8100e3b2>] ? check_events+0x12/0x20
> Apr  9 08:39:49 host kernel:  [<ffffffff8100e3b2>] ? check_events+0x12/0x20
> Apr  9 08:39:49 host kernel:  [<ffffffff8100e359>] ? 
> xen_irq_enable_direct_end+0x0/0x7
> Apr  9 08:39:49 host kernel:  [<ffffffff8103b070>] ? 
> finish_task_switch+0x27/0x6e
> Apr  9 08:39:49 host kernel:  [<ffffffff814d6a77>] ? thread_return+0x3e/0x109
> Apr  9 08:39:49 host kernel:  [<ffffffff8103d336>] ? 
> default_wake_function+0x0/0x9
> Apr  9 08:39:49 host kernel:  [<ffffffff81261668>] ? 
> cap_socket_recvmsg+0x0/0x3
> Apr  9 08:39:49 host kernel:  [<ffffffff81261668>] ? 
> cap_socket_recvmsg+0x0/0x3
> Apr  9 08:39:49 host kernel:  [<ffffffff814d6e59>] ? 
> schedule_timeout+0x23/0x1d9
> Apr  9 08:39:49 host kernel:  [<ffffffff8100dd39>] ? 
> xen_force_evtchn_callback+0x9/0xa
> Apr  9 08:39:49 host kernel:  [<ffffffff8100e3b2>] ? check_events+0x12/0x20
> Apr  9 08:39:49 host kernel:  [<ffffffff81261668>] ? 
> cap_socket_recvmsg+0x0/0x3
> Apr  9 08:39:49 host kernel:  [<ffffffff814d824c>] ? 
> _spin_unlock_irqrestore+0xc/0xd
> Apr  9 08:39:49 host kernel:  [<ffffffff81456ebe>] ? 
> unix_stream_recvmsg+0x2c4/0x644
> Apr  9 08:39:49 host kernel:  [<ffffffff810561b8>] ? 
> autoremove_wake_function+0x0/0x2e
> Apr  9 08:39:49 host kernel:  [<ffffffff81457231>] ? 
> unix_stream_recvmsg+0x637/0x644
> Apr  9 08:39:49 host kernel:  [<ffffffff813b6303>] ? sock_aio_read+0x103/0x117
> Apr  9 08:39:49 host kernel:  [<ffffffff810bac91>] ? do_sync_read+0xce/0x113
> Apr  9 08:39:49 host kernel:  [<ffffffff81011c1b>] ? 
> xen_hypervisor_callback+0x1b/0x20
> Apr  9 08:39:49 host kernel:  [<ffffffff810561b8>] ? 
> autoremove_wake_function+0x0/0x2e
> Apr  9 08:39:49 host kernel:  [<ffffffff81261668>] ? 
> cap_socket_recvmsg+0x0/0x3
> Apr  9 08:39:49 host kernel:  [<ffffffff814d824c>] ? 
> _spin_unlock_irqrestore+0xc/0xd
> Apr  9 08:39:49 host kernel:  [<ffffffff81011c1b>] ? 
> xen_hypervisor_callback+0x1b/0x20
> Apr  9 08:39:49 host kernel:  [<ffffffff810bb6f0>] ? vfs_read+0xbd/0x146
> Apr  9 08:39:49 host kernel:  [<ffffffff8100922a>] ? 
> hypercall_page+0x22a/0x1001
> Apr  9 08:39:49 host kernel:  [<ffffffff810bb835>] ? sys_read+0x45/0x6e
> Apr  9 08:39:49 host kernel:  [<ffffffff81010ac2>] ? 
> system_call_fastpath+0x16/0x1b
> Apr  9 09:11:36 host kernel: BUG: soft lockup - CPU#1 stuck for 1776s! 
> [swapper:0]
> Apr  9 09:11:36 host kernel: Modules linked in:
> Apr  9 09:11:36 host kernel: CPU 1:
> Apr  9 09:11:36 host kernel: Modules linked in:
> Apr  9 09:11:36 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 09:11:36 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 09:11:36 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 09:11:36 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 09:11:36 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 09:11:36 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 09:11:36 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 09:11:36 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 09:11:36 host kernel: FS:  00007f44bc51f710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 09:11:36 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 09:11:36 host kernel: CR2: 00007f44bc52c000 CR3: 0000000071c86000 CR4: 
> 0000000000002660
> Apr  9 09:11:36 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 09:11:36 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 09:11:36 host kernel: Call Trace:
> Apr  9 09:11:36 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 09:11:36 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 09:11:36 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 09:11:36 host kernel: BUG: soft lockup - CPU#0 stuck for 1776s! 
> [swapper:0]
> Apr  9 09:11:36 host kernel: Modules linked in:
> Apr  9 09:11:36 host kernel: CPU 0:
> Apr  9 09:11:36 host kernel: Modules linked in:
> Apr  9 09:11:36 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 09:11:36 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 09:11:36 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 09:11:36 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 09:11:36 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 09:11:36 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 09:11:36 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 09:11:36 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 09:11:36 host kernel: FS:  00007f07704fe6e0(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 09:11:36 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 09:11:36 host kernel: CR2: 00000000004884b2 CR3: 0000000071c86000 CR4: 
> 0000000000002660
> Apr  9 09:11:36 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 09:11:36 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 09:11:36 host kernel: Call Trace:
> Apr  9 09:11:36 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 09:11:36 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 09:11:36 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 09:11:36 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 09:20:24 host kernel: BUG: soft lockup - CPU#1 stuck for 492s! 
> [ssh:2324]
> Apr  9 09:20:24 host kernel: Modules linked in:
> Apr  9 09:20:24 host kernel: CPU 1:
> Apr  9 09:20:24 host kernel: Modules linked in:
> Apr  9 09:20:24 host kernel: Pid: 2324, comm: ssh Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 09:20:24 host kernel: RIP: e033:[<00007f1b0e259f20>]  
> [<00007f1b0e259f20>] 0x7f1b0e259f20
> Apr  9 09:20:24 host kernel: RSP: e02b:00007fffa45a7338  EFLAGS: 00000206
> Apr  9 09:20:24 host kernel: RAX: 00007f1b11f32b20 RBX: 00007f1b0e5399c0 RCX: 
> 00000000000000a0
> Apr  9 09:20:24 host kernel: RDX: 00007f1b11f32b80 RSI: 0000000000000004 RDI: 
> 00007f1b0e539c30
> Apr  9 09:20:24 host kernel: RBP: 0000000000000064 R08: 0000000000000001 R09: 
> 0000000000000001
> Apr  9 09:20:24 host kernel: R10: 00007f1b0e539a30 R11: 00007f1b0e25bfb0 R12: 
> 0000000000000000
> Apr  9 09:20:24 host kernel: R13: 00007f1b0fcd2aec R14: 0000000000000111 R15: 
> 0000000000000071
> Apr  9 09:20:24 host kernel: FS:  00007f1b10528710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 09:20:24 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 09:20:24 host kernel: CR2: 00007fa913f53000 CR3: 000000006ad44000 CR4: 
> 0000000000002660
> Apr  9 09:20:24 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 09:20:24 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 09:20:24 host kernel: Call Trace:
> Apr  9 09:20:24 host kernel: BUG: soft lockup - CPU#0 stuck for 492s! 
> [swapper:0]
> Apr  9 09:20:24 host kernel: Modules linked in:
> Apr  9 09:20:24 host kernel: CPU 0:
> Apr  9 09:20:24 host kernel: Modules linked in:
> Apr  9 09:20:24 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 09:20:24 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 09:20:24 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 09:20:24 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 09:20:24 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 09:20:24 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 09:20:24 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 09:20:24 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 09:20:24 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 09:20:24 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 09:20:24 host kernel: CR2: 00007fff892a1e30 CR3: 000000006acb1000 CR4: 
> 0000000000002660
> Apr  9 09:20:24 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 09:20:24 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 09:20:24 host kernel: Call Trace:
> Apr  9 09:20:24 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 09:20:24 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 09:20:24 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 09:20:24 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 09:24:36 host kernel: BUG: soft lockup - CPU#1 stuck for 234s! 
> [gzip:4496]
> Apr  9 09:24:36 host kernel: Modules linked in:
> Apr  9 09:24:36 host kernel: CPU 1:
> Apr  9 09:24:36 host kernel: Modules linked in:
> Apr  9 09:24:36 host kernel: Pid: 4496, comm: gzip Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 09:24:36 host kernel: RIP: e033:[<0000000000409804>]  
> [<0000000000409804>] 0x409804
> Apr  9 09:24:36 host kernel: RSP: e02b:00007fff87e009f8  EFLAGS: 00000202
> Apr  9 09:24:36 host kernel: RAX: 00000000f6019e6d RBX: 0000000000008000 RCX: 
> 00007f529598c610
> Apr  9 09:24:36 host kernel: RDX: 0000000000f6019e RSI: 00000000000045a6 RDI: 
> 00000000006534fb
> Apr  9 09:24:36 host kernel: RBP: 0000000000005800 R08: 0000000000004000 R09: 
> 7464697720223333
> Apr  9 09:24:36 host kernel: R10: 6c61202236223d68 R11: 0000000000000246 R12: 
> 00000000006522a0
> Apr  9 09:24:36 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 09:24:36 host kernel: FS:  00007f5295e2a6e0(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 09:24:36 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 09:24:36 host kernel: CR2: 0000000000d550c8 CR3: 0000000026474000 CR4: 
> 0000000000002660
> Apr  9 09:24:37 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 09:24:37 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 09:24:37 host kernel: Call Trace:
> Apr  9 09:24:37 host kernel: BUG: soft lockup - CPU#0 stuck for 234s! 
> [swapper:0]
> Apr  9 09:24:37 host kernel: Modules linked in:
> Apr  9 09:24:37 host kernel: CPU 0:
> Apr  9 09:24:37 host kernel: Modules linked in:
> Apr  9 09:24:37 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 09:24:37 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 09:24:38 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 09:24:38 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 09:24:38 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 09:24:38 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 09:24:38 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 09:24:38 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 09:24:38 host kernel: FS:  00007f2f6a31a710(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 09:24:38 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 09:24:38 host kernel: CR2: 00007f2f6a327000 CR3: 000000003f62e000 CR4: 
> 0000000000002660
> Apr  9 09:24:38 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 09:24:38 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 09:24:38 host kernel: Call Trace:
> Apr  9 09:24:38 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 09:24:38 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 09:24:38 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 09:24:38 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 10:05:51 host kernel: BUG: soft lockup - CPU#1 stuck for 2305s! 
> [swapper:0]
> Apr  9 10:05:51 host kernel: Modules linked in:
> Apr  9 10:05:51 host kernel: CPU 1:
> Apr  9 10:05:51 host kernel: Modules linked in:
> Apr  9 10:05:51 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 10:05:51 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 10:05:51 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 10:05:51 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 10:05:51 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 10:05:51 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 10:05:51 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 10:05:51 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 10:05:51 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 10:05:51 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 10:05:51 host kernel: CR2: 0000000004786158 CR3: 000000007e2a6000 CR4: 
> 0000000000002660
> Apr  9 10:05:51 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 10:05:51 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 10:05:51 host kernel: Call Trace:
> Apr  9 10:05:51 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 10:05:51 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 10:05:51 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 10:05:51 host kernel: BUG: soft lockup - CPU#0 stuck for 2305s! 
> [httpd:9922]
> Apr  9 10:05:51 host kernel: Modules linked in:
> Apr  9 10:05:51 host kernel: CPU 0:
> Apr  9 10:05:51 host kernel: Modules linked in:
> Apr  9 10:05:51 host kernel: Pid: 9922, comm: httpd Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 10:05:51 host kernel: RIP: e033:[<00007ff39a44a36d>]  
> [<00007ff39a44a36d>] 0x7ff39a44a36d
> Apr  9 10:05:51 host kernel: RSP: e02b:00007fff4b9ad118  EFLAGS: 00000a86
> Apr  9 10:05:51 host kernel: RAX: c0c176e403831574 RBX: 0000000003643b60 RCX: 
> c0c176e403831574
> Apr  9 10:05:51 host kernel: RDX: 0000000000000000 RSI: 0000000000000004 RDI: 
> 000000000420e43b
> Apr  9 10:05:51 host kernel: RBP: 0000000003643b60 R08: 0000000000000004 R09: 
> 00007fff4b9ad270
> Apr  9 10:05:51 host kernel: R10: 00007ff39aaa9fa0 R11: 00007fff4b9ad188 R12: 
> 000000000320e6d0
> Apr  9 10:05:51 host kernel: R13: 0000000000000000 R14: 00007fff4b9ad368 R15: 
> 0000000000000000
> Apr  9 10:05:51 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 10:05:51 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 10:05:51 host kernel: CR2: 00000000047a4018 CR3: 000000007972c000 CR4: 
> 0000000000002660
> Apr  9 10:05:51 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 10:05:51 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 10:05:51 host kernel: Call Trace:
> Apr  9 10:50:04 host kernel: BUG: soft lockup - CPU#1 stuck for 2471s! 
> [swapper:0]
> Apr  9 10:50:04 host kernel: Modules linked in:
> Apr  9 10:50:04 host kernel: CPU 1:
> Apr  9 10:50:04 host kernel: Modules linked in:
> Apr  9 10:50:04 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 10:50:04 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 10:50:04 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 10:50:04 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 10:50:04 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 10:50:04 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 10:50:04 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 10:50:04 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 10:50:04 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 10:50:04 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 10:50:04 host kernel: CR2: 00007ff398a0ce08 CR3: 000000004ddb4000 CR4: 
> 0000000000002660
> Apr  9 10:50:04 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 10:50:04 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 10:50:04 host kernel: Call Trace:
> Apr  9 10:50:04 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 10:50:04 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 10:50:04 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 10:50:04 host kernel: BUG: soft lockup - CPU#0 stuck for 2471s! 
> [httpd:16797]
> Apr  9 10:50:04 host kernel: Modules linked in:
> Apr  9 10:50:04 host kernel: CPU 0:
> Apr  9 10:50:04 host kernel: Modules linked in:
> Apr  9 10:50:04 host kernel: Pid: 16797, comm: httpd Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 10:50:04 host kernel: RIP: e033:[<00007ff39a49ae55>]  
> [<00007ff39a49ae55>] 0x7ff39a49ae55
> Apr  9 10:50:04 host kernel: RSP: e02b:00007fff4b9b10b8  EFLAGS: 00000246
> Apr  9 10:50:04 host kernel: RAX: 000000000368dd40 RBX: 00000000036868d0 RCX: 
> 222879726575713e
> Apr  9 10:50:05 host kernel: RDX: 000000000000000c RSI: 0000000003686a6c RDI: 
> 00007fff4b9b1d00
> Apr  9 10:50:05 host kernel: RBP: 00007fff4b9b1da0 R08: 0000000000000001 R09: 
> 0101010101010101
> Apr  9 10:50:05 host kernel: R10: 0000000000000001 R11: 00007ff39c4ac0be R12: 
> 00007fff4b9b1d00
> Apr  9 10:50:05 host kernel: R13: 00007ff39aaa9fa0 R14: 00007ff39aaa9fa0 R15: 
> 0000000003695df0
> Apr  9 10:50:05 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 10:50:05 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 10:50:05 host kernel: CR2: 00000000032068c0 CR3: 000000004ddb4000 CR4: 
> 0000000000002660
> Apr  9 10:50:05 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 10:50:05 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 10:50:05 host kernel: Call Trace:
> Apr  9 10:57:01 host kernel: BUG: soft lockup - CPU#1 stuck for 389s! 
> [exim:17977]
> Apr  9 10:57:01 host kernel: Modules linked in:
> Apr  9 10:57:01 host kernel: CPU 1:
> Apr  9 10:57:01 host kernel: Modules linked in:
> Apr  9 10:57:01 host kernel: Pid: 17977, comm: exim Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 10:57:01 host kernel: RIP: e030:[<ffffffff8100922a>]  
> [<ffffffff8100922a>] hypercall_page+0x22a/0x1001
> Apr  9 10:57:01 host kernel: RSP: e02b:ffff880056611ea0  EFLAGS: 00000246
> Apr  9 10:57:01 host kernel: RAX: 0000000000030004 RBX: 0000000000000000 RCX: 
> ffffffff8100922a
> Apr  9 10:57:01 host kernel: RDX: 00007fd75a6d614e RSI: 0000000000000000 RDI: 
> 0000000000000000
> Apr  9 10:57:01 host kernel: RBP: 00007fd75a6d614e R08: 00007fd75c048710 R09: 
> 0000000000000003
> Apr  9 10:57:01 host kernel: R10: 00007fd75a909a30 R11: 0000000000000246 R12: 
> 0000000000000004
> Apr  9 10:57:01 host kernel: R13: ffff880056611f58 R14: ffff88005aa996d0 R15: 
> ffff88005a51b3c0
> Apr  9 10:57:01 host kernel: FS:  00007fd75c048710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 10:57:01 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 10:57:01 host kernel: CR2: 00007fd75a6d614e CR3: 000000007eb9a000 CR4: 
> 0000000000002660
> Apr  9 10:57:01 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 10:57:01 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 10:57:01 host kernel: Call Trace:
> Apr  9 10:57:01 host kernel:  [<ffffffff8100dd39>] ? 
> xen_force_evtchn_callback+0x9/0xa
> Apr  9 10:57:01 host kernel:  [<ffffffff8100e3b2>] ? check_events+0x12/0x20
> Apr  9 10:57:01 host kernel:  [<ffffffff8100e359>] ? 
> xen_irq_enable_direct_end+0x0/0x7
> Apr  9 10:57:01 host kernel:  [<ffffffff814da477>] ? do_page_fault+0xfb/0x28d
> Apr  9 10:57:01 host kernel:  [<ffffffff814d85b5>] ? page_fault+0x25/0x30
> Apr  9 10:57:01 host kernel: BUG: soft lockup - CPU#0 stuck for 389s! 
> [netstat:17966]
> Apr  9 10:57:01 host kernel: Modules linked in:
> Apr  9 10:57:01 host kernel: CPU 0:
> Apr  9 10:57:01 host kernel: Modules linked in:
> Apr  9 10:57:01 host kernel: Pid: 17966, comm: netstat Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 10:57:01 host kernel: RIP: e030:[<ffffffff814d8226>]  
> [<ffffffff814d8226>] _spin_lock+0xa/0x15
> Apr  9 10:57:01 host kernel: RSP: e02b:ffff88006ad4dce0  EFLAGS: 00000202
> Apr  9 10:57:01 host kernel: RAX: 0000000000001c1c RBX: ffff88007ff77180 RCX: 
> 0000000000000012
> Apr  9 10:57:01 host kernel: RDX: 018721dfffc35c9f RSI: ffff88006ad4ddc8 RDI: 
> ffff88007ff77188
> Apr  9 10:57:01 host kernel: RBP: ffff88007ff77198 R08: 0000000000000000 R09: 
> 0000000000300000
> Apr  9 10:57:01 host kernel: R10: 00000000fffffff2 R11: ffffffff8126158e R12: 
> ffff88000ff859c0
> Apr  9 10:57:01 host kernel: R13: ffff88006ad4ddc8 R14: 0000000000034b38 R15: 
> 0000000000000002
> Apr  9 10:57:01 host kernel: FS:  00007fb435545710(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 10:57:01 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 10:57:01 host kernel: CR2: 00007f0e429d6000 CR3: 000000007ee3d000 CR4: 
> 0000000000002660
> Apr  9 10:57:01 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 10:57:01 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 10:57:01 host kernel: Call Trace:
> Apr  9 10:57:01 host kernel:  [<ffffffff810cb01b>] ? __d_lookup+0x7a/0x105
> Apr  9 10:57:01 host kernel:  [<ffffffff810c251e>] ? do_lookup+0x2a/0x1b3
> Apr  9 10:57:01 host kernel:  [<ffffffff810c4498>] ? 
> __link_path_walk+0x348/0xdcd
> Apr  9 10:57:01 host kernel:  [<ffffffff8100dd39>] ? 
> xen_force_evtchn_callback+0x9/0xa
> Apr  9 10:57:01 host kernel:  [<ffffffff8100e3b2>] ? check_events+0x12/0x20
> Apr  9 10:57:01 host kernel:  [<ffffffff810c514b>] ? path_walk+0x66/0xca
> Apr  9 10:57:01 host kernel:  [<ffffffff810c51cf>] ? do_path_lookup+0x20/0x77
> Apr  9 10:57:01 host kernel:  [<ffffffff810c5ae9>] ? user_path_at+0x48/0x79
> Apr  9 10:57:01 host kernel:  [<ffffffff8100e39f>] ? 
> xen_restore_fl_direct_end+0x0/0x1
> Apr  9 10:57:01 host kernel:  [<ffffffff810cfd6b>] ? 
> mntput_no_expire+0x23/0xed
> Apr  9 10:57:01 host kernel:  [<ffffffff8129b703>] ? 
> _atomic_dec_and_lock+0x33/0x50
> Apr  9 10:57:01 host kernel:  [<ffffffff810cfd6b>] ? 
> mntput_no_expire+0x23/0xed
> Apr  9 10:57:01 host kernel:  [<ffffffff810bdf4d>] ? sys_readlinkat+0x25/0x8d
> Apr  9 10:57:01 host kernel:  [<ffffffff81010ac2>] ? 
> system_call_fastpath+0x16/0x1b
> Apr  9 11:00:36 host kernel: BUG: soft lockup - CPU#1 stuck for 200s! 
> [swapper:0]
> Apr  9 11:00:36 host kernel: Modules linked in:
> Apr  9 11:00:36 host kernel: CPU 1:
> Apr  9 11:00:36 host kernel: Modules linked in:
> Apr  9 11:00:36 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 11:00:36 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 11:00:36 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 11:00:36 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 11:00:36 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 11:00:36 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 11:00:36 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 11:00:36 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 11:00:36 host kernel: FS:  00007f719f5fa6e0(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 11:00:36 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 11:00:36 host kernel: CR2: 00007ff39c43cfa8 CR3: 000000007d0db000 CR4: 
> 0000000000002660
> Apr  9 11:00:36 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 11:00:36 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 11:00:37 host kernel: Call Trace:
> Apr  9 11:00:37 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 11:00:37 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 11:00:37 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 11:00:37 host kernel: BUG: soft lockup - CPU#0 stuck for 200s! 
> [swapper:0]
> Apr  9 11:00:37 host kernel: Modules linked in:
> Apr  9 11:00:37 host kernel: CPU 0:
> Apr  9 11:00:37 host kernel: Modules linked in:
> Apr  9 11:00:37 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 11:00:37 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 11:00:37 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 11:00:37 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 11:00:37 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 11:00:37 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 11:00:37 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 11:00:37 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 11:00:37 host kernel: FS:  00007f07704fe6e0(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 11:00:37 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 11:00:37 host kernel: CR2: 00000000035750c8 CR3: 000000007ea99000 CR4: 
> 0000000000002660
> Apr  9 11:00:37 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 11:00:37 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 11:00:37 host kernel: Call Trace:
> Apr  9 11:00:37 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 11:00:37 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 11:00:37 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 11:00:37 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 11:08:50 host kernel: BUG: soft lockup - CPU#0 stuck for 460s! 
> [httpd:19360]
> Apr  9 11:08:50 host kernel: Modules linked in:
> Apr  9 11:08:50 host kernel: CPU 0:
> Apr  9 11:08:50 host kernel: Modules linked in:
> Apr  9 11:08:50 host kernel: Pid: 19360, comm: httpd Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 11:08:50 host kernel: RIP: e030:[<ffffffff813b995e>]  
> [<ffffffff813b995e>] sock_alloc_send_skb+0x3/0xc
> Apr  9 11:08:50 host kernel: RSP: e02b:ffff88005675dc80  EFLAGS: 00000246
> Apr  9 11:08:50 host kernel: RAX: 0000000000003e00 RBX: ffff88005675dcb8 RCX: 
> ffff88005675dce4
> Apr  9 11:08:50 host kernel: RDX: 0000000000000000 RSI: 0000000000000057 RDI: 
> ffff88002678c900
> Apr  9 11:08:50 host kernel: RBP: ffff88007f740b80 R08: ffff88005675dce4 R09: 
> 75646f72705f6d76
> Apr  9 11:08:50 host kernel: R10: 5245485720607463 R11: ffffffff81261683 R12: 
> 0000000000000057
> Apr  9 11:08:50 host kernel: R13: 0000000000000000 R14: ffff88006add3c80 R15: 
> ffff88005675dd60
> Apr  9 11:08:50 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 11:08:50 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 11:08:50 host kernel: CR2: 00000000045cf000 CR3: 000000007f97d000 CR4: 
> 0000000000002660
> Apr  9 11:08:50 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 11:08:50 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 11:08:50 host kernel: Call Trace:
> Apr  9 11:08:50 host kernel:  [<ffffffff814573c8>] ? 
> unix_stream_sendmsg+0x18a/0x31f
> Apr  9 11:08:50 host kernel:  [<ffffffff813b61ec>] ? sock_aio_write+0xf6/0x10a
> Apr  9 11:08:50 host kernel:  [<ffffffff8100dd39>] ? 
> xen_force_evtchn_callback+0x9/0xa
> Apr  9 11:08:50 host kernel:  [<ffffffff8100e39f>] ? 
> xen_restore_fl_direct_end+0x0/0x1
> Apr  9 11:08:50 host kernel:  [<ffffffff810bab7e>] ? do_sync_write+0xce/0x113
> Apr  9 11:08:50 host kernel:  [<ffffffff810c5af3>] ? user_path_at+0x52/0x79
> Apr  9 11:08:50 host kernel:  [<ffffffff810be0b0>] ? cp_new_stat+0xe9/0xfc
> Apr  9 11:08:50 host kernel:  [<ffffffff810561b8>] ? 
> autoremove_wake_function+0x0/0x2e
> Apr  9 11:08:50 host kernel:  [<ffffffff8100e3b2>] ? check_events+0x12/0x20
> Apr  9 11:08:50 host kernel:  [<ffffffff8100dd39>] ? 
> xen_force_evtchn_callback+0x9/0xa
> Apr  9 11:08:50 host kernel:  [<ffffffff8100e3b2>] ? check_events+0x12/0x20
> Apr  9 11:08:50 host kernel:  [<ffffffff810bb4c5>] ? vfs_write+0xc0/0x149
> Apr  9 11:08:50 host kernel:  [<ffffffff810bb60a>] ? sys_write+0x45/0x6e
> Apr  9 11:08:50 host kernel:  [<ffffffff81010ac2>] ? 
> system_call_fastpath+0x16/0x1b
> Apr  9 11:08:50 host kernel: BUG: soft lockup - CPU#1 stuck for 460s! 
> [swapper:0]
> Apr  9 11:08:50 host kernel: Modules linked in:
> Apr  9 11:08:50 host kernel: CPU 1:
> Apr  9 11:08:50 host kernel: Modules linked in:
> Apr  9 11:08:50 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 11:08:50 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 11:08:50 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 11:08:50 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 11:08:50 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 11:08:50 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 11:08:50 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 11:08:50 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 11:08:50 host kernel: FS:  00007f0e429cb6e0(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 11:08:50 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 11:08:50 host kernel: CR2: 0000000004598000 CR3: 000000007e2a6000 CR4: 
> 0000000000002660
> Apr  9 11:08:50 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 11:08:50 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 11:08:50 host kernel: Call Trace:
> Apr  9 11:08:50 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 11:08:50 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 11:08:50 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 12:00:40 host kernel: BUG: soft lockup - CPU#1 stuck for 2897s! 
> [httpd:26903]
> Apr  9 12:00:40 host kernel: Modules linked in:
> Apr  9 12:00:40 host kernel: CPU 1:
> Apr  9 12:00:40 host kernel: Modules linked in:
> Apr  9 12:00:40 host kernel: Pid: 26903, comm: httpd Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 12:00:40 host kernel: RIP: e033:[<00007ff39a427cd2>]  
> [<00007ff39a427cd2>] 0x7ff39a427cd2
> Apr  9 12:00:40 host kernel: RSP: e02b:00007fff4b9afa20  EFLAGS: 00000202
> Apr  9 12:00:40 host kernel: RAX: 0000000000000020 RBX: 00007fff4b9aff30 RCX: 
> 0000000000000013
> Apr  9 12:00:40 host kernel: RDX: 00007ff39a77be20 RSI: 00007fff4b9aff50 RDI: 
> 0000000003bd6418
> Apr  9 12:00:40 host kernel: RBP: 0000000000000000 R08: 00007ff39a782e60 R09: 
> 00007ff39a783d80
> Apr  9 12:00:40 host kernel: R10: 0000000000000000 R11: 00007ff39c4ac0a2 R12: 
> 00007ff39aaaa3e0
> Apr  9 12:00:40 host kernel: R13: 00007fff4b9aff50 R14: 00007fff4b9aff50 R15: 
> 000000000000015e
> Apr  9 12:00:40 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 12:00:40 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 12:00:40 host kernel: CR2: 0000000003bdc000 CR3: 0000000056566000 CR4: 
> 0000000000002660
> Apr  9 12:00:40 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 12:00:40 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 12:00:40 host kernel: Call Trace:
> Apr  9 12:00:40 host kernel: BUG: soft lockup - CPU#0 stuck for 2897s! 
> [swapper:0]
> Apr  9 12:00:40 host kernel: Modules linked in:
> Apr  9 12:00:40 host kernel: CPU 0:
> Apr  9 12:00:40 host kernel: Modules linked in:
> Apr  9 12:00:40 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 12:00:40 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 12:00:40 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 12:00:40 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 12:00:40 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 12:00:40 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 12:00:40 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 12:00:40 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 12:00:40 host kernel: FS:  00007f74fde746e0(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 12:00:40 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 12:00:40 host kernel: CR2: 00007ff39aec11a7 CR3: 000000007e2a6000 CR4: 
> 0000000000002660
> Apr  9 12:00:40 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 12:00:40 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 12:00:40 host kernel: Call Trace:
> Apr  9 12:00:40 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 12:00:40 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 12:00:40 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 12:00:40 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 12:19:45 host kernel: BUG: soft lockup - CPU#1 stuck for 1065s! 
> [swapper:0]
> Apr  9 12:19:45 host kernel: Modules linked in:
> Apr  9 12:19:45 host kernel: CPU 1:
> Apr  9 12:19:45 host kernel: Modules linked in:
> Apr  9 12:19:45 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 12:19:45 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 12:19:45 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 12:19:45 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 12:19:45 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 12:19:45 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 12:19:45 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 12:19:45 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 12:19:45 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 12:19:45 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 12:19:45 host kernel: CR2: 00007ff39c5002b0 CR3: 0000000079588000 CR4: 
> 0000000000002660
> Apr  9 12:19:45 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 12:19:45 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 12:19:45 host kernel: Call Trace:
> Apr  9 12:19:45 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 12:19:45 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 12:19:45 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 12:19:45 host kernel: BUG: soft lockup - CPU#0 stuck for 1065s! 
> [swapper:0]
> Apr  9 12:19:45 host kernel: Modules linked in:
> Apr  9 12:19:45 host kernel: CPU 0:
> Apr  9 12:19:45 host kernel: Modules linked in:
> Apr  9 12:19:45 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 12:19:45 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 12:19:45 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 12:19:45 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 12:19:45 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 12:19:45 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 12:19:45 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 12:19:45 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 12:19:45 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 12:19:45 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 12:19:45 host kernel: CR2: 00007ff39e404000 CR3: 000000007e2a6000 CR4: 
> 0000000000002660
> Apr  9 12:19:45 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 12:19:45 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 12:19:45 host kernel: Call Trace:
> Apr  9 12:19:45 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 12:19:45 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 12:19:45 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 12:19:45 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 12:37:26 host kernel: BUG: soft lockup - CPU#1 stuck for 989s! 
> [swapper:0]
> Apr  9 12:37:26 host kernel: Modules linked in:
> Apr  9 12:37:26 host kernel: CPU 1:
> Apr  9 12:37:26 host kernel: Modules linked in:
> Apr  9 12:37:26 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 12:37:26 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 12:37:26 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 12:37:26 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 12:37:26 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 12:37:26 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 12:37:26 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 12:37:26 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 12:37:26 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 12:37:26 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 12:37:26 host kernel: CR2: 00000000039f7330 CR3: 000000007e2a6000 CR4: 
> 0000000000002660
> Apr  9 12:37:26 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 12:37:26 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 12:37:26 host kernel: Call Trace:
> Apr  9 12:37:26 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 12:37:26 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 12:37:26 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 12:37:26 host kernel: BUG: soft lockup - CPU#0 stuck for 989s! 
> [httpd:31684]
> Apr  9 12:37:26 host kernel: Modules linked in:
> Apr  9 12:37:26 host kernel: CPU 0:
> Apr  9 12:37:26 host kernel: Modules linked in:
> Apr  9 12:37:26 host kernel: Pid: 31684, comm: httpd Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 12:37:26 host kernel: RIP: e033:[<00007ff39a41f7d3>]  
> [<00007ff39a41f7d3>] 0x7ff39a41f7d3
> Apr  9 12:37:26 host kernel: RSP: e02b:00007fff4b9ad330  EFLAGS: 00000246
> Apr  9 12:37:26 host kernel: RAX: 00007ff39a41f7ac RBX: 00007ff39aaaa3e0 RCX: 
> 0000000000000001
> Apr  9 12:37:26 host kernel: RDX: 00007ff39a781bdc RSI: 000000000000003e RDI: 
> 0000000000000000
> Apr  9 12:37:26 host kernel: RBP: 00007ff39aaa9e60 R08: 00007ff39a782e60 R09: 
> 0000000000000003
> Apr  9 12:37:26 host kernel: R10: 0000000000000000 R11: 00007ff39c4ac0a2 R12: 
> 0000000003a2b297
> Apr  9 12:37:26 host kernel: R13: 00007ff39aaaa3e0 R14: 00007ff39a784da0 R15: 
> 00007ff39a783e80
> Apr  9 12:37:26 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 12:37:26 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 12:37:26 host kernel: CR2: 0000000003aa1000 CR3: 000000004debb000 CR4: 
> 0000000000002660
> Apr  9 12:37:26 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 12:37:26 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 12:37:26 host kernel: Call Trace:
> Apr  9 13:44:52 host kernel: BUG: soft lockup - CPU#1 stuck for 3768s! 
> [swapper:0]
> Apr  9 13:44:52 host kernel: Modules linked in:
> Apr  9 13:44:52 host kernel: CPU 1:
> Apr  9 13:44:52 host kernel: Modules linked in:
> Apr  9 13:44:52 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 13:44:52 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 13:44:52 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 13:44:52 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 13:44:52 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 13:44:52 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 13:44:52 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 13:44:52 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 13:44:52 host kernel: FS:  00007fd75c048710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 13:44:52 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 13:44:52 host kernel: CR2: 00007fff892a1e30 CR3: 0000000071c86000 CR4: 
> 0000000000002660
> Apr  9 13:44:52 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 13:44:52 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 13:44:52 host kernel: Call Trace:
> Apr  9 13:44:52 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 13:44:52 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 13:44:52 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 13:44:52 host kernel: BUG: soft lockup - CPU#0 stuck for 3768s! 
> [swapper:0]
> Apr  9 13:44:52 host kernel: Modules linked in:
> Apr  9 13:44:52 host kernel: CPU 0:
> Apr  9 13:44:52 host kernel: Modules linked in:
> Apr  9 13:44:52 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 13:44:52 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 13:44:52 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 13:44:52 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 13:44:52 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 13:44:52 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 13:44:52 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 13:44:52 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 13:44:52 host kernel: FS:  00007f0e429cb6e0(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 13:44:52 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 13:44:52 host kernel: CR2: 00007f0e429d6000 CR3: 000000007d0de000 CR4: 
> 0000000000002660
> Apr  9 13:44:52 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 13:44:52 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 13:44:52 host kernel: Call Trace:
> Apr  9 13:44:52 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 13:44:52 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 13:44:52 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 13:44:52 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 13:55:43 host kernel: BUG: soft lockup - CPU#1 stuck for 607s! 
> [swapper:0]
> Apr  9 13:55:43 host kernel: Modules linked in:
> Apr  9 13:55:43 host kernel: CPU 1:
> Apr  9 13:55:43 host kernel: Modules linked in:
> Apr  9 13:55:43 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 13:55:43 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 13:55:43 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 13:55:43 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 13:55:43 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 13:55:43 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 13:55:43 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 13:55:43 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 13:55:43 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 13:55:43 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 13:55:43 host kernel: CR2: 0000000003c7b27f CR3: 00000000266e8000 CR4: 
> 0000000000002660
> Apr  9 13:55:43 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 13:55:43 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 13:55:43 host kernel: Call Trace:
> Apr  9 13:55:43 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 13:55:43 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 13:55:43 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 13:55:43 host kernel: BUG: soft lockup - CPU#0 stuck for 607s! 
> [swapper:0]
> Apr  9 13:55:43 host kernel: Modules linked in:
> Apr  9 13:55:43 host kernel: CPU 0:
> Apr  9 13:55:43 host kernel: Modules linked in:
> Apr  9 13:55:43 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 13:55:43 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 13:55:43 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 13:55:43 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 13:55:43 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 13:55:43 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 13:55:43 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 13:55:43 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 13:55:43 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 13:55:43 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 13:55:43 host kernel: CR2: 00007f2f28bf62e8 CR3: 000000007e2a6000 CR4: 
> 0000000000002660
> Apr  9 13:55:43 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 13:55:43 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 13:55:43 host kernel: Call Trace:
> Apr  9 13:55:43 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 13:55:43 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 13:55:43 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 13:55:43 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 14:16:21 host kernel: BUG: soft lockup - CPU#1 stuck for 1153s! 
> [swapper:0]
> Apr  9 14:16:21 host kernel: Modules linked in:
> Apr  9 14:16:21 host kernel: CPU 1:
> Apr  9 14:16:21 host kernel: Modules linked in:
> Apr  9 14:16:21 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 14:16:21 host kernel: RIP: e030:[<ffffffff8100922a>]  
> [<ffffffff8100922a>] hypercall_page+0x22a/0x1001
> Apr  9 14:16:21 host kernel: RSP: e02b:ffff88007f8a1dd8  EFLAGS: 00000246
> Apr  9 14:16:21 host kernel: RAX: 0000000000030004 RBX: ffff88007db55810 RCX: 
> ffffffff8100922a
> Apr  9 14:16:21 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000000
> Apr  9 14:16:21 host kernel: RBP: 0000000000000001 R08: ffff88007f8a0000 R09: 
> ffffffff8100e3b2
> Apr  9 14:16:21 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 14:16:21 host kernel: R13: ffff88007e0040c0 R14: 0000000000000000 R15: 
> 0000000000001000
> Apr  9 14:16:21 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 14:16:21 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 14:16:21 host kernel: CR2: 00007ff39e4a7000 CR3: 000000007e2a6000 CR4: 
> 0000000000002660
> Apr  9 14:16:21 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 14:16:21 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 14:16:21 host kernel: Call Trace:
> Apr  9 14:16:21 host kernel:  [<ffffffff8100dd39>] ? 
> xen_force_evtchn_callback+0x9/0xa
> Apr  9 14:16:21 host kernel:  [<ffffffff8100e3b2>] ? check_events+0x12/0x20
> Apr  9 14:16:21 host kernel:  [<ffffffff8100e3b2>] ? check_events+0x12/0x20
> Apr  9 14:16:21 host kernel:  [<ffffffff8100e359>] ? 
> xen_irq_enable_direct_end+0x0/0x7
> Apr  9 14:16:21 host kernel:  [<ffffffff8103b070>] ? 
> finish_task_switch+0x27/0x6e
> Apr  9 14:16:21 host kernel:  [<ffffffff814d6a77>] ? thread_return+0x3e/0x109
> Apr  9 14:16:21 host kernel:  [<ffffffff814d87ba>] ? error_exit+0x2a/0x60
> Apr  9 14:16:21 host kernel:  [<ffffffff8101149d>] ? 
> retint_restore_args+0x5/0x6
> Apr  9 14:16:21 host kernel:  [<ffffffff8100fe5f>] ? cpu_idle+0x80/0x82
> Apr  9 14:16:21 host kernel: BUG: soft lockup - CPU#0 stuck for 1153s! 
> [swapper:0]
> Apr  9 14:16:21 host kernel: Modules linked in:
> Apr  9 14:16:21 host kernel: CPU 0:
> Apr  9 14:16:21 host kernel: Modules linked in:
> Apr  9 14:16:21 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 14:16:21 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 14:16:21 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 14:16:21 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 14:16:21 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 14:16:21 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 14:16:21 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 14:16:21 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 14:16:21 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 14:16:21 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 14:16:21 host kernel: CR2: 00007ff39e4a7000 CR3: 000000007e2a6000 CR4: 
> 0000000000002660
> Apr  9 14:16:21 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 14:16:21 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 14:16:21 host kernel: Call Trace:
> Apr  9 14:16:21 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 14:16:21 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 14:16:21 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 14:16:21 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 15:40:21 host kernel: BUG: soft lockup - CPU#1 stuck for 4104s! 
> [swapper:0]
> Apr  9 15:40:21 host kernel: Modules linked in:
> Apr  9 15:40:21 host kernel: CPU 1:
> Apr  9 15:40:21 host kernel: Modules linked in:
> Apr  9 15:40:21 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 15:40:21 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 15:40:21 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 15:40:21 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 15:40:21 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 15:40:21 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 15:40:21 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 15:40:21 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 15:40:21 host kernel: FS:  00007fd75c048710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 15:40:21 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 15:40:21 host kernel: CR2: 000000000040d0e2 CR3: 0000000071c86000 CR4: 
> 0000000000002660
> Apr  9 15:40:21 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 15:40:21 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 15:40:21 host kernel: Call Trace:
> Apr  9 15:40:21 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 15:40:21 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 15:40:21 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 15:40:21 host kernel: BUG: soft lockup - CPU#0 stuck for 4104s! 
> [swapper:0]
> Apr  9 15:40:21 host kernel: Modules linked in:
> Apr  9 15:40:21 host kernel: CPU 0:
> Apr  9 15:40:21 host kernel: Modules linked in:
> Apr  9 15:40:21 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 15:40:21 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 15:40:21 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 15:40:21 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 15:40:21 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 15:40:21 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 15:40:21 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 15:40:21 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 15:40:21 host kernel: FS:  00007f07704fe6e0(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 15:40:21 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 15:40:21 host kernel: CR2: 00000000004c04a8 CR3: 0000000071c86000 CR4: 
> 0000000000002660
> Apr  9 15:40:21 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 15:40:21 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 15:40:21 host kernel: Call Trace:
> Apr  9 15:40:21 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 15:40:21 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 15:40:21 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 15:40:21 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 16:06:21 host kernel: BUG: soft lockup - CPU#1 stuck for 1452s! 
> [swapper:0]
> Apr  9 16:06:21 host kernel: Modules linked in:
> Apr  9 16:06:21 host kernel: CPU 1:
> Apr  9 16:06:21 host kernel: Modules linked in:
> Apr  9 16:06:21 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 16:06:21 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 16:06:21 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 16:06:21 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 16:06:21 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 16:06:21 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 16:06:21 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 16:06:21 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 16:06:21 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 16:06:25 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 16:06:25 host kernel: CR2: 00007fbd3c012909 CR3: 0000000071c86000 CR4: 
> 0000000000002660
> Apr  9 16:06:26 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 16:06:26 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 16:06:26 host kernel: Call Trace:
> Apr  9 16:06:26 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 16:06:26 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 16:06:26 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 16:06:26 host kernel: BUG: soft lockup - CPU#0 stuck for 1452s! 
> [swapper:0]
> Apr  9 16:06:26 host kernel: Modules linked in:
> Apr  9 16:06:26 host kernel: CPU 0:
> Apr  9 16:06:26 host kernel: Modules linked in:
> Apr  9 16:06:26 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 16:06:26 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 16:06:26 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 16:06:26 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 16:06:26 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 16:06:26 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 16:06:26 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 16:06:26 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 16:06:26 host kernel: FS:  00007f07704fe6e0(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 16:06:26 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 16:06:26 host kernel: CR2: 00007f09a4413000 CR3: 000000007ea99000 CR4: 
> 0000000000002660
> Apr  9 16:06:26 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 16:06:26 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 16:06:26 host kernel: Call Trace:
> Apr  9 16:06:26 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 16:06:26 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 16:06:26 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 16:06:26 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 16:17:43 host kernel: BUG: soft lockup - CPU#1 stuck for 637s! 
> [httpd:32342]
> Apr  9 16:17:43 host kernel: Modules linked in:
> Apr  9 16:17:43 host kernel: CPU 1:
> Apr  9 16:17:43 host kernel: Modules linked in:
> Apr  9 16:17:43 host kernel: Pid: 32342, comm: httpd Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 16:17:43 host kernel: RIP: e030:[<ffffffff8100922a>]  
> [<ffffffff8100922a>] hypercall_page+0x22a/0x1001
> Apr  9 16:17:43 host kernel: RSP: e02b:ffff88007955fea0  EFLAGS: 00000246
> Apr  9 16:17:43 host kernel: RAX: 0000000000030004 RBX: 0000000000000000 RCX: 
> ffffffff8100922a
> Apr  9 16:17:43 host kernel: RDX: 000000000002ba30 RSI: 0000000000000000 RDI: 
> 0000000000000000
> Apr  9 16:17:43 host kernel: RBP: 000000000394ca98 R08: 0000000000033240 R09: 
> 000000310f6de059
> Apr  9 16:17:43 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000006
> Apr  9 16:17:43 host kernel: R13: ffff88007955ff58 R14: ffff88007f9a8ab0 R15: 
> ffff88007d093b40
> Apr  9 16:17:43 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 16:17:43 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 16:17:43 host kernel: CR2: 000000000394ca98 CR3: 000000006adde000 CR4: 
> 0000000000002660
> Apr  9 16:17:43 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 16:17:43 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 16:17:43 host kernel: Call Trace:
> Apr  9 16:17:43 host kernel:  [<ffffffff8100dd39>] ? 
> xen_force_evtchn_callback+0x9/0xa
> Apr  9 16:17:43 host kernel:  [<ffffffff8100e3b2>] ? check_events+0x12/0x20
> Apr  9 16:17:43 host kernel:  [<ffffffff8100e359>] ? 
> xen_irq_enable_direct_end+0x0/0x7
> Apr  9 16:17:43 host kernel:  [<ffffffff814da477>] ? do_page_fault+0xfb/0x28d
> Apr  9 16:17:43 host kernel:  [<ffffffff814d85b5>] ? page_fault+0x25/0x30
> Apr  9 16:17:43 host kernel: BUG: soft lockup - CPU#0 stuck for 637s! 
> [swapper:0]
> Apr  9 16:17:43 host kernel: Modules linked in:
> Apr  9 16:17:43 host kernel: CPU 0:
> Apr  9 16:17:43 host kernel: Modules linked in:
> Apr  9 16:17:43 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 16:17:43 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 16:17:44 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 16:17:44 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 16:17:44 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 16:17:44 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 16:17:44 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 16:17:44 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 16:17:44 host kernel: FS:  00007f07704fe6e0(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 16:17:44 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 16:17:44 host kernel: CR2: 00000000004505b0 CR3: 000000007ea99000 CR4: 
> 0000000000002660
> Apr  9 16:17:44 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 16:17:44 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 16:17:44 host kernel: Call Trace:
> Apr  9 16:17:44 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 16:17:44 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 16:17:44 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 16:17:44 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 16:20:22 host kernel: BUG: soft lockup - CPU#1 stuck for 147s! 
> [swapper:0]
> Apr  9 16:20:22 host kernel: Modules linked in:
> Apr  9 16:20:22 host kernel: CPU 1:
> Apr  9 16:20:22 host kernel: Modules linked in:
> Apr  9 16:20:22 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 16:20:22 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 16:20:22 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 16:20:22 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 16:20:22 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 16:20:22 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 16:20:22 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 16:20:22 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 16:20:22 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 16:20:22 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 16:20:22 host kernel: CR2: 00007ff39aec11a7 CR3: 0000000071c86000 CR4: 
> 0000000000002660
> Apr  9 16:20:22 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 16:20:22 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 16:20:22 host kernel: Call Trace:
> Apr  9 16:20:22 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 16:20:22 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 16:20:22 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 16:20:22 host kernel: BUG: soft lockup - CPU#0 stuck for 147s! 
> [swapper:0]
> Apr  9 16:20:22 host kernel: Modules linked in:
> Apr  9 16:20:22 host kernel: CPU 0:
> Apr  9 16:20:22 host kernel: Modules linked in:
> Apr  9 16:20:22 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 16:20:22 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 16:20:22 host kernel: RSP: e02b:ffffffff81691f70  EFLAGS: 00000246
> Apr  9 16:20:22 host kernel: RAX: 0000000000000000 RBX: ffffffff81690000 RCX: 
> ffffffff810093aa
> Apr  9 16:20:22 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 16:20:22 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 16:20:22 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> ffffffffffffffff
> Apr  9 16:20:22 host kernel: R13: ffffffff818ebf20 R14: ffffffff818eec70 R15: 
> 0000000000000000
> Apr  9 16:20:22 host kernel: FS:  00007f07704fe6e0(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 16:20:22 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 16:20:22 host kernel: CR2: 00000000006ddc20 CR3: 0000000071c86000 CR4: 
> 0000000000002660
> Apr  9 16:20:22 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 16:20:22 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 16:20:22 host kernel: Call Trace:
> Apr  9 16:20:22 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 16:20:22 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 16:20:22 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 16:20:22 host kernel:  [<ffffffff818b6c42>] ? start_kernel+0x353/0x35f
> Apr  9 16:22:29 host kernel: BUG: soft lockup - CPU#1 stuck for 119s! 
> [swapper:0]
> Apr  9 16:22:29 host kernel: Modules linked in:
> Apr  9 16:22:29 host kernel: CPU 1:
> Apr  9 16:22:29 host kernel: Modules linked in:
> Apr  9 16:22:29 host kernel: Pid: 0, comm: swapper Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 16:22:29 host kernel: RIP: e030:[<ffffffff810093aa>]  
> [<ffffffff810093aa>] hypercall_page+0x3aa/0x1001
> Apr  9 16:22:29 host kernel: RSP: e02b:ffff88007f8a1f08  EFLAGS: 00000246
> Apr  9 16:22:29 host kernel: RAX: 0000000000000000 RBX: ffff88007f8a0000 RCX: 
> ffffffff810093aa
> Apr  9 16:22:29 host kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> 0000000000000001
> Apr  9 16:22:29 host kernel: RBP: ffffffff81896d30 R08: 0000000000000000 R09: 
> ffffffff8100e3b2
> Apr  9 16:22:29 host kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 
> 0000000000000000
> Apr  9 16:22:29 host kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 
> 0000000000000000
> Apr  9 16:22:29 host kernel: FS:  00007ff39e495710(0000) 
> GS:ffff8800022c7000(0000) knlGS:0000000000000000
> Apr  9 16:22:29 host kernel: CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
> Apr  9 16:22:29 host kernel: CR2: 00007ff39c506460 CR3: 0000000026469000 CR4: 
> 0000000000002660
> Apr  9 16:22:29 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 16:22:29 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 16:22:29 host kernel: Call Trace:
> Apr  9 16:22:29 host kernel:  [<ffffffff8100ddb7>] ? xen_safe_halt+0xc/0x15
> Apr  9 16:22:29 host kernel:  [<ffffffff8100bdcf>] ? xen_idle+0x37/0x40
> Apr  9 16:22:29 host kernel:  [<ffffffff8100fe2e>] ? cpu_idle+0x4f/0x82
> Apr  9 16:22:29 host kernel: BUG: soft lockup - CPU#0 stuck for 119s! 
> [syslogd:1848]
> Apr  9 16:22:29 host kernel: Modules linked in:
> Apr  9 16:22:29 host kernel: CPU 0:
> Apr  9 16:22:29 host kernel: Modules linked in:
> Apr  9 16:22:29 host kernel: Pid: 1848, comm: syslogd Not tainted 
> 2.6.32.9xls-domU #2 
> Apr  9 16:22:29 host kernel: RIP: 0033:[<00007f74fd9e4ad5>]  
> [<00007f74fd9e4ad5>] 0x7f74fd9e4ad5
> Apr  9 16:22:29 host kernel: RSP: 002b:00007fff22fa00d0  EFLAGS: 00000246
> Apr  9 16:22:29 host kernel: RAX: 000000000000002e RBX: 0000000000000000 RCX: 
> ffffffff810092eb
> Apr  9 16:22:29 host kernel: RDX: 00000000000003fe RSI: 00007fff22fa01b0 RDI: 
> 0000000000000000
> Apr  9 16:22:30 host kernel: RBP: 00000000000003fe R08: 0000000000000000 R09: 
> 0000000000000000
> Apr  9 16:22:30 host kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 
> 00007fff22fa01b0
> Apr  9 16:22:30 host kernel: R13: 0000000000000000 R14: 00007fff22fa0120 R15: 
> 00007f74fde74690
> Apr  9 16:22:30 host kernel: FS:  00007f74fde746e0(0000) 
> GS:ffff8800022ac000(0000) knlGS:0000000000000000
> Apr  9 16:22:30 host kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Apr  9 16:22:30 host kernel: CR2: 00007f0e429d6000 CR3: 000000007d03a000 CR4: 
> 0000000000002660
> Apr  9 16:22:30 host kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> Apr  9 16:22:30 host kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> Apr  9 16:22:30 host kernel: Call Trace:
> 

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.