[Xen-devel] NFS related netback hang

["G.R." <firemeteor@xxxxxxxxxxxxxxxxxxxxx>]

Hi,
I'm suffering from strange NFS related network issue for a while.

The issue shows up when copying from dom0 to domU through a NFS mount.
After a short while, the transfer suddenly freezes and the domU
network simply stops any response. Force mounting the NFS mount
generally resolves the freeze. But some times you can really be in
bad luck that the trick does not work.

Lucky enough, I captured the following log in a recent instance. It
appears to be a dead-lock when the netback tries to get some free
pages from NFS. I'm not sure if this is the whole story. Any
suggestion how to solve the issue?

Thanks,
Timothy

Apr 11 21:22:27 gaia kernel: [429242.015643] INFO: task netback/0:2255
blocked for more than 120 seconds.
Apr 11 21:22:27 gaia kernel: [429242.015665] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 11 21:22:27 gaia kernel: [429242.015690] netback/0       D
ffff880210213900     0  2255      2 0x00000000
Apr 11 21:22:27 gaia kernel: [429242.015693]  ffff8801fee04ea0
0000000000000246 0000000000000000 ffffffff818133f0
Apr 11 21:22:27 gaia kernel: [429242.015697]  0000000000013900
ffff8801fed87fd8 ffff8801fed87fd8 ffff8801fee04ea0
Apr 11 21:22:27 gaia kernel: [429242.015700]  ffff8801fed87488
ffff880210213900 ffff8801fee04ea0 ffff8801fed87488
Apr 11 21:22:27 gaia kernel: [429242.015703] Call Trace:
Apr 11 21:22:27 gaia kernel: [429242.015711]  [<ffffffff810c1bb5>] ?
__lock_page+0x66/0x66
Apr 11 21:22:27 gaia kernel: [429242.015715]  [<ffffffff814d06cb>] ?
io_schedule+0x55/0x6b
Apr 11 21:22:27 gaia kernel: [429242.015718]  [<ffffffff810c1bbc>] ?
sleep_on_page+0x7/0xc
Apr 11 21:22:27 gaia kernel: [429242.015720]  [<ffffffff814cf6c0>] ?
__wait_on_bit_lock+0x3c/0x85
Apr 11 21:22:27 gaia kernel: [429242.015723]  [<ffffffff810c3f7a>] ?
find_get_pages+0xea/0x100
Apr 11 21:22:27 gaia kernel: [429242.015726]  [<ffffffff810c1bb0>] ?
__lock_page+0x61/0x66
Apr 11 21:22:27 gaia kernel: [429242.015729]  [<ffffffff81058364>] ?
autoremove_wake_function+0x2a/0x2a
Apr 11 21:22:27 gaia kernel: [429242.015732]  [<ffffffff810cd110>] ?
truncate_inode_pages_range+0x28b/0x2f8
Apr 11 21:22:27 gaia kernel: [429242.015737]  [<ffffffff811c91d2>] ?
nfs_evict_inode+0x12/0x23
Apr 11 21:22:27 gaia kernel: [429242.015740]  [<ffffffff8111cdae>] ?
evict+0xa3/0x153
Apr 11 21:22:27 gaia kernel: [429242.015743]  [<ffffffff8111ce85>] ?
dispose_list+0x27/0x31
Apr 11 21:22:27 gaia kernel: [429242.015746]  [<ffffffff8111db6b>] ?
evict_inodes+0xe7/0xf4
Apr 11 21:22:27 gaia kernel: [429242.015749]  [<ffffffff8110b3af>] ?
generic_shutdown_super+0x3e/0xc5
Apr 11 21:22:27 gaia kernel: [429242.015752]  [<ffffffff8110b49e>] ?
kill_anon_super+0x9/0x11
Apr 11 21:22:27 gaia kernel: [429242.015755]  [<ffffffff811ca7b0>] ?
nfs_kill_super+0xd/0x16
Apr 11 21:22:27 gaia kernel: [429242.015758]  [<ffffffff8110b717>] ?
deactivate_locked_super+0x2c/0x5c
Apr 11 21:22:27 gaia kernel: [429242.015761]  [<ffffffff811c901d>] ?
__put_nfs_open_context+0xbf/0xe1
Apr 11 21:22:27 gaia kernel: [429242.015764]  [<ffffffff811d07db>] ?
nfs_commitdata_release+0x10/0x19
Apr 11 21:22:27 gaia kernel: [429242.015766]  [<ffffffff811d0f8c>] ?
nfs_initiate_commit+0xd9/0xe4
Apr 11 21:22:27 gaia kernel: [429242.015769]  [<ffffffff811d1bae>] ?
nfs_commit_inode+0x81/0x111
Apr 11 21:22:27 gaia kernel: [429242.015772]  [<ffffffff811c86f4>] ?
nfs_release_page+0x40/0x4f
Apr 11 21:22:27 gaia kernel: [429242.015775]  [<ffffffff810d0940>] ?
shrink_page_list+0x4f5/0x6d8
Apr 11 21:22:27 gaia kernel: [429242.015780]  [<ffffffff810d0f03>] ?
shrink_inactive_list+0x1dd/0x33f
Apr 11 21:22:27 gaia kernel: [429242.015783]  [<ffffffff810d15fa>] ?
shrink_lruvec+0x2e0/0x44d
Apr 11 21:22:27 gaia kernel: [429242.015787]  [<ffffffff810d17ba>] ?
shrink_zone+0x53/0x8a
Apr 11 21:22:27 gaia kernel: [429242.015790]  [<ffffffff810d1bcd>] ?
do_try_to_free_pages+0x1c6/0x3f4
Apr 11 21:22:27 gaia kernel: [429242.015794]  [<ffffffff810d20a3>] ?
try_to_free_pages+0xc4/0x11e
Apr 11 21:22:27 gaia kernel: [429242.015797]  [<ffffffff810c9018>] ?
__alloc_pages_nodemask+0x440/0x72f
Apr 11 21:22:27 gaia kernel: [429242.015801]  [<ffffffff810f592d>] ?
alloc_pages_current+0xb2/0xcd
Apr 11 21:22:27 gaia kernel: [429242.015805]  [<ffffffffa01be188>] ?
xen_netbk_alloc_page.isra.17+0x15/0x4d [xen_netback]
Apr 11 21:22:27 gaia kernel: [429242.015808]  [<ffffffffa01bf30c>] ?
xen_netbk_tx_build_gops+0x3fc/0x7ab [xen_netback]
Apr 11 21:22:27 gaia kernel: [429242.015812]  [<ffffffff812c0068>] ?
pnp_show_options+0x43e/0x482
Apr 11 21:22:27 gaia kernel: [429242.015816]  [<ffffffffa01beb3f>] ?
xen_netbk_rx_action+0x688/0x6f8 [xen_netback]
Apr 11 21:22:27 gaia kernel: [429242.015819]  [<ffffffff81061198>] ?
finish_task_switch+0x4c/0x7c
Apr 11 21:22:27 gaia kernel: [429242.015823]  [<ffffffffa01bf7e9>] ?
xen_netbk_kthread+0x12e/0x790 [xen_netback]
Apr 11 21:22:27 gaia kernel: [429242.015826]  [<ffffffff8105833a>] ?
abort_exclusive_wait+0x79/0x79
Apr 11 21:22:27 gaia kernel: [429242.015829]  [<ffffffffa01bf6bb>] ?
xen_netbk_tx_build_gops+0x7ab/0x7ab [xen_netback]
Apr 11 21:22:27 gaia kernel: [429242.015832]  [<ffffffffa01bf6bb>] ?
xen_netbk_tx_build_gops+0x7ab/0x7ab [xen_netback]
Apr 11 21:22:27 gaia kernel: [429242.015834]  [<ffffffff81057a31>] ?
kthread+0x67/0x6f
Apr 11 21:22:27 gaia kernel: [429242.015838]  [<ffffffff814d6904>] ?
kernel_thread_helper+0x4/0x10
Apr 11 21:22:27 gaia kernel: [429242.015841]  [<ffffffff814d143c>] ?
retint_restore_args+0x5/0x6
Apr 11 21:22:27 gaia kernel: [429242.015844]  [<ffffffff814d6900>] ?
gs_change+0x13/0x13

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel