[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH] drivers/net/netfront: Fix NULL sring after live migration


  • To: Jakub Kicinski <kuba@xxxxxxxxxx>
  • From: Lin Liu (刘林) <lin.liu@xxxxxxxxxx>
  • Date: Thu, 1 Dec 2022 08:41:35 +0000
  • Accept-language: zh-CN, en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IqH9dux42rt1NMmnxoHxWz1BJpDwzaRegLFfvWRjWOs=; b=cSFlEwOjNOlJsRQsFksdtxcD4LEymhXEgizJovgex4TlZ8ziuVAR6eggZaKMSCebLBraEq/LZADH7Ay+SjN1SwJitGrjkflFU/8sQVhgpMnoaDGcVubQ2HY0oKlKt8EFxubTuAK4QHzJw5UUPphmw2tysQ9StkdYdgBrbi3k7fAN1HK3egW89XCJK0lKLL52JCm8gtmdZGS4hnR4yxV/Ovc0JgdLgzmdS0YhE1dJ5FNYfOzA65A+hQUyqMAv7lb36e3rKQ81AkMjEcV7OGOgNt1RlMfxD6YOvqUxYeN4GyK+mpDXie8q36PobUAZbs9EV81tQZy70hE4iNkKzYPsvQ==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QWzXsxVMquf8wZyAvgcaLI/gvDXGPD2LkaDRvDzdIdkv42Hqhbxq1IsdUaA/iX8YfmEGddcPPp1h3S83klCz0UE4xaolwfGTOgkK5avhWZYle3nhb4U+x5NosQaOusrjYGPXOT85ucUm25RFYMOTNg6EwBAEZ/DvZhdKnDM8keYJQnkJhSNbAHF0/eCtbJlqsOVAbDx53Rwic8mtODCMKhKKRBYiE4Ocy8/4RnIQGwa+ngRKbI16IpeNbEtfyta/IzGzuWfExEalE2fdil5KVkan57+yBdqvRpUYkHWHpk9d/8wF+p4qghbGKxxT1WH7rZ7bXizChYN1+0i1V2kHgA==
  • Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=citrix.com;
  • Cc: Juergen Gross <jgross@xxxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Oleksandr Tyshchenko <oleksandr_tyshchenko@xxxxxxxx>, "David S. Miller" <davem@xxxxxxxxxxxxx>, Eric Dumazet <edumazet@xxxxxxxxxx>, Paolo Abeni <pabeni@xxxxxxxxxx>, "moderated list:XEN HYPERVISOR INTERFACE" <xen-devel@xxxxxxxxxxxxxxxxxxxx>, "open list:NETWORKING DRIVERS" <netdev@xxxxxxxxxxxxxxx>, open list <linux-kernel@xxxxxxxxxxxxxxx>
  • Delivery-date: Thu, 01 Dec 2022 08:41:57 +0000
  • Ironport-data: A9a23:RBc8n6DMPgSGzxVW/2Hjw5YqxClBgxIJ4kV8jS/XYbTApD5z0mMCz mobUGHTO//fNzP1L492YY3n/RwHv5PQxt5jQQY4rX1jcSlH+JHPbTi7wuUcHAvJd5GeExg3h yk6QoOdRCzhZiaE/n9BCpC48T8nk/nNHuCnYAL9EngZbRd+Tys8gg5Ulec8g4p56fC0GArIs t7pyyHlEAbNNwVcbyRFtcpvlDs15K6o4WlA5wRnDRx2lAS2e0c9Xcp3yZ6ZdxMUcqEMdsamS uDKyq2O/2+x13/B3fv8z94X2mVTKlLjFVDmZkh+AsBOsTAbzsAG6Y4pNeJ0VKtio27hc+ada jl6ncfYpQ8BZsUgkQmGOvVSO3kW0aZuoNcrLZUj2CA6IoKvn3bEmp1T4E8K0YIw8McvHkpgx +ckMHMAaRCbmLKU5+KaVbw57igjBJGD0II3nFhFlGicJ9B2BJfJTuPN+MNS2yo2ioZWB/HCa sEFaD1pKhPdfxlIPVRRA5U79AuqriCnL3sE9xTI9exuvTi7IA9ZidABNPLRe9rMQcRItk2Zu njH7yLyBRRy2Nm3mWfdrij23rCncSXTRr45BOKx8cFTpnKY+0VPCDosSUWCiKzs4qK5c5cFQ 6AOwQItsKI73E+mVN/wW1u/unHslhwTQd1LCMUh9RqAjKHT5m6xHXMYRzRMbNgnss4eRjEw0 FKN2dTzClRHt7mYSmmU8LuOmi+jIigeLWIEZigsQBMM5p/op4Rbph7GUNtmOKK8kNDwHTb+3 3aMoTRWr7gfkMsCxain1V/AhD2oq97CSQtdzh3eVG+j7ytjaYKlbpDu4l/ehd5QIYDcQlSfs Xwsn8mF8PtIHZyLjDaKQugGAPeu/fntGDvRnVJiAbE/+Dm3vX2uZ4Zd5Hd5PkgBGsIFfyL5J UzeowVc4LdNM3awK6x6eYS8D4It16eIKDj+fvXdb94LaJ4hcgaCpXhqfRTJgTCrl1Uwm6YiP 5vdadyrEXsRFaVgynyxWvsZ1rgogCs5wAs/WKzG8vhu6pLGDFb9dFvPGAHmgjwRhE9cnDjoz g==
  • Ironport-hdrordr: A9a23:t2uKUKB0ge6nwAvlHeg3sseALOsnbusQ8zAXPh9KOHlom52j5q KTdaogpHzJYW4qKQIdcLC7WZVpQRvnhOVICPoqTMWftW7dyS6VxeBZnMnfKljbcFybygcz79 YeT0ETMqyIMbE+t7eL3ODaKadl/DDkytHXuQ629R4EJ28LGsYQiTuRSDzrY3GeLzM2f6bRYa Dsmvav0ADQBEj/AP7LfkUtbqzugOLqufvdDCIuNloG5hWHni/t0ITbPXGjsSs2Ynduz6sr4n SAqwz1j5/Oj9iLjjvnk0PD5ZVfn9XsjvNFGcy3k8AQbhn8lwqyY4xlerua+BQ4uvum5loGmM TF505IBbU011rhOkWO5Tf90Qjp1zgjr1fk1F+jmHPm5eDpWT4gDMJFpIRBNj/U8VApst1Q2L 9Cm0iZq51UJxXdmzmV3amOazha0m6P5VYym+8aiHJSFaEEbqVKkIAZ9ERJVL8dASPT8ukcYb FTJfCZwMwTXUKRbnjfsGUq6serRG4PEhCPRVVHktCJ0gJRgGtywyIjtZcidzY7hcgAoqt/lr j525dT5e5zpwgtHPxA7dI6MJaK46r2MEPx2SypUATa/eo8SgfwQtbMkcEIDaeRCcU1JdIJ6e P8eUIdumgod030D8qSmJVN7xDWWW24GS/g08dE+vFCy/zBrRXQQGa+oX0V4oSdSs8kc4Pmcu f2PJJdGPn4K2TyXYpZ1Qz3XJdfbXQfWMMTvMwnW1iV5t7NN5b3rPHWfZ/oVfDQOCdhXnm6Dm oIXTD1KskF5ke3WmXgiByUX3/2YETw8Z95DaCfpoEoucswH5wJthJQhUWy58mNJzEHuqsqfF FmKLeilq+gv2G58WvB8m0sMBtACURe5qnmThpx1E83GlKxdaxGt8SUeGhU0nfCLhhjT9nOGA oavFhz8bLfFe3d+cnjMaPrDou3tQpamJvRdeZrpkSq37atRq8F
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Thread-index: AQHZA7o3+GR9FRj8kECyz/57PLyGOK5Yg4yAgAA1/qA=
  • Thread-topic: [PATCH] drivers/net/netfront: Fix NULL sring after live migration

>[CAUTION - EXTERNAL EMAIL] DO NOT reply, click links, or open attachments unless you have verified the sender and know the content is safe.

> 

>On Tue, 29 Nov 2022 06:17:02 +0000 Lin Liu wrote:

>> A NAPI is setup for each network sring to poll data to kernel

>> The sring with source host is destroyed before live migration and

>> new sring with target host is setup after live migration.

>> The NAPI for the old sring is not deleted until setup new sring

>> with target host after migration. With busy_poll/busy_read enabled,

>> the NAPI can be polled before got deleted when resume VM.

>>

>> [50116.602938] BUG: unable to handle kernel NULL pointer dereference at

>> 0000000000000008

>> [50116.603047] IP: xennet_poll+0xae/0xd20

>> [50116.603090] PGD 0 P4D 0

>> [50116.603118] Oops: 0000 [#1] SMP PTI

>> [50116.604624] Call Trace:

>> [50116.604674]  ? finish_task_switch+0x71/0x230

>> [50116.604745]  ? timerqueue_del+0x1d/0x40

>> [50116.604807]  ? hrtimer_try_to_cancel+0xb5/0x110

>> [50116.604882]  ? xennet_alloc_rx_buffers+0x2a0/0x2a0

>> [50116.604958]  napi_busy_loop+0xdb/0x270

>> [50116.605017]  sock_poll+0x87/0x90

>> [50116.605066]  do_sys_poll+0x26f/0x580

>> [50116.605125]  ? tracing_map_insert+0x1d4/0x2f0

>> [50116.605196]  ? event_hist_trigger+0x14a/0x260

> 

>You can trim all the ' ? ' entries from the stack trace,

>and the time stamps, FWIW. Makes it easier to read.

 

Sure, will do in next version

 

>> [50116.613598]  ? finish_task_switch+0x71/0x230

>> [50116.614131]  ? __schedule+0x256/0x890

>> [50116.614640]  ? recalc_sigpending+0x1b/0x50

>> [50116.615144]  ? xen_sched_clock+0x15/0x20

>> [50116.615643]  ? __rb_reserve_next+0x12d/0x140

>> [50116.616138]  ? ring_buffer_lock_reserve+0x123/0x3d0

>> [50116.616634]  ? event_triggers_call+0x87/0xb0

>> [50116.617138]  ? trace_event_buffer_commit+0x1c4/0x210

>> [50116.617625]  ? xen_clocksource_get_cycles+0x15/0x20

>> [50116.618112]  ? ktime_get_ts64+0x51/0xf0

>> [50116.618578]  SyS_ppoll+0x160/0x1a0

>> [50116.619029]  ? SyS_ppoll+0x160/0x1a0

>> [50116.619475]  do_syscall_64+0x73/0x130

>> [50116.619901]  entry_SYSCALL_64_after_hwframe+0x41/0xa6

>> ...

>> [50116.806230] RIP: xennet_poll+0xae/0xd20 RSP: ffffb4f041933900

>> [50116.806772] CR2: 0000000000000008

>> [50116.807337] ---[ end trace f8601785b354351c ]---

>>

>> xen frontend should remove the NAPIs for the old srings before live

>> migration as the bond srings are destroyed

>>

>> There is a tiny window between the srings are set to NULL and

>> the NAPIs are disabled, It is safe as the NAPI threads are still

>> frozen at that time

>>

> 

>Since this is a fix please add a Fixes tag, and add [PATCH net]

>to the subject.

> 

 

Will do in next version

 

>> diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c

>> index 9af2b027c19c..dc404e05970c 100644

>> --- a/drivers/net/xen-netfront.c

>> +++ b/drivers/net/xen-netfront.c

>> @@ -1862,6 +1862,12 @@ static int netfront_resume(struct xenbus_device *dev)

>>        netif_tx_unlock_bh(info->netdev);

>> 

>>        xennet_disconnect_backend(info);

>> +

>> +     rtnl_lock();

>> +     if (info->queues)

>> +             xennet_destroy_queues(info);

>> +     rtnl_unlock();

 

>Now all callers of xennet_disconnect_backend() destroy queues soon

>after, can we just move the destroy queues into disconnect ?

 

After the sring is destroyed, the queue and the bond NAPI should also be destroyed,

so Yes, destroy queues can be part of xennet_disconnect_backend,

However, some caller of xennet_disconnect_backend hold rtnl_lock while some others

not, I think it is simpler to keep seperate.

> 

>>        return 0;

>>  }

>> 


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.