[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

DOMU: Virtual Function FLR in PCI passthrough is crashing


  • To: "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • From: Naresh Bhat <nareshb@xxxxxxxxxxx>
  • Date: Wed, 20 Apr 2022 12:48:09 +0000
  • Accept-language: en-GB, en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tZ4XQfscFW4XNzWpat06473mKPO2/jW7rq0jSMlmmFA=; b=B9eTMSrnTo2tbA0vGQjmjBjy2mjMqJvhTaoeqSmJOvG/MHUP0GViF5pzmSaVPE2fKYQqfM2zpumw58bki6Gu8A1hNU+a72BWw3/oWCX1VDWGFcKLBDK772Z6WLAk6NOVvRnxdadVuEG8nq5aVv6QyRe66JgYECdkDALwU5KUj1bH3RmEuSMYjRR3q3pZ58430PekbhRYuGaemuCF0fVxohaLNFMfb+Ym7XsekPJ1QroCyhGsIYFzM212y1Z018JraotQwqpU+l7BiZpQJmYEhpcG3TjPqYC0WpWwmuvX01iHqwxmaXiovwop5sxLtRmrouhs87r0mZaSBe3a3Nl19g==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n8ZOIPNiWx5JW5GFsNa+m8hNtSz1o9ouA4C/xXk6mB+kccEQpZzmaZA2NXTwYg0KxjGEQ/04xFGdPD/3P2sJ7tzGQN1LTfPx+RA5DAf3fhxhbT9JSRksx676qMsEOn0C0NheX7DGrc100RqzKm4pMzzN9XE5uJsAYkfcgd9ELAhlwE9r5wYPcWP7vXHvYlVTwYcCTPThPe5i/YakVckmbDdIywueYF/06n7VvasBnFxaFZNX7MWHokyqaV1f3+x9RMrZI81B9OWJORjN4kaNcMCikLi1avIh4MNxlGiQTWoPZ4kkG7S8Ijxbu3bl6ubgObvxRzUcB+H11oFXMGzT/A==
  • Cc: "julien@xxxxxxx" <julien@xxxxxxx>, "sstabellini@xxxxxxxxxx" <sstabellini@xxxxxxxxxx>
  • Delivery-date: Wed, 20 Apr 2022 12:48:26 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Suggested_attachment_session_id: 3881f8c8-5b13-69d0-d21d-4a02acccd4cb
  • Thread-index: AQHYVLOBoc8zYi1khUW56WNBox6p/A==
  • Thread-topic: DOMU: Virtual Function FLR in PCI passthrough is crashing

Hi,

I have the following setup and try to test the Function Level Reset feature.  
Any suggestions or pointers will be very much helpful.

DOM0
Distribution: Ubuntu-20.04.3 (kernel 5.8.0-43)
Xen version : 4.11.4-pre

DOMU
Distribution: Ubuntu-18.04.6 LTS (kernel 5.8.0)
PCIe device with SRIOV support, VF (Virtual Function) interface connected to 
DOMU via PCI pass-through

Issue on DOMU: 
1. Enable MSIX on DOMU (We have used the following kernel APIs 
pci_enable_msix_range, pci_alloc_irq_vectors)
2. Execute FLR (Function Level Reset) via sysfs interface on the PCIe 
passthrough device in DOMU
   # echo "1" > /sys/bus/pci/devices/<ID>/reset

The following crash observed 

[ 4126.391455] BUG: unable to handle page fault for address: ffffc90040029000
[ 4126.391489] #PF: supervisor write access in kernel mode
[ 4126.391503] #PF: error_code(0x0003) - permissions violation
[ 4126.391516] PGD 94980067 P4D 94980067 PUD 16a155067 PMD 16a156067 PTE 
80100000a000c075
[ 4126.391537] Oops: 0003 [#1] SMP NOPTI
[ 4126.391550] CPU: 0 PID: 971 Comm: bash Tainted: G           OE     5.8.0 #1
[ 4126.391570] RIP: e030:__pci_write_msi_msg+0x59/0x150
[ 4126.391580] Code: 8b 50 d8 85 d2 75 31 83 78 fc 03 74 2b f6 47 54 01 74 6e 
f6 47 55 02 75 1f 0f b7 47 56 c1 e0 04 48 98 48 03 47 60 74 10 8b 16 <89> 10 8b 
56 04 89 50 04 8b 56 08 89 50 08 48 8b 03 49 89 44 24 20
[ 4126.391606] RSP: e02b:ffffc90040407cc0 EFLAGS: 00010286
[ 4126.391623] RAX: ffffc90040029000 RBX: ffff888164cfb120 RCX: 0000000000000000
[ 4126.391639] RDX: 0000000000000000 RSI: ffff888164cfb120 RDI: ffff888164cfb100
[ 4126.391653] RBP: ffffc90040407cf8 R08: 000053f2d6975617 R09: ffff888169c4e238
[ 4126.391672] R10: 0000000000000000 R11: ffffffff8266b248 R12: ffff888164cfb100
[ 4126.391688] R13: ffff88815e81c2e0 R14: ffff88815e81c130 R15: ffff8881648394a0
[ 4126.391723] FS:  00007f72b4b9b740(0000) GS:ffff88816ac00000(0000) 
knlGS:0000000000000000
[ 4126.391742] CS:  e030 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4126.391756] CR2: ffffc90040029000 CR3: 0000000167b10000 CR4: 0000000000040660
[ 4126.391781] Call Trace:
[ 4126.391799]  default_restore_msi_irq+0x38/0x70
[ 4126.391818]  default_restore_msi_irqs+0x2f/0x80
[ 4126.391836]  arch_restore_msi_irqs+0x15/0x20
[ 4126.391851]  pci_restore_msi_state+0xa1/0x230
[ 4126.391870]  pci_restore_state.part.0+0x319/0x440
[ 4126.391888]  pci_dev_restore+0x4a/0x60
[ 4126.391901]  pci_reset_function+0x4b/0x70
[ 4126.391915]  reset_store+0x5d/0xa0
[ 4126.391931]  dev_attr_store+0x17/0x30
[ 4126.391944]  sysfs_kf_write+0x3e/0x50
[ 4126.391958]  kernfs_fop_write+0xda/0x1b0
[ 4126.391973]  vfs_write+0xc9/0x200
[ 4126.391986]  ksys_write+0x67/0xe0
[ 4126.392002]  __x64_sys_write+0x1a/0x20
[ 4126.392018]  do_syscall_64+0x52/0xc0
[ 4126.392033]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 4126.392049] RIP: 0033:0x7f72b4277224
[ 4126.392064] Code: 89 02 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 
00 66 90 48 8d 05 c1 07 2e 00 8b 00 85 c0 75 13 b8 01 00 00 00 0f 05 <48> 3d 00 
f0 ff ff 77 54 f3 c3 66 90 41 54 55 49 89 d4 53 48 89 f5
[ 4126.392093] RSP: 002b:00007ffc5236f578 EFLAGS: 00000246 ORIG_RAX: 
0000000000000001
[ 4126.392114] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f72b4277224
[ 4126.392128] RDX: 0000000000000002 RSI: 0000562bde52f440 RDI: 0000000000000001
[ 4126.392140] RBP: 0000562bde52f440 R08: 000000000000000a R09: 0000000000000001
[ 4126.392155] R10: 000000000000000a R11: 0000000000000246 R12: 00007f72b4553760
[ 4126.392171] R13: 0000000000000002 R14: 00007f72b454f2a0 R15: 00007f72b454e760
[ 4126.392185] Modules linked in: <driver function> intel_rapl_msr 
intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel 
crypto_simd cryptd glue_helper rapl xen_pcifront sch_fq_codel parport_pc ppdev 
lp parport ip_tables x_tables autofs4
[ 4126.392228] CR2: ffffc90040029000
[ 4126.392238] ---[ end trace 23e8ad345e1ef956 ]---


Thanks and Regards
-Naresh Bhat


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.