Re: [Xen-users] NIC PCI-Passthrough problems (PV domU driver crashes)

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Tue, 2013-08-06 at 01:07 +0530, Mark Jones wrote:
> Hi,
> 
> 
> I have been trying to get NIC PCI Passthrough to work on Xen 4.3.0 and
> have hit a wall. On attaching the NIC the domU network driver (igb)
> crashes.

Is 4.3.0 the only version you have tried or did it used to work on an
older Xen?

> root@dom0:~# xl -f pci-attach 54 0000:05:00.1,permissive=1

What is the -f for? Are you working around some other issue?

Does it work if you list the device in the config file, i.e. do a cold
plug? (isolating to a hotplug only vs. a general issue is useful)

It might also be interesting to disable MSI for the guest, there's a
guest kernel commandline option I think, not sure what it is though,
maybe pci=nomsi or perhaps just nomsi?

I assume this device works fine if you leave it to dom0? What about if
you boot the domU 3.2 kernel natively on the host? (I think this is
unlikely to be a driver bug, but it is worth ruling it out).

Google doesn't show up any relevant looking results for
msix_program_entries or msix_capability_init. I'm not really sure how
MSIs work with PCI passthrough, it might be worth escalating this to the
-devel list.

> Aug  5 15:05:17 ubuntu kernel: [  139.781460] Intel(R) Gigabit Ethernet 
> Network Driver - version 3.2.10-k
> Aug  5 15:05:17 ubuntu kernel: [  139.781465] Copyright (c) 2007-2011 Intel 
> Corporation.
> Aug  5 15:05:17 ubuntu kernel: [  139.781534] igb 0000:00:00.1: enabling 
> device (0000 -> 0002)
> Aug  5 15:05:17 ubuntu kernel: [  139.781688] igb 0000:00:00.1: Xen PCI 
> mapped GSI42 to IRQ27
> Aug  5 15:05:17 ubuntu kernel: [  139.781816] igb 0000:00:00.1: setting 
> latency timer to 64
> Aug  5 15:05:17 ubuntu kernel: [  139.793427] BUG: unable to handle kernel 
> paging request at ffffc9000031800c
> Aug  5 15:05:17 ubuntu kernel: [  139.793442] IP: [<ffffffff8134b30b>] 
> msix_program_entries+0x7b/0xa0
> Aug  5 15:05:17 ubuntu kernel: [  139.793457] PGD 77e1c067 PUD 77e1d067 PMD 
> 77f23067 PTE 80100000fa3dc465
> Aug  5 15:05:17 ubuntu kernel: [  139.793471] Oops: 0003 [#1] SMP 
> Aug  5 15:05:17 ubuntu kernel: [  139.793479] CPU 0 
> Aug  5 15:05:17 ubuntu kernel: [  139.793483] Modules linked in: igb(+) dca 
> xen_gntdev xen_evtchn xen_pcifront xenfs ext2 lp parport
> Aug  5 15:05:17 ubuntu kernel: [  139.793502] 
> Aug  5 15:05:17 ubuntu kernel: [  139.793507] Pid: 1163, comm: modprobe Not 
> tainted 3.2.0-51-generic #77-Ubuntu  
> Aug  5 15:05:17 ubuntu kernel: [  139.793517] RIP: e030:[<ffffffff8134b30b>]  
> [<ffffffff8134b30b>] msix_program_entries+0x7b/0xa0
> Aug  5 15:05:17 ubuntu kernel: [  139.793528] RSP: e02b:ffff880076c77aa8  
> EFLAGS: 00010286
> Aug  5 15:05:17 ubuntu kernel: [  139.793534] RAX: ffffc9000031800c RBX: 
> ffff8800771b2ec0 RCX: 00000000fffffffa
> Aug  5 15:05:17 ubuntu kernel: [  139.793542] RDX: 0000000000000001 RSI: 
> ffff880075550200 RDI: ffff880075550200
> Aug  5 15:05:17 ubuntu kernel: [  139.793549] RBP: ffff880076c77ad8 R08: 
> ffff880075558000 R09: ffff880078c000f8
> Aug  5 15:05:17 ubuntu kernel: [  139.793556] R10: 0000000000000000 R11: 
> 0000000000000000 R12: 0000000000000000
> Aug  5 15:05:17 ubuntu kernel: [  139.793564] R13: ffffc9000031800c R14: 
> ffff880074ee3918 R15: ffff880076161800
> Aug  5 15:05:17 ubuntu kernel: [  139.793576] FS:  00007fa3fcb91700(0000) 
> GS:ffff88007fc00000(0000) knlGS:0000000000000000
> Aug  5 15:05:17 ubuntu kernel: [  139.793584] CS:  e033 DS: 0000 ES: 0000 
> CR0: 000000008005003b
> Aug  5 15:05:17 ubuntu kernel: [  139.793592] CR2: ffff8000007d1ee0 CR3: 
> 0000000076f1a000 CR4: 0000000000002660
> Aug  5 15:05:17 ubuntu kernel: [  139.793600] DR0: 0000000000000000 DR1: 
> 0000000000000000 DR2: 0000000000000000
> Aug  5 15:05:17 ubuntu kernel: [  139.793608] DR3: 0000000000000000 DR6: 
> 00000000ffff0ff0 DR7: 0000000000000400
> Aug  5 15:05:17 ubuntu kernel: [  139.793616] Process modprobe (pid: 1163, 
> threadinfo ffff880076c76000, task ffff880003d4ae00)
> Aug  5 15:05:17 ubuntu kernel: [  139.793625] Stack:
> Aug  5 15:05:17 ubuntu kernel: [  139.793629]  0000000000000062 
> ffff880074ee3000 0000000000000000 0000000000000003
> Aug  5 15:05:17 ubuntu kernel: [  139.793641]  ffff880076161800 
> 0000000000000062 ffff880076c77b48 ffffffff8134b501
> Aug  5 15:05:17 ubuntu kernel: [  139.793654]  0000000000000002 
> ffff880074ee3918 0000000274ee3000 ffffc90000318000
> Aug  5 15:05:17 ubuntu kernel: [  139.793666] Call Trace:
> Aug  5 15:05:17 ubuntu kernel: [  139.793674]  [<ffffffff8134b501>] 
> msix_capability_init+0x1d1/0x260
> Aug  5 15:05:17 ubuntu kernel: [  139.793683]  [<ffffffff8134c31a>] 
> pci_enable_msix.part.11+0x9a/0x100
> Aug  5 15:05:17 ubuntu kernel: [  139.793692]  [<ffffffff8134c3c6>] 
> pci_enable_msix+0x46/0x60
> Aug  5 15:05:17 ubuntu kernel: [  139.793705]  [<ffffffffa0051688>] 
> igb_set_interrupt_capability+0x1b8/0x1d0 [igb]
> Aug  5 15:05:17 ubuntu kernel: [  139.793720]  [<ffffffffa0054448>] 
> igb_init_interrupt_scheme+0x28/0xf0 [igb]
> Aug  5 15:05:17 ubuntu kernel: [  139.793734]  [<ffffffffa00634c5>] 
> igb_sw_init+0x182/0x1d6 [igb]
> Aug  5 15:05:17 ubuntu kernel: [  139.793747]  [<ffffffffa00638a8>] 
> igb_probe+0x38f/0xa21 [igb]
> Aug  5 15:05:17 ubuntu kernel: [  139.793758]  [<ffffffff816606ee>] ? 
> _raw_spin_unlock_irqrestore+0x1e/0x30
> Aug  5 15:05:17 ubuntu kernel: [  139.793767]  [<ffffffff816606ee>] ? 
> _raw_spin_unlock_irqrestore+0x1e/0x30
> Aug  5 15:05:17 ubuntu kernel: [  139.793778]  [<ffffffff81337bdc>] 
> local_pci_probe+0x5c/0xd0
> Aug  5 15:05:17 ubuntu kernel: [  139.793787]  [<ffffffff813394d9>] 
> __pci_device_probe+0xf9/0x100
> Aug  5 15:05:17 ubuntu kernel: [  139.793796]  [<ffffffff813103fa>] ? 
> kobject_get+0x1a/0x30
> Aug  5 15:05:17 ubuntu kernel: [  139.793804]  [<ffffffff8133951a>] 
> pci_device_probe+0x3a/0x60
> Aug  5 15:05:17 ubuntu kernel: [  139.793814]  [<ffffffff813f9658>] 
> really_probe+0x68/0x190
> Aug  5 15:05:17 ubuntu kernel: [  139.793822]  [<ffffffff813f98e5>] 
> driver_probe_device+0x45/0x70
> Aug  5 15:05:17 ubuntu kernel: [  139.793830]  [<ffffffff813f99bb>] 
> __driver_attach+0xab/0xb0
> Aug  5 15:05:17 ubuntu kernel: [  139.793838]  [<ffffffff813f9910>] ? 
> driver_probe_device+0x70/0x70
> Aug  5 15:05:17 ubuntu kernel: [  139.793847]  [<ffffffff813f9910>] ? 
> driver_probe_device+0x70/0x70
> Aug  5 15:05:17 ubuntu kernel: [  139.793856]  [<ffffffff813f8744>] 
> bus_for_each_dev+0x64/0xa0
> Aug  5 15:05:17 ubuntu kernel: [  139.793864]  [<ffffffff813f941e>] 
> driver_attach+0x1e/0x20
> Aug  5 15:05:17 ubuntu kernel: [  139.793872]  [<ffffffff813f9070>] 
> bus_add_driver+0x1a0/0x270
> Aug  5 15:05:17 ubuntu kernel: [  139.793880]  [<ffffffffa006f000>] ? 
> 0xffffffffa006efff
> Aug  5 15:05:17 ubuntu kernel: [  139.793888]  [<ffffffff813f9f26>] 
> driver_register+0x76/0x140
> Aug  5 15:05:17 ubuntu kernel: [  139.793898]  [<ffffffff81647ccc>] ? 
> printk+0x51/0x53
> Aug  5 15:05:17 ubuntu kernel: [  139.793906]  [<ffffffffa006f000>] ? 
> 0xffffffffa006efff
> Aug  5 15:05:17 ubuntu kernel: [  139.793914]  [<ffffffff813391b6>] 
> __pci_register_driver+0x56/0xd0
> Aug  5 15:05:17 ubuntu kernel: [  139.793927]  [<ffffffffa006f05b>] 
> igb_init_module+0x5b/0x1000 [igb]
> Aug  5 15:05:17 ubuntu kernel: [  139.793938]  [<ffffffff81002040>] 
> do_one_initcall+0x40/0x180
> Aug  5 15:05:17 ubuntu kernel: [  139.793948]  [<ffffffff810a9eae>] 
> sys_init_module+0xbe/0x230
> Aug  5 15:05:17 ubuntu kernel: [  139.793958]  [<ffffffff81668d02>] 
> system_call_fastpath+0x16/0x1b
> Aug  5 15:05:17 ubuntu kernel: [  139.793965] Code: c1 e5 04 41 83 c5 0c e8 
> 34 20 d9 ff 4d 63 ed 4c 03 6b 20 41 8b 55 00 0f b7 43 02 89 53 08 83 ca 01 c1 
> e0 04 83 c0 0c 48 03 43 20 <89> 10 48 8b 43 10 41 83 c4 01 89 53 08 49 39 c6 
> 48 8d 58 f0 75 
> Aug  5 15:05:17 ubuntu kernel: [  139.794041] RIP  [<ffffffff8134b30b>] 
> msix_program_entries+0x7b/0xa0
> Aug  5 15:05:17 ubuntu kernel: [  139.794051]  RSP <ffff880076c77aa8>
> Aug  5 15:05:17 ubuntu kernel: [  139.794055] CR2: ffffc9000031800c
> Aug  5 15:05:17 ubuntu kernel: [  139.794061] ---[ end trace dea4e19be79d2d73 
> ]---
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxx
> http://lists.xen.org/xen-users



_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users