[Xen-devel] VF passthrough problems

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>]

I am having reports of PCI passthrough problems with SR-IOV, in
particular it looks like we are hitting the warning at msi:635 and
msi:648 in xen; see the following log:

(XEN) [VT-D]iommu.c:1785: VT-d page table not sharing with EPT table
xenbus dev backend/vbd/1/5696 error 6 opening device
(XEN) PCI add Virtual Function 04:1a.0
(XEN) HVM1: HVM Loader
(XEN) HVM1: Detected Xen v4.1.0.rc2.pre
(XEN) HVM1: CPU speed is 2660 MHz
(XEN) HVM1: Xenbus rings @0xfeffc000, event channel 2
(XEN) irq.c:258: Dom1 PCI link 0 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:258: Dom1 PCI link 1 changed 0 -> 10
(XEN) HVM1: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:258: Dom1 PCI link 2 changed 0 -> 11
(XEN) HVM1: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:258: Dom1 PCI link 3 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 3 routed to IRQ5
(XEN) HVM1: pci dev 01:2 INTD->IRQ5
(XEN) HVM1: pci dev 01:3 INTA->IRQ10
(XEN) HVM1: pci dev 02:0 INTA->IRQ11
(XEN) HVM1: pci dev 03:0 INTC->IRQ10
(XEN) HVM1: pci dev 04:0 INTA->IRQ5
(XEN) HVM1: pci dev 02:0 bar 10 size 02000000: f0000008
(XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f2000008
(XEN) HVM1: pci dev 05:0 bar 10 size 00004000: f3000004
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=dd5a0 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3004 mfn=dd6a0 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3004 mfn=dd6a0 nr_mfns=1
(XEN) HVM1: pci dev 05:0 bar 1c size 00004000: f3004004
(XEN) HVM1: pci dev 02:0 bar 14 size 00001000: f3008000
(XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001
(XEN) HVM1: pci dev 04:0 bar 10 size 00000100: 0000c101
(XEN) HVM1: pci dev 04:0 bar 14 size 00000100: f3009000
(XEN) HVM1: pci dev 01:2 bar 20 size 00000020: 0000c201
(XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c221
(XEN) HVM1: Multiprocessor initialisation:
(XEN) HVM1:Â - CPU0 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ...
done.
(XEN) HVM1: Testing HVM environment:
(XEN) HVM1:Â - REP INSB across page boundaries ... passed
(XEN) HVM1:Â - GS base MSRs and SWAPGS ... passed
(XEN) HVM1: Passed 2 of 2 tests
(XEN) HVM1: Writing SMBIOS tables ...
(XEN) HVM1: Loading ROMBIOS ...
(XEN) HVM1: 9628 bytes of ROMBIOS high-memory extensions:
(XEN) HVM1:ÂÂ Relocating to 0xfc000000-0xfc00259c ... done
(XEN) HVM1: Creating MP tables ...
(XEN) HVM1: Loading Cirrus VGABIOS ...
(XEN) HVM1: Loading PCI Option ROM ...
(XEN) HVM1:Â - Manufacturer: http://etherboot.org
(XEN) HVM1:Â - Product name: gPXE
[root@localhost bug-report-20110217180212]# (XEN) HVM1: Loading ACPI ...
(XEN) HVM1:Â - Lo data: 000ea020-000ea04f
(XEN) HVM1:Â - Hi data: fc002800-fc01293f
(XEN) HVM1: vm86 TSS at fc012c00
(XEN) HVM1: BIOS map:
(XEN) HVM1:Â c0000-c8fff: VGA BIOS
(XEN) HVM1:Â c9000-d6fff: Etherboot ROM
(XEN) HVM1:Â eb000-eb193: SMBIOS tables
(XEN) HVM1:Â f0000-fffff: Main BIOS
(XEN) HVM1: E820 table:
(XEN) HVM1:Â [00]: 00000000:00000000 - 00000000:0009e000: RAM
(XEN) HVM1:Â [01]: 00000000:0009e000 - 00000000:0009fc00: RESERVED
(XEN) HVM1:Â [02]: 00000000:0009fc00 - 00000000:000a0000: RESERVED
(XEN) HVM1:Â HOLE: 00000000:000a0000 - 00000000:000e0000
(XEN) HVM1:Â [03]: 00000000:000e0000 - 00000000:00100000: RESERVED
(XEN) HVM1:Â [04]: 00000000:00100000 - 00000000:2ec00000: RAM
(XEN) HVM1:Â HOLE: 00000000:2ec00000 - 00000000:fc000000
(XEN) HVM1:Â [05]: 00000000:fc000000 - 00000001:00000000: RESERVED
(XEN) HVM1: Invoking ROMBIOS ...
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) stdvga.c:147:d1 entering stdvga and caching modes
(XEN) HVM1: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $
(XEN) HVM1: Processor 1: Xen(R) Virtual CPU
(XEN) HVM1: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM1: XS Virtual IDE ControllerÂÂÂÂÂ Hard Drive (8192MB)
(XEN) HVM1: IDE time out
(XEN) HVM1: XS Virtual ATAPI-4Â ÂÂÂÂÂÂÂÂÂÂÂCD-Rom/DVD-Rom
(XEN) HVM1:
(XEN) HVM1:
(XEN) HVM1:
(XEN) HVM1: Press F12 for boot menu.
(XEN) HVM1:
(XEN) HVM1: Boot device: CD-Romata_is_ready returned 1
(XEN) HVM1:Â - failure: could not read boot disk
(XEN) HVM1:
(XEN) HVM1: Boot device: Hard Drive - success.
(XEN) HVM1: Booting from 0000:7c00
(XEN) HVM1: int13_harddisk: function 41, unmapped device for ELDL=81
(XEN) HVM1: int13_harddisk: function 08, unmapped device for ELDL=81
(XEN) HVM1: *** int 15h function AX=00c0, BX=0000 not yet supported!
(XEN) HVM1: int13_harddisk: function 15, unmapped device for ELDL=81
(XEN) HVM1: KBD: unsupported int 16h function 03
(XEN) HVM1: int13_harddisk: function 15, unmapped device for ELDL=81
(XEN) HVM1: *** int 15h function AX=e980, BX=e6f5 not yet supported!
(XEN) HVM1: int13_harddisk: function 02, unmapped device for ELDL=81
(XEN) HVM1: int13_harddisk: function 41, unmapped device for ELDL=81
(XEN) irq.c:258: Dom1 PCI link 0 changed 5 -> 0
(XEN) irq.c:258: Dom1 PCI link 1 changed 10 -> 0
(XEN) irq.c:258: Dom1 PCI link 2 changed 11 -> 0
(XEN) irq.c:258: Dom1 PCI link 3 changed 5 -> 0
(XEN) irq.c:324: Dom1 callback via changed to PCI INTx Dev 0x03 IntC
(XEN) Xen WARN at msi.c:635
(XEN) ----[ Xen-4.1.0.rc2.pre x86_64 debug=y Not tainted ]----
(XEN) CPU:ÂÂÂ 7
(XEN) RIP:ÂÂÂ e008:[<ffff82c480161a08>] pci_enable_msi+0x464/0x960
(XEN) RFLAGS: 0000000000010206ÂÂ CONTEXT: hypervisor
(XEN) rax: 0000000000000000ÂÂ rbx: 00000000dd6a0000ÂÂ rcx: 0000000000000001
(XEN) rdx: ffff82c48025bc40 ÂÂrsi: 0000000000000246ÂÂ rdi: ffff82c48025bc40
(XEN) rbp: ffff830919aafdc8ÂÂ rsp: ffff830919aafcf8ÂÂ r8:Â 000000000000001a
(XEN) r9:Â 000000000000003fÂÂ r10: 0000000000000072ÂÂ r11: 000000000000001a
(XEN) r12: ffff830919aafe98ÂÂ r13: 0000000000000003ÂÂ r14: ffff83091996ce40
(XEN) r15: ffff830919aafe18ÂÂ cr0: 000000008005003bÂÂ cr4: 00000000000026f0
(XEN) cr3: 00000009199b0000ÂÂ cr2: 00000000e8da6fe8
(XEN) ds: 007bÂÂ es: 007bÂÂ fs: 00d8ÂÂ gs: 0033ÂÂ ss: 0000ÂÂ cs: e008
(XEN) Xen stack trace from rsp=ffff830919aafcf8:
(XEN)ÂÂÂ ffff82c48012422a ffff83092f508000 ffff830919aafd48 00ff82c400000003
(XEN)ÂÂÂ ffff830900000003 0000000000000010 0000000000000020 0000007000020056
(XEN)ÂÂÂ ffff830821aa9f60 0000000400000000 000000000000001a 0000000400000072
(XEN)ÂÂÂ 000000000000001a 00000000000dd6a0 00000000000dd6a0 00000000dd6a0000
(XEN)ÂÂÂ 00000000dd6a0000 ffff83092f46c4b8 0000000000000202 ffff830919aafda8
(XEN)ÂÂÂ ffff82c48012419f ffff830919aafe98 ffff830919708000 0000000000000056
(XEN)ÂÂÂ 000000000000005a 00000000ffffffed ffff830919aafe48 ffff82c480163850
(XEN)ÂÂÂ 0000000000000001 0000000000000000 0000000000000158 000000000000005a
(XEN)ÂÂÂ 0000000000000168 ffff830919c05a80 ffff83091996ce40 ffff830919aafe28
(XEN)ÂÂ Âffff82c48012419f ffff830919708000 ffff830919aafe98 000000000a0fa004
(XEN)ÂÂÂ 0000000000000056 000000000000005a ffff830919aafef8 ffff82c4802121d0
(XEN)ÂÂÂ ffff830919aafe68 ffff82c48020b5bf 0000000000000001 ffffffffffffffff
(XEN)ÂÂÂ 000000d000000004 dd6a000000000000 ffff830900000000 ffff8300ceeea1f8
(XEN)ÂÂÂ 000000d000000004 000000000000005a 00000000dd6a0000 aaaaaaaaaaaaaaaa
(XEN)ÂÂÂ aaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaa ffff8300ceeea000 ffff8300ceeea000
(XEN)ÂÂÂ 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)ÂÂÂ 00007cf6e65500c7 ffff82c480214ec1 00000000c0101427 0000000000000021
(XEN)ÂÂÂ 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)ÂÂÂ 00000000edab3ed4 000000000000000d 0000000000000000 0000000000000000
(XEN) ÂÂÂ0000000000000000 0000000000000000 0000000000000021 000000000a0fa004
(XEN) Xen call trace:
(XEN)ÂÂÂ [<ffff82c480161a08>] pci_enable_msi+0x464/0x960
(XEN)ÂÂÂ [<ffff82c480163850>] map_domain_pirq+0x20f/0x309
(XEN)ÂÂÂ [<ffff82c4802121d0>] compat_physdev_op+0x9e0/0x1420
(XEN)ÂÂÂ [<ffff82c480214ec1>] compat_hypercall+0xa1/0xfa
(XEN)
(XEN) Xen WARN at msi.c:648
(XEN) ----[ Xen-4.1.0.rc2.pre x86_64 debug=y Not tainted ]----
(XEN) CPU:ÂÂÂ 7
(XEN) RIP:ÂÂÂ e008:[<ffff82c480161a8c>] pci_enable_msi+0x4e8/0x960
(XEN) RFLAGS: 0000000000010246ÂÂ CONTEXT: hypervisor
(XEN) rax: 0000000000000000ÂÂ rbx: 0000000000002003ÂÂ rcx: 0000000000000001
(XEN) rdx: 0000000000000000ÂÂ rsi: 0000000000000246ÂÂ rdi: ffff82c48025bc40
(XEN) rbp: ffff830919aafdc8ÂÂ rsp: ffff830919aafcf8ÂÂ r8:Â 000000000000001a
(XEN) r9:Â 000000000000003fÂÂ r10: 0000000000000072ÂÂ r11: 000000000000001a
(XEN) r12: ffff830919aafe98ÂÂ r13: 0000000000000003ÂÂ r14: ffff83091996ce40
(XEN) r15: ffff830919aafe18ÂÂ cr0: 000000008005003bÂÂ cr4: 00000000000026f0
(XEN) cr3: 00000009199b0000ÂÂ cr2: 00000000e8da6fe8
(XEN) ds: 007bÂÂ es: 007bÂÂ fs: 00d8ÂÂ gs: 0033ÂÂ ss: 0000ÂÂ cs: e008
(XEN) Xen stack trace from rsp=ffff830919aafcf8:
(XEN)ÂÂÂ ffff82c48012422a ffff83092f508000 ffff830919aafd48 00ff82c400000003
(XEN)ÂÂÂ ffff830900000003 0000000000000010 0000000000000020 0000007000020056
(XEN)ÂÂÂ ffff830821aa9f60 0000000400000000 000000000000001a 0000000400000072
(XEN)ÂÂÂ 000000000000001a 00000000000dd6a0 00000000000dd6a0 00000000dd6a0000
(XEN)ÂÂÂ 00000000dd6a0000 ffff83092f46c4b8 0000000000000202 ffff830919aafda8
(XEN)ÂÂÂ ffff82c48012419f ffff830919aafe98 ffff830919708000 0000000000000056
(XEN)ÂÂÂ 000000000000005a 00000000ffffffed ffff830919aafe48 ffff82c480163850
(XEN)ÂÂÂ 0000000000000001 0000000000000000 0000000000000158 000000000000005a
(XEN)ÂÂÂ 0000000000000168 ffff830919c05a80 ffff83091996ce40 ffff830919aafe28
(XEN)ÂÂÂ ffff82c48012419f ffff830919708000 ffff830919aafe98 000000000a0fa004
(XEN)ÂÂÂ 0000000000000056 000000000000005a ffff830919aafef8 ffff82c4802121d0
(XEN)ÂÂÂ ffff830919aafe68 ffff82c48020b5bf 0000000000000001 ffffffffffffffff
(XEN)ÂÂÂ 000000d000000004 dd6a000000000000 ffff830900000000 ffff8300ceeea1f8
(XEN)ÂÂÂ 000000d000000004 000000000000005a 00000000dd6a0000 aaaaaaaaaaaaaaaa
(XEN)ÂÂÂ aaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaa ffff8300ceeea000 ffff8300ceeea000
(XEN)ÂÂÂ 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)ÂÂÂ 00007cf6e65500c7 ffff82c480214ec1 00000000c0101427 0000000000000021
(XEN)ÂÂÂ 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)ÂÂÂ 00000000edab3ed4 000000000000000d 0000000000000000 0000000000000000
(XEN)ÂÂÂ 0000000000000000 0000000000000000 0000000000000021 000000000a0fa004
(XEN) Xen call trace:
(XEN)ÂÂÂ [<ffff82c480161a8c>] pci_enable_msi+0x4e8/0x960
(XEN)ÂÂÂ [<ffff82c480163850>] map_domain_pirq+0x20f/0x309
(XEN)ÂÂÂ [<ffff82c4802121d0>] compat_physdev_op+0x9e0/0x1420
(XEN)ÂÂÂ [<ffff82c480214ec1>] compat_hypercall+0xa1/0xfa
(XEN)


Similar problems have been reported also in the bugzilla bug 1742:

http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1742

Do you have any ideas on the cause of the issue?

Cheers,

Stefano

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel