[Top] [All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] IOMMU fault with IGD passthrough setup on XEN 4.8.0

Hi all,
I have a working IGD passthrough setup running for 4 years on XEN 4.3.2.
And it no longer works after I upgraded to XEN4.8.0 yesterday. Really need suggestions this time.

My previous setup was built upon some local fixes in qemu-xen-traditional (for vendor specific pci cap).
With the same set of patches, I'm seeing hanging Linux domU and XEN dmesg flooded with IOMMU fault message in version 4.8.0.
I haven't got chance to try out with a stock build, but the chance of relevance is pretty low to me.
It could be a security related change (like the 'rdm_policy=relaxed' config change), but I really have no idea.
(BTW, I also tried the new qemu-upstream device model, but it doesn't work either.)

Here are some relevant logs, please help comment what's going on here and what's the next step of diagnose.
It appears that the fault address 0xcfxxxxxx falls within the host RMRR region.
However, the hvmloader is setting up memory region starting from address 0xe0000000.
Is the hvmloader memory map relevant here?
Unfortunately the iommu.c does not provide detailed log on the mapping except a simple 'd2:PCI: map 0000:00:02.0'

Thanks,
G.R.

Errors look like this:
(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr cfa57000, iommu reg = ffff82c000201000
(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
(XEN) print_vtd_entries: iommu ffff8304152ec600 dev 0000:00:02.0 gmfn cfa57
(XEN)     root_entry = ffff8304152e9000
(XEN)     root_entry[0] = 251bcb001
(XEN)     context = ffff830251bcb000
(XEN)     context[10] = 1_2215f6001
(XEN)     l3 = ffff8302215f6000
(XEN)     l3_index = 3
(XEN)     l3[3] = 221476003
(XEN)     l2 = ffff830221476000
(XEN)     l2_index = 7d
(XEN)     l2[7d] = 0
(XEN)     l2[7d] not present

(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr cfa70000, iommu reg = ffff82c000201000
(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
(XEN) print_vtd_entries: iommu ffff8304152ec600 dev 0000:00:02.0 gmfn cfa70
(XEN)     root_entry = ffff8304152e9000
(XEN)     root_entry[0] = 251bcb001
(XEN)     context = ffff830251bcb000
(XEN)     context[10] = 1_2215f6001
(XEN)     l3 = ffff8302215f6000
(XEN)     l3_index = 3
(XEN)     l3[3] = 221476003
(XEN)     l2 = ffff830221476000
(XEN)     l2_index = 7d
(XEN)     l2[7d] = 0
(XEN)     l2[7d] not present

From xl dmesg:
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009d800 (usable)
(XEN)  000000000009d800 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000020000000 (usable)
(XEN)  0000000020000000 - 0000000020200000 (reserved)
(XEN)  0000000020200000 - 0000000040004000 (usable)
(XEN)  0000000040004000 - 0000000040005000 (reserved)
(XEN)  0000000040005000 - 00000000cd0b9000 (usable)
(XEN)  00000000cd0b9000 - 00000000cd881000 (reserved)
(XEN)  00000000cd881000 - 00000000cd90d000 (usable)
(XEN)  00000000cd90d000 - 00000000cd9ae000 (ACPI NVS)
(XEN)  00000000cd9ae000 - 00000000ce180000 (reserved)
(XEN)  00000000ce180000 - 00000000ce181000 (usable)
(XEN)  00000000ce181000 - 00000000ce1c4000 (ACPI NVS)
(XEN)  00000000ce1c4000 - 00000000cec19000 (usable)
(XEN)  00000000cec19000 - 00000000ceff2000 (reserved)
(XEN)  00000000ceff2000 - 00000000cf000000 (usable)
(XEN)  00000000cf800000 - 00000000dfa00000 (reserved)
(XEN)  00000000f8000000 - 00000000fc000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fed00000 - 00000000fed04000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000041f600000 (usable)

(XEN) [VT-D]Host address width 36
(XEN) [VT-D]found ACPI_DMAR_DRHD:
(XEN) [VT-D]  dmaru->address = fed90000
(XEN) [VT-D]drhd->address = fed90000 iommu->reg = ffff82c000201000
(XEN) [VT-D]cap = c0000020e60262 ecap = f0101a
(XEN) [VT-D] endpoint: 0000:00:02.0
(XEN) [VT-D]found ACPI_DMAR_DRHD:
(XEN) [VT-D]  dmaru->address = fed91000
(XEN) [VT-D]drhd->address = fed91000 iommu->reg = ffff82c000203000
(XEN) [VT-D]cap = c9008020660262 ecap = f0105a
(XEN) [VT-D] IOAPIC: 0000:f0:1f.0
(XEN) [VT-D] MSI HPET: 0000:f0:0f.0
(XEN) [VT-D]  flags: INCLUDE_ALL
(XEN) [VT-D]found ACPI_DMAR_RMRR:
(XEN) [VT-D] endpoint: 0000:00:1d.0
(XEN) [VT-D] endpoint: 0000:00:1a.0
(XEN) [VT-D] endpoint: 0000:00:14.0
(XEN) [VT-D]  RMRR region: base_addr cd7ea000 end_address cd814fff
(XEN) [VT-D]found ACPI_DMAR_RMRR:
(XEN) [VT-D] endpoint: 0000:00:02.0
(XEN) [VT-D]  RMRR region: base_addr cf800000 end_address df9fffff


For the failing domU (dom2):
(XEN) d2: bind: m_gsi=16 g_gsi=24 dev=00.00.2 intx=0
(XEN) [VT-D]d0:PCI: unmap 0000:00:02.0
(XEN) [VT-D]d2:PCI: map 0000:00:02.0
(XEN) d2: bind: m_gsi=22 g_gsi=36 dev=00.00.5 intx=0
(XEN) [VT-D]d0:PCIe: unmap 0000:00:1b.0
(XEN) [VT-D]d2:PCIe: map 0000:00:1b.0
(XEN) d2: bind: m_gsi=16 g_gsi=40 dev=00.00.6 intx=0
(XEN) [VT-D] It's risky to assign 0000:00:1a.0 with shared RMRR at cd7ea000 for Dom2.
(XEN) [VT-D]d0:PCI: unmap 0000:00:1a.0
(XEN) [VT-D]d2:PCI: map 0000:00:1a.0

(d2) Relocating guest memory for lowmem MMIO space enabled
(d2) PCI-ISA link 0 routed to IRQ5
(d2) PCI-ISA link 1 routed to IRQ10
(d2) PCI-ISA link 2 routed to IRQ11
(d2) PCI-ISA link 3 routed to IRQ5
(d2) pci dev 01:3 INTA->IRQ10
(d2) pci dev 02:0 INTA->IRQ11
(d2) pci dev 03:0 INTA->IRQ5
(d2) pci dev 04:0 INTA->IRQ5
(d2) pci dev 05:0 INTA->IRQ10
(d2) pci dev 06:0 INTA->IRQ11
(d2) RAM in high memory; setting high_mem resource base to 100000000
(d2) pci dev 02:0 bar 18 size 010000000: 0e000000c
(XEN) memory_map:add: dom2 gfn=e0000 mfn=e0000 nr=10000

(d2) BIOS map:
(d2)  f0000-fffff: Main BIOS
(d2) E820 table:
(d2)  [00]: 00000000:00000000 - 00000000:0009e000: RAM
(d2)  [01]: 00000000:0009e000 - 00000000:000a0000: RESERVED
(d2)  HOLE: 00000000:000a0000 - 00000000:000e0000
(d2)  [02]: 00000000:000e0000 - 00000000:00100000: RESERVED
(d2)  [03]: 00000000:00100000 - 00000000:2fc00000: RAM
(d2)  HOLE: 00000000:2fc00000 - 00000000:cd7ea000
(d2)  [04]: 00000000:cd7ea000 - 00000000:cd815000: RESERVED
(d2)  HOLE: 00000000:cd815000 - 00000000:fc000000
(d2)  [05]: 00000000:fc000000 - 00000000:fdffc000: RESERVED
(d2)  [06]: 00000000:fdffc000 - 00000000:fdfff000: NVS
(d2)  [07]: 00000000:fdfff000 - 00000001:00000000: RESERVED
(d2)  [08]: 00000001:00000000 - 00000001:00000000: RAM
(d2) Invoking ROMBIOS ...
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel

 

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.

Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.