Re: i915 dma faults on Xen

[Jason Andryuk <jandryuk@xxxxxxxxx>]

On Thu, Oct 15, 2020 at 7:31 AM Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
>
> On Wed, Oct 14, 2020 at 08:37:06PM +0100, Andrew Cooper wrote:
> > On 14/10/2020 20:28, Jason Andryuk wrote:
> > > Hi,
> > >
> > > Bug opened at https://gitlab.freedesktop.org/drm/intel/-/issues/2576
> > >
> > > I'm seeing DMA faults for the i915 graphics hardware on a Dell
> > > Latitude 5500. These were captured when I plugged into a Dell
> > > Thunderbolt dock with two DisplayPort monitors attached.  Xen 4.12.4
> > > staging and Linux 5.4.70 (and some earlier versions).
> > >
> > > Oct 14 18:41:49.056490 kernel:[   85.570347] [drm:gen8_de_irq_handler
> > > [i915]] *ERROR* Fault errors on pipe A: 0x00000080
> > > Oct 14 18:41:49.056494 kernel:[   85.570395] [drm:gen8_de_irq_handler
> > > [i915]] *ERROR* Fault errors on pipe A: 0x00000080
> > > Oct 14 18:41:49.056589 VM hypervisor: (XEN) [VT-D]DMAR:[DMA Read]
> > > Request device [0000:00:02.0] fault addr 39b5845000, iommu reg =
> > > ffff82c00021d000
> > > Oct 14 18:41:49.056594 VM hypervisor: (XEN) [VT-D]DMAR: reason 06 -
> > > PTE Read access is not set
> > > Oct 14 18:41:49.056784 kernel:[   85.570668] [drm:gen8_de_irq_handler
> > > [i915]] *ERROR* Fault errors on pipe A: 0x00000080
> > > Oct 14 18:41:49.056789 kernel:[   85.570687] [drm:gen8_de_irq_handler
> > > [i915]] *ERROR* Fault errors on pipe A: 0x00000080
> > > Oct 14 18:41:49.056885 VM hypervisor: (XEN) [VT-D]DMAR:[DMA Read]
> > > Request device [0000:00:02.0] fault addr 4238d0a000, iommu reg =
> > > ffff82c00021d000
> > > Oct 14 18:41:49.056890 VM hypervisor: (XEN) [VT-D]DMAR: reason 06 -
> > > PTE Read access is not set
> > >
> > > They repeat. In the log attached to
> > > https://gitlab.freedesktop.org/drm/intel/-/issues/2576, they start at
> > > "Oct 14 18:41:49.056589" and continue until I unplug the dock around
> > > "Oct 14 18:41:54.801802".
> > >
> > > I've also seen similar messages when attaching the laptop's HDMI port
> > > to a 4k monitor. The eDP display by itself seems okay.
> > >
> > > I tried Fedora 31 & 32 live images with intel_iommu=on, so no Xen, and
> > > didn't see any errors
> > >
> > > This is a kernel & xen log with drm.debug=0x1e. It also includes some
> > > application (glass) logging when it changes resolutions which seems to
> > > set off the DMA faults. 5500-igfx-messages-kern-xen-glass
> > >
> > > Running xen with iommu=no-igfx disables the iommu for the i915
> > > graphics and no faults are reported. However, that breaks some other
> > > devices (Dell Latitude 7200 and 5580) giving a black screen with:
> > >
> > > Oct 10 13:24:37.022117 kernel:[   14.884759] i915 0000:00:02.0: Failed
> > > to idle engines, declaring wedged!
> > > Oct 10 13:24:37.022118 kernel:[   14.964794] i915 0000:00:02.0: Failed
> > > to initialize GPU, declaring it wedged!
> > >
> > > Any suggestions welcome.
> >
> > Presumably this is with a PV dom0.  What are 39b5845000 and 4238d0a000
> > in the machine memory map?

They are bogus?
End of RAM is 0x47c800000
Thats:
0x047c800000
vs.
0x39b5845000
0x4238d0a000

> > This smells like a missing RMRR in the ACPI tables.
>
> I agree.
>
> Can you paste the memory map as printed by Xen when booting, and what
> command line are you using to boot Xen.

So this is OpenXT, and it's booting EFI -> xen -> tboot -> xen

There's the memory map
(XEN) TBOOT RAM map:
(XEN)  0000000000000000 - 0000000000060000 (usable)
(XEN)  0000000000060000 - 0000000000068000 (reserved)
(XEN)  0000000000068000 - 000000000009e000 (usable)
(XEN)  000000000009e000 - 000000000009f000 (reserved)
(XEN)  000000000009f000 - 00000000000a0000 (usable)
(XEN)  00000000000a0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000040000000 (usable)
(XEN)  0000000040000000 - 0000000040400000 (reserved)
(XEN)  0000000040400000 - 000000007024b000 (usable)
(XEN)  000000007024b000 - 000000007024c000 (ACPI NVS)
(XEN)  000000007024c000 - 000000007024d000 (reserved)
(XEN)  000000007024d000 - 0000000077f19000 (usable)
(XEN)  0000000077f19000 - 0000000078987000 (reserved)
(XEN)  0000000078987000 - 0000000078a04000 (ACPI data)
(XEN)  0000000078a04000 - 0000000078ea3000 (ACPI NVS)
(XEN)  0000000078ea3000 - 000000007acff000 (reserved)
(XEN)  000000007acff000 - 000000007ad00000 (usable)
(XEN)  000000007ad00000 - 000000007f800000 (reserved)
(XEN)  00000000f0000000 - 00000000f8000000 (reserved)
(XEN)  00000000fe000000 - 00000000fe011000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000047c800000 (usable)
(XEN) EFI memory map:
(XEN)  0000000000000-000000009dfff type=7 attr=000000000000000f
(XEN)  000000009e000-000000009efff type=0 attr=000000000000000f
(XEN)  000000009f000-000000009ffff type=3 attr=000000000000000f
(XEN)  0000000100000-000003fffffff type=7 attr=000000000000000f
(XEN)  0000040000000-00000403fffff type=0 attr=000000000000000f
(XEN)  0000040400000-000005e359fff type=7 attr=000000000000000f
(XEN)  000005e35a000-000005e399fff type=4 attr=000000000000000f
(XEN)  000005e39a000-000006a47dfff type=7 attr=000000000000000f
(XEN)  000006a47e000-000006c3eefff type=2 attr=000000000000000f
(XEN)  000006c3ef000-000006d5eefff type=1 attr=000000000000000f
(XEN)  000006d5ef000-000006d86cfff type=2 attr=000000000000000f
(XEN)  000006d86d000-000006d978fff type=1 attr=000000000000000f
(XEN)  000006d979000-000006dc7afff type=4 attr=000000000000000f
(XEN)  000006dc7b000-000006dc98fff type=3 attr=000000000000000f
(XEN)  000006dc99000-000006dcc7fff type=4 attr=000000000000000f
(XEN)  000006dcc8000-000006dccdfff type=3 attr=000000000000000f
(XEN)  000006dcce000-00000701a5fff type=4 attr=000000000000000f
(XEN)  00000701a6000-00000701c8fff type=3 attr=000000000000000f
(XEN)  00000701c9000-00000701edfff type=4 attr=000000000000000f
(XEN)  00000701ee000-0000070204fff type=3 attr=000000000000000f
(XEN)  0000070205000-000007022cfff type=4 attr=000000000000000f
(XEN)  000007022d000-000007024afff type=3 attr=000000000000000f
(XEN)  000007024b000-000007024bfff type=10 attr=000000000000000f
(XEN)  000007024c000-000007024cfff type=6 attr=800000000000000f
(XEN)  000007024d000-000007024dfff type=4 attr=000000000000000f
(XEN)  000007024e000-0000070282fff type=3 attr=000000000000000f
(XEN)  0000070283000-00000702c3fff type=4 attr=000000000000000f
(XEN)  00000702c4000-00000702c8fff type=3 attr=000000000000000f
(XEN)  00000702c9000-00000702defff type=4 attr=000000000000000f
(XEN)  00000702df000-0000070307fff type=3 attr=000000000000000f
(XEN)  0000070308000-0000070317fff type=4 attr=000000000000000f
(XEN)  0000070318000-0000070319fff type=3 attr=000000000000000f
(XEN)  000007031a000-0000070331fff type=4 attr=000000000000000f
(XEN)  0000070332000-0000070349fff type=3 attr=000000000000000f
(XEN)  000007034a000-0000070356fff type=2 attr=000000000000000f
(XEN)  0000070357000-0000070357fff type=7 attr=000000000000000f
(XEN)  0000070358000-0000070358fff type=2 attr=000000000000000f
(XEN)  0000070359000-0000076f3efff type=4 attr=000000000000000f
(XEN)  0000076f3f000-00000772affff type=7 attr=000000000000000f
(XEN)  00000772b0000-0000077f18fff type=3 attr=000000000000000f
(XEN)  0000077f19000-0000078986fff type=0 attr=000000000000000f
(XEN)  0000078987000-0000078a03fff type=9 attr=000000000000000f
(XEN)  0000078a04000-0000078ea2fff type=10 attr=000000000000000f
(XEN)  0000078ea3000-000007ab22fff type=6 attr=800000000000000f
(XEN)  000007ab23000-000007acfefff type=5 attr=800000000000000f
(XEN)  000007acff000-000007acfffff type=4 attr=000000000000000f
(XEN)  0000100000000-000047c7fffff type=7 attr=000000000000000f
(XEN)  00000000a0000-00000000fffff type=0 attr=0000000000000000
(XEN)  000007ad00000-000007adfffff type=0 attr=070000000000000f
(XEN)  000007ae00000-000007f7fffff type=0 attr=0000000000000000
(XEN)  00000f0000000-00000f7ffffff type=11 attr=800000000000100d
(XEN)  00000fe000000-00000fe010fff type=11 attr=8000000000000001
(XEN)  00000fec00000-00000fec00fff type=11 attr=8000000000000001
(XEN)  00000fee00000-00000fee00fff type=11 attr=8000000000000001
(XEN)  00000ff000000-00000ffffffff type=11 attr=800000000000100d

Command line
console=com1 dom0_mem=min:420M,max:420M,420M efi=no-rs,attr=uc
com1=115200,8n1,pci mbi-video vga=current flask=enforcing loglvl=debug
guest_loglvl=debug smt=0 ucode=-1 bootscrub=1
argo=yes,mac-permissive=1 iommu=force,igfx

iommu=force,igfx was to force igfx back on.  I added a dmi quirk to
set no-igfx on this platform as a temporary workaround.

> Have you tried adding dom0-iommu=map-inclusive to the Xen command
> line?

I have not.  I can try that tomorrow when I have access to the system again.

Thanks,
Jason