Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Tamas K Lengyel <tamas.k.lengyel@xxxxxxxxx>]

On Wed, Aug 15, 2018 at 12:40 AM Jan Beulich <JBeulich@xxxxxxxx> wrote:
>
> >>> On 15.08.18 at 03:00, <tamas.k.lengyel@xxxxxxxxx> wrote:
> > On Wed, Aug 8, 2018 at 3:54 AM Jan Beulich <JBeulich@xxxxxxxx> wrote:
> >>
> >> >>> On 08.08.18 at 10:25, <roger.pau@xxxxxxxxxx> wrote:
> >> > On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote:
> >> >> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel
> >> >> <tamas.k.lengyel@xxxxxxxxx> wrote:
> >> >> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
> >> >> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
> >> >> (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> >> >> 428f926000, iommu reg = ffff82c000a0c000
> >> >> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> >> >> (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 428f926
> >> >> (XEN)     root_entry[00] = 43aaae001
> >> >> (XEN)     context[10] = 2_43cf92001
> >> >> (XEN)     l4[000] = 9c0000043cf91107
> >> >> (XEN)     l3[10a] = 8000000000000000
> >> >> (XEN)     l3[10a] not present
> >> >>
> >> >> The fault is repeated a million times per second and the system is
> >> >> pretty much stalled.
> >> >
> >> > As Jan says, this page is outside of any range in the memory map. I
> >> > wonder however what's in there.
> >> >
> >> > I think (also seeing the PV issues) you should bring this up with the
> >> > driver maintainers, it might actually be a bug that the driver is
> >> > trying to access such address.
> >>
> >> Right, especially considering that the address does not appear to be
> >> invariant, I suspect the driver sets up some I/O from (presumably)
> >> uninitialized data. This goes unnoticed until DMA translation comes
> >> into play. Tamas - did you try enabling DMA translation in Linux
> >> when not running on top of Xen? Depending on the
> >> CONFIG_INTEL_IOMMU_DEFAULT_ON setting this may not be the
> >> default.
> >
> > I checked and this kernel option is not enabled. Are you suggesting to
> > try booting just Linux with this option enabled to see what happens?
>
> Well, you don't need to rebuild the kernel with the config option
> enabled, you can use what you have and add "intel_iommu=on"
> to the kernel command line. In case this works without any faults,
> changing to "intel_iommu=on,igfx_off" may or may not provide
> further insight.

Booting just linux with both options works just fine, I grepped the
dmesg log for the device that causes the issues with Xen but there are
no errors in the log:

# dmesg | grep "00\:02\.0"
[    0.165669] pci 0000:00:02.0: [8086:5916] type 00 class 0x030000
[    0.165681] pci 0000:00:02.0: reg 0x10: [mem 0xdb000000-0xdbffffff 64bit]
[    0.165687] pci 0000:00:02.0: reg 0x18: [mem 0x90000000-0x9fffffff
64bit pref]
[    0.165692] pci 0000:00:02.0: reg 0x20: [io  0xf000-0xf03f]
[    0.165707] pci 0000:00:02.0: BAR 2: assigned to efifb
[    0.184223] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[    0.184223] pci 0000:00:02.0: vgaarb: VGA device added:
decodes=io+mem,owns=io+mem,locks=none
[    0.184223] pci 0000:00:02.0: vgaarb: bridge control possible
[    0.248925] pci 0000:00:02.0: Video device with shadowed ROM at
[mem 0x000c0000-0x000dffff]
[    1.896026] i915 0000:00:02.0: vgaarb: changed VGA decodes:
olddecodes=io+mem,decodes=io+mem:owns=io+mem
[    1.903881] [drm] Initialized i915 1.6.0 20180514 for 0000:00:02.0 on minor 0
[    3.271519] i915 0000:00:02.0: fb0: inteldrmfb frame buffer device
[   12.519683] snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops
i915_audio_component_bind_ops [i915])


> >> > In the meantime, you can try to add to the command line:
> >> >
> >> > rmrr=0x428f926=0:0:2.0
> >> >
> >> > In order to force an iommu mapping of this address.
> >>
> >> I suspect this won't help much.
> >>
> >
> > The mfn is not always the same but seems to be at least on some
> > occasion.
>
> I expect that's because many things early at boot go pretty
> deterministically.
>
> > I managed to reboot with the right rmrr= option set but I'm
> > still getting the same faults over and over for that mfn I set in the
> > rmrr= option.
>
> Hmm, and the faults still show non-present entries for these? This
> options is at risk of getting misspelled - did you check that the
> option you've specified actually took effect?

I double checked and the option is set properly but I'm still getting
the same non-present entry faults as before. At the moment I don't
have serial access so not sure how to verify that the option took
effect. This is my boot info:

[pvh]
options=loglvl=all guest_loglvl=all dom0_mem=4096M,max:4096M
dom0_max_vcpus=2 sched=null dom0=pvh iommu=required,debug
dom0-iommu=relaxed console=vga
rmrr=0x5f48385=0:0:2.0,0x216183=0:0:2.0,0x70b6d19=0:0:2.0
kernel=vmlinuz-4.18.0-rc8 root=/dev/mapper/ubuntu--vg-root ro quiet
splash vt.handoff=1
ramdisk=initrd.img-4.18.0-rc8

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel