[Top] [All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: IOMMU faults after S3

  • To: Jan Beulich <jbeulich@xxxxxxxx>
  • From: Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>
  • Date: Wed, 1 Apr 2026 22:30:10 +0200
  • Authentication-results: eu.smtp.expurgate.cloud; dkim=pass header.s=fm1 header.d=invisiblethingslab.com header.i="@invisiblethingslab.com" header.h="Cc:Content-Type:Date:From:In-Reply-To:Message-ID:MIME-Version:References:Subject:To"; dkim=pass header.s=fm2 header.d=messagingengine.com header.i="@messagingengine.com" header.h="Cc:Content-Type:Date:Feedback-ID:From:In-Reply-To:Message-ID:MIME-Version:References:Subject:To:X-ME-Proxy:X-ME-Sender"
  • Cc: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Wed, 01 Apr 2026 20:30:33 +0000
  • Feedback-id: i1568416f:Fastmail
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
On Wed, Apr 01, 2026 at 10:11:12AM +0200, Jan Beulich wrote:
> On 01.04.2026 09:20, Andrew Cooper wrote:
> > On 01/04/2026 9:14 am, Jan Beulich wrote:
> >> On 27.03.2026 11:19, Marek Marczykowski-Górecki wrote:
> >>> I noticed that on some systems, there are a lot of IOMMU faults after
> >>> S3. I can see it also on a laptop with MTL, but it affects also the ADL
> >>> gitlab runner:
> >>>
> >>>     https://gitlab.com/xen-project/hardware/xen/-/jobs/13661033722
> >>>     (XEN) [   37.201160] [VT-D]DMAR:[DMA Write] Request device 
> >>> [0000:00:1e.6] fault addr 0
> >>>     (XEN) [   37.201164] [VT-D]DMAR: reason 02 - Present bit in context 
> >>> entry is clear
> >>>     (XEN) [   37.202332] [VT-D]DMAR:[DMA Write] Request device 
> >>> [0000:00:1e.6] fault addr 0
> >>>     (XEN) [   37.202339] [VT-D]DMAR: reason 02 - Present bit in context 
> >>> entry is clear
> >>>
> >>> Interestingly, the 0000:00:1e.6 device is not even listed by lspci.
> >>>
> >>> The issue is present only on staging, not staging-4.21.
> >>>
> >>> Bisect says:
> >>>
> >>> 5ec93b2f19ff8873fca65d38c1164b0a56d3898b is the first bad commit
> >>> commit 5ec93b2f19ff8873fca65d38c1164b0a56d3898b
> >>> Author: Jan Beulich <jbeulich@xxxxxxxx>
> >>> Date:   Thu Jan 22 14:13:35 2026 +0100
> >>>
> >>>     x86/HPET: drop .set_affinity hook
> >> Looking into this, I find several things I can't quite understand (yet).
> >> First there is
> >>
> >> (XEN) [000000456c0fe39f] Disabling HPET for being unreliable
> >>
> >> which looks to only affect clocksource selection, but not use as
> >> broadcast source for CPU-idle management. (This may be an independent
> >> issue.)
> >>
> >> Then there is
> >>
> >> (XEN) [    2.760248] HPET: 8 timers usable for broadcast (8 total)
> >>
> >> which should only occur on ARAT-incapable systems. That should only be
> >> older hardware.
> > 
> > I'm not sure that's a reasonable assertion to draw.  The number of HPET
> > channels is down to the HPET alone, not anything to do with the CPU
> > capabilities.
> 
> My statement was about the mere presence of that message, not the number
> of channels that are reported.
> 
> >>  (On my much older Skylake I don't see this line, for
> >> example.) What does CPUID leaf 6 have on this system? Sadly xen-cpuid
> >> is purely featureset based, and hence doesn't expose info about that
> >> leaf.
> > 
> > xen-cpuid -p
> > 
> > That will get you leaf 6, but there's no human-readable decode of it.
> 
> Raw numbers is good enough here. How did I miss that option when looking
> at --help output? Oh, simply because it isn't shown there.
> 
> Marek, that'll be better than bare metal kernel data, as it gives us both
> raw and host policies.

Here is the output from ADL runner:

Xen reports there are maximum 120 leaves and 2 MSRs
Raw policy: 48 leaves, 2 MSRs
 CPUID:
  leaf     subleaf  -> eax      ebx      ecx      edx     
  00000000:ffffffff -> 00000020:756e6547:6c65746e:49656e69
  00000001:ffffffff -> 00090672:00800800:77fafbff:bfebfbff
  00000002:ffffffff -> 00feff01:000000f0:00000000:00000000
  00000004:00000000 -> fc004121:02c0003f:0000003f:00000000
  00000004:00000001 -> fc004122:01c0003f:0000003f:00000000
  00000004:00000002 -> fc01c143:0240003f:000007ff:00000000
  00000004:00000003 -> fc1fc163:0240003f:00007fff:00000004
  00000005:ffffffff -> 00000040:00000040:00000003:10102020
  00000006:ffffffff -> 00df8ff7:00000002:00000409:00000003
  00000007:00000000 -> 00000002:239c27eb:98c027ac:fc1cc410
  00000007:00000001 -> 00400810:00000000:00000000:00040000
  00000007:00000002 -> 00000000:00000000:00000000:00000017
  0000000a:ffffffff -> 07300605:00000000:00000007:00008603
  0000000b:00000000 -> 00000001:00000002:00000100:00000000
  0000000b:00000001 -> 00000007:00000010:00000201:00000000
  0000000d:00000000 -> 00000207:00000000:00000a88:00000000
  0000000d:00000001 -> 0000000f:00000000:00019900:00000000
  0000000d:00000002 -> 00000100:00000240:00000000:00000000
  0000000d:00000008 -> 00000080:00000000:00000001:00000000
  0000000d:00000009 -> 00000008:00000a80:00000000:00000000
  0000000d:0000000b -> 00000010:00000000:00000001:00000000
  0000000d:0000000c -> 00000018:00000000:00000001:00000000
  0000000d:0000000f -> 00000328:00000000:00000001:00000000
  0000000d:00000010 -> 00000008:00000000:00000001:00000000
  80000000:ffffffff -> 80000008:00000000:00000000:00000000
  80000001:ffffffff -> 00000000:00000000:00000121:2c100800
  80000002:ffffffff -> 68743231:6e654720:746e4920:52286c65
  80000003:ffffffff -> 6f432029:54286572:6920294d:32312d35
  80000004:ffffffff -> 4b303036:00000000:00000000:00000000
  80000006:ffffffff -> 00000000:00000000:05007040:00000000
  80000007:ffffffff -> 00000000:00000000:00000000:00000100
  80000008:ffffffff -> 0000302e:00000000:00000000:00000000
 MSRs:
  index    -> value           
  000000ce -> 0000000080000000
  0000010a -> 000000001488fd6b
Host policy: 41 leaves, 2 MSRs
 CPUID:
  leaf     subleaf  -> eax      ebx      ecx      edx     
  00000000:ffffffff -> 0000000d:756e6547:6c65746e:49656e69
  00000001:ffffffff -> 00090672:00800800:77fafbff:bfebfbff
  00000002:ffffffff -> 00feff01:000000f0:00000000:00000000
  00000004:00000000 -> fc004121:02c0003f:0000003f:00000000
  00000004:00000001 -> fc004122:01c0003f:0000003f:00000000
  00000004:00000002 -> fc01c143:0240003f:000007ff:00000000
  00000004:00000003 -> fc1fc163:0240003f:00007fff:00000004
  00000005:ffffffff -> 00000040:00000040:00000003:10102020
  00000006:ffffffff -> 00df8ff7:00000002:00000409:00000003
  00000007:00000000 -> 00000002:239c27eb:984027ac:fc1cc410
  00000007:00000001 -> 00000810:00000000:00000000:00040000
  00000007:00000002 -> 00000000:00000000:00000000:00000017
  0000000b:00000000 -> 00000001:00000002:00000100:00000000
  0000000b:00000001 -> 00000007:00000010:00000201:00000000
  0000000d:00000000 -> 00000207:00000000:00000a88:00000000
  0000000d:00000001 -> 0000000f:00000000:00000000:00000000
  0000000d:00000002 -> 00000100:00000240:00000000:00000000
  0000000d:00000009 -> 00000008:00000a80:00000000:00000000
  80000000:ffffffff -> 80000008:00000000:00000000:00000000
  80000001:ffffffff -> 00000000:00000000:00000121:2c100800
  80000002:ffffffff -> 68743231:6e654720:746e4920:52286c65
  80000003:ffffffff -> 6f432029:54286572:6920294d:32312d35
  80000004:ffffffff -> 4b303036:00000000:00000000:00000000
  80000006:ffffffff -> 00000000:00000000:05007040:00000000
  80000007:ffffffff -> 00000000:00000000:00000000:00000100
  80000008:ffffffff -> 0000302e:00000000:00000000:00000000
 MSRs:
  index    -> value           
  000000ce -> 0000000080000000
  0000010a -> 400000001488fd6b
PV Max policy: 58 leaves, 2 MSRs
 CPUID:
  leaf     subleaf  -> eax      ebx      ecx      edx     
  00000000:ffffffff -> 0000000d:756e6547:6c65746e:49656e69
  00000001:ffffffff -> 00090672:00800800:f6f83203:1fc9cbf5
  00000002:ffffffff -> 00feff01:000000f0:00000000:00000000
  00000004:00000000 -> fc004121:02c0003f:0000003f:00000000
  00000004:00000001 -> fc004122:01c0003f:0000003f:00000000
  00000004:00000002 -> fc01c143:0240003f:000007ff:00000000
  00000004:00000003 -> fc1fc163:0240003f:00007fff:00000004
  00000007:00000000 -> 00000002:218c0329:18400700:ac004410
  00000007:00000001 -> 00000810:00000000:00000000:00000000
  00000007:00000002 -> 00000000:00000000:00000000:00000001
  0000000d:00000000 -> 00000007:00000000:00000340:00000000
  0000000d:00000001 -> 00000007:00000000:00000000:00000000
  0000000d:00000002 -> 00000100:00000240:00000000:00000000
  80000000:ffffffff -> 80000021:00000000:00000000:00000000
  80000001:ffffffff -> 00000000:00000000:00000123:28100800
  80000002:ffffffff -> 68743231:6e654720:746e4920:52286c65
  80000003:ffffffff -> 6f432029:54286572:6920294d:32312d35
  80000004:ffffffff -> 4b303036:00000000:00000000:00000000
  80000006:ffffffff -> 00000000:00000000:05007040:00000000
  80000007:ffffffff -> 00000000:00000000:00000000:00000100
  80000008:ffffffff -> 0000302e:00001000:00000000:00000000
 MSRs:
  index    -> value           
  000000ce -> 0000000080000000
  0000010a -> 40000000140ae167
HVM Max policy: 65 leaves, 2 MSRs
 CPUID:
  leaf     subleaf  -> eax      ebx      ecx      edx     
  00000000:ffffffff -> 0000000d:756e6547:6c65746e:49656e69
  00000001:ffffffff -> 00090672:00800800:f7fa3223:1fcbfbff
  00000002:ffffffff -> 00feff01:000000f0:00000000:00000000
  00000004:00000000 -> fc004121:02c0003f:0000003f:00000000
  00000004:00000001 -> fc004122:01c0003f:0000003f:00000000
  00000004:00000002 -> fc01c143:0240003f:000007ff:00000000
  00000004:00000003 -> fc1fc163:0240003f:00007fff:00000004
  00000007:00000000 -> 00000002:219c07ab:9840070c:bc004410
  00000007:00000001 -> 00000810:00000000:00000000:00000000
  00000007:00000002 -> 00000000:00000000:00000000:00000017
  0000000d:00000000 -> 00000207:00000000:00000a88:00000000
  0000000d:00000001 -> 0000000f:00000000:00000000:00000000
  0000000d:00000002 -> 00000100:00000240:00000000:00000000
  0000000d:00000009 -> 00000008:00000a80:00000000:00000000
  80000000:ffffffff -> 80000021:00000000:00000000:00000000
  80000001:ffffffff -> 00000000:00000000:00000123:2c100800
  80000002:ffffffff -> 68743231:6e654720:746e4920:52286c65
  80000003:ffffffff -> 6f432029:54286572:6920294d:32312d35
  80000004:ffffffff -> 4b303036:00000000:00000000:00000000
  80000006:ffffffff -> 00000000:00000000:05007040:00000000
  80000007:ffffffff -> 00000000:00000000:00000000:00000100
  80000008:ffffffff -> 0000302e:00101000:00000000:00000000
 MSRs:
  index    -> value           
  000000ce -> 0000000080000000
  0000010a -> 40000000140ae167
PV Default policy: 33 leaves, 2 MSRs
 CPUID:
  leaf     subleaf  -> eax      ebx      ecx      edx     
  00000000:ffffffff -> 0000000d:756e6547:6c65746e:49656e69
  00000001:ffffffff -> 00090672:00800800:f6d83203:1fc9cbf5
  00000002:ffffffff -> 00feff01:000000f0:00000000:00000000
  00000004:00000000 -> fc004121:02c0003f:0000003f:00000000
  00000004:00000001 -> fc004122:01c0003f:0000003f:00000000
  00000004:00000002 -> fc01c143:0240003f:000007ff:00000000
  00000004:00000003 -> fc1fc163:0240003f:00007fff:00000004
  00000007:00000000 -> 00000002:218c0329:00400700:ac004410
  00000007:00000001 -> 00000810:00000000:00000000:00000000
  00000007:00000002 -> 00000000:00000000:00000000:00000001
  0000000d:00000000 -> 00000007:00000000:00000340:00000000
  0000000d:00000001 -> 00000007:00000000:00000000:00000000
  0000000d:00000002 -> 00000100:00000240:00000000:00000000
  80000000:ffffffff -> 80000008:00000000:00000000:00000000
  80000001:ffffffff -> 00000000:00000000:00000121:28100800
  80000002:ffffffff -> 68743231:6e654720:746e4920:52286c65
  80000003:ffffffff -> 6f432029:54286572:6920294d:32312d35
  80000004:ffffffff -> 4b303036:00000000:00000000:00000000
  80000006:ffffffff -> 00000000:00000000:05007040:00000000
  80000008:ffffffff -> 0000302e:00001000:00000000:00000000
 MSRs:
  index    -> value           
  000000ce -> 0000000080000000
  0000010a -> 400000001408e163
HVM Default policy: 40 leaves, 2 MSRs
 CPUID:
  leaf     subleaf  -> eax      ebx      ecx      edx     
  00000000:ffffffff -> 0000000d:756e6547:6c65746e:49656e69
  00000001:ffffffff -> 00090672:00800800:f7fa3203:1fcbfbff
  00000002:ffffffff -> 00feff01:000000f0:00000000:00000000
  00000004:00000000 -> fc004121:02c0003f:0000003f:00000000
  00000004:00000001 -> fc004122:01c0003f:0000003f:00000000
  00000004:00000002 -> fc01c143:0240003f:000007ff:00000000
  00000004:00000003 -> fc1fc163:0240003f:00007fff:00000004
  00000007:00000000 -> 00000002:219c07ab:8040070c:bc004410
  00000007:00000001 -> 00000810:00000000:00000000:00000000
  00000007:00000002 -> 00000000:00000000:00000000:00000017
  0000000d:00000000 -> 00000207:00000000:00000a88:00000000
  0000000d:00000001 -> 0000000f:00000000:00000000:00000000
  0000000d:00000002 -> 00000100:00000240:00000000:00000000
  0000000d:00000009 -> 00000008:00000a80:00000000:00000000
  80000000:ffffffff -> 80000008:00000000:00000000:00000000
  80000001:ffffffff -> 00000000:00000000:00000121:2c100800
  80000002:ffffffff -> 68743231:6e654720:746e4920:52286c65
  80000003:ffffffff -> 6f432029:54286572:6920294d:32312d35
  80000004:ffffffff -> 4b303036:00000000:00000000:00000000
  80000006:ffffffff -> 00000000:00000000:05007040:00000000
  80000008:ffffffff -> 0000302e:00101000:00000000:00000000
 MSRs:
  index    -> value           
  000000ce -> 0000000080000000
  0000010a -> 400000001408e163


-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

Attachment: signature.asc
Description: PGP signature

 

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.

Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.