Re: Linux dom0 hangs on boot with Xen 4.22 and Heads firmware

[Jan Beulich <jbeulich@xxxxxxxx>]

On 19.06.2026 16:38, Marek Marczykowski-Górecki wrote:
> After updating Xen to 4.22-rc2 on a system with Heads firmware, dom0
> doesn't start anymore. It worked fine with Xen 4.19.
> The last messages on console are:
> 
>     [    1.495140] installing Xen timer for CPU 2
>     [    1.496149] installing Xen timer for CPU 4
>     [    1.496587] installing Xen timer for CPU 5
>     [    1.496809] installing Xen timer for CPU 7
>     [    0.008235] [Firmware Bug]: CPU   2: APIC ID mismatch. CPUID: 0x0002 
> APIC: 0x0028
>     [    0.008235] [Firmware Bug]: CPU   2: APIC ID mismatch. Firmware: 
> 0x0011 APIC: 0x0028
>     [    1.497055] cpu 2 spinlock event irq 200
>     [    0.008235] [Firmware Bug]: CPU   4: APIC ID mismatch. CPUID: 0x0004 
> APIC: 0x0000
>     [    0.008235] [Firmware Bug]: CPU   4: APIC ID mismatch. Firmware: 
> 0x0019 APIC: 0x0000
>     [    1.497074] cpu 4 spinlock event irq 201
>     [    0.008235] [Firmware Bug]: CPU   5: APIC ID mismatch. CPUID: 0x0005 
> APIC: 0x0002
>     [    0.008235] [Firmware Bug]: CPU   5: APIC ID mismatch. Firmware: 
> 0x0021 APIC: 0x0002
>     [    1.497074] cpu 5 spinlock event irq 202
>     [    0.008235] [Firmware Bug]: CPU   7: APIC I
> 
> Full console log (containing both successful boot of Xen 4.19, and then
> reboot into 4.22):
> https://openqa.qubes-os.org/tests/184780/logfile?filename=serial0.txt

The 4.19 log also has an anomaly around this point in time. Can you try
again with sync_console added to both the 4.19 and the 4.22 attempt?

> It doesn't reach loading graphics driver in dom0, so I don't have
> anything interesting on VGA (the last output is about the kexec call
> done by Heads). But at least I have a serial console.

Yet interestingly Xen's "  VGA is ... mode ..." lines differ as well.
That may be a separate issue. Or maybe that's how one of the EFI changes
that were done a while ago actually surfaces?

Another difference is

(XEN) HWP: Using HWP for cpufreq

in the 4.19 log vs

(XEN) HPET: 8 timers usable for broadcast (8 total)

in the 4.22 one. Does the latter perhaps go away if you include [1] in
the build?

For the HWP driver do you perhaps have a patch in 4.19 to default-enable
it, but you don't have a similar patch in 4.22? If so, to compare apples
to apples the driver wants enabling/disabling consistently (via command
line option) for both runs.

And then "mwait-idle: lapic_timer_reliable_states ..." differ as well.

Jan

[1] https://lists.xen.org/archives/html/xen-devel/2026-04/msg00200.html