Re: [help] Xen 4.14.5 on Devuan 4.0 Chimaera, regression from Xen 4.0.1

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 09/03/2023 7:34 pm, tachyon_gun@xxxxxx wrote:
> Subject:
> [help] Xen 4.14.5 on Devuan 4.0 Chimaera
> From:
> tachyon_gun@xxxxxx
> Date:
> 09/03/2023, 7:34 pm
> 
> To:
> xen-devel@xxxxxxxxxxxxxxxxxxxx
> 
> 
> Hello.
> 
> Following the advice of Andrew Cooper (thanks for helping out) over on
> OFTC.net in #xen, I'll post this here.
> If this is the wrong place, please move it to the right section of your
> mailing lists.
>  
> I got some problems running Xen 4.14.5 on Devuan 4.0.
>  
> The AMD-Vi and I/O virtualisation are not being enabled when booting up
> the host system with Xen.
>  
> Hardware used:
> Mainboard: Gigabyte GA-890FXA-UD5, BIOS version F6 (2010.11.24, the
> latest version)
> CPU: AMD Phenom II X4 910e
> Memory: 16GB
> Storage: 2TB Crucial MX500
>  
> A short snippet of what I see when invoking "xl dmesg":
>  
> (XEN) No southbridge IO-APIC found in IVRS table
> (XEN) AMD-Vi: Error initialization
> (XEN) I/O virtualisation disabled 
>  
> What I would like to see (taken from Xen 4.0.1 running on Debian
> Squeeze, in use since 2011):
>  
> (XEN) IOAPIC[0]: apic_id 8, version 33, address 0xfec00000, GSI 0-23
> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 2611.936 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) HVM: ASIDs enabled.
> (XEN) HVM: SVM enabled
> (XEN) HVM: Hardware Assisted Paging detected.
> (XEN) AMD-Vi: IOMMU 0 Enabled.
> (XEN) I/O virtualisation enabled
>  
> My question would be if this is "normal" behaviour due to older hardware
> being used with newer versions of Xen (compared to the old 4.0.1) or if
> this is a bug.
> If the latter, has this been addressed already in newer version (4.14+)?
>  
> I'll attach some log files (hypervisor.log, dom0.log, xl_info.log,
> lspci_vvv.log, acpi.dmp, ivrs.dat, ivrs.dsl).
>  
> Thank you for your time.

Let me braindump the investigation so far before I forget it.

Xen requires that there is an IVRS special-device record describing an
IO-APIC 00:14.0.  This check failing is the source of the "No
southbridge" message, and the cause of the IOMMU(s) being turned off.

The MADT and IVRS tables agree that there is one IO-APIC in the system,
but that's the northbridge IO-APIC, not the southbridge.

The block diagram for the southbridge does have a PIC/IO-APIC as part of
the PCI bridge, so honestly I was expecting the MADT to describe 2
IO-APICs.  But OTOH, I could see this legitimately not existing in
configurations where the PCI bridge isn't in use.

`xl dmesg` does have a few unknown irqs, so there might be something
down in the southbridge really generating interrupts.  Or there might be
a IRQ misconfiguration elsewhere, and this is just a red herring.

However, a consequence of the northbridge and southbridge being separate
chips means that all southbridge IO is fully encapsulated by the IOMMU
in the northbridge.

So irrespective of whether there is ah IO-APIC operating properly in the
southbridge, and whether or not it's properly described, I think Xen's
insistence that there must be an IVRS special-device entry for it is bogus.


Furthermore, Xen's decisions are monumentally stupid.  It takes a
specifically safe (IOMMU-wise) system, and because it can't figure out a
partial aspect of interrupt handling the southbridge, decided that the
system can't be safe (a false conclusion) and turns the IOMMU off fully
to compensate, which makes the system concretely less safe.

~Andrew