Aw: Re: [help] Xen 4.14.5 on Devuan 4.0 Chimaera, regression from Xen 4.0.1

[Denis <tachyon_gun@xxxxxx>]

On 13.03.2023 12:54, Jan Beulich wrote:
> On 13.03.2023 12:43, Andrew Cooper wrote:
> > On 13/03/2023 9:36 am, Jan Beulich wrote:
> >> On 10.03.2023 21:50, Denis wrote:
> >>> On 10.03.2023 09:51, Jan Beulich wrote:
> >>>> On 09.03.2023 21:37, Andrew Cooper wrote:
> >>>>> On 09/03/2023 7:34 pm, tachyon_gun@xxxxxx wrote:
> >>>>>> A short snippet of what I see when invoking "xl dmesg":
> >>>>>>  
> >>>>>> (XEN) No southbridge IO-APIC found in IVRS table
> >>>>>> (XEN) AMD-Vi: Error initialization
> >>>>>> (XEN) I/O virtualisation disabled 
> >>>>>>  
> >>>>>> What I would like to see (taken from Xen 4.0.1 running on Debian
> >>>>>> Squeeze, in use since 2011):
> >>>>>>  
> >>>>>> (XEN) IOAPIC[0]: apic_id 8, version 33, address 0xfec00000, GSI 0-23
> >>>>>> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
> >>>>>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> >>>>>> (XEN) Detected 2611.936 MHz processor.
> >>>>>> (XEN) Initing memory sharing.
> >>>>>> (XEN) HVM: ASIDs enabled.
> >>>>>> (XEN) HVM: SVM enabled
> >>>>>> (XEN) HVM: Hardware Assisted Paging detected.
> >>>>>> (XEN) AMD-Vi: IOMMU 0 Enabled.
> >>>>>> (XEN) I/O virtualisation enabled
> >>>>>>  
> >>>>>> My question would be if this is "normal" behaviour due to older 
> >>>>>> hardware
> >>>>>> being used with newer versions of Xen (compared to the old 4.0.1) or if
> >>>>>> this is a bug.
> >>>>>> If the latter, has this been addressed already in newer version 
> >>>>>> (4.14+)?
> >>>> No, the code there is still the same. The commit introducing the check
> >>>> (06bbcaf48d09 ["AMD IOMMU: fail if there is no southbridge IO-APIC"])
> >>>> specifically provided for a workaround: "iommu=no-intremap" on the Xen
> >>>> command line. Could you give this a try? (As per below this could be
> >>>> what we want to do "automatically" in such a situation, i.e. without
> >>>> the need for a command line option. But you then still would face a
> >>>> perceived regression of interrupt remapping being disabled on such a
> >>>> system.)
> >>>>
> >>>> The other possible workaround, "iommu=no-amd-iommu-perdev-intremap",
> >>>> is something I rather wouldn't want to recommend, but you may still
> >>>> want to give it a try.
> >>>  
> >>> Thanks for your reply.
> >>>
> >>> I added the lines you suggested and it seems that "AMD-Vi: IOMMU 0" and
> >>> "I/O virtualisation" is enabled again.
> >> Good - that'll have to do as a workaround for the time being.
> > 
> > Not really.  Booting this system with no-intremap is still a regression
> > vs Xen 4.0.1
> 
> Well, "for the time being" meant untiol we figure out what exactly we can
> do here.
> 
> > Disabling interrupt remapping on PCIe devices because we can't figure
> > out interrupt handling around the PCI bridge is still bad behaviour.
> > 
> > 
> > What we need to figure out here is how interrupts from the PCI bridge
> > actually work.  The IVRS table does contain records covering the devices
> > on the Southbridge, including the PCI bridge and it's entire subordinate
> > range.
> > 
> > MSI/MSI-X interrupts from the PCI devices will work fine (they'll have a
> > proper source id), so the only question is about line interrupts.  They
> > ought to appear with the bridge's source id, and ought to be fine too.
> > 
> > 
> > I see no evidence to suggest that the IVRS/MADT are incorrect or
> > incomplete.  Xen's believe that there must be a southbridge IO-APIC
> > special device seems to be the false entity here.
> 
> It may be possible to behave better here, but my reading of Linux sources
> suggests that they would do exactly that on this system - disable
> interrupt remapping (I had asked Denis for trying that out to double
> check, but so far he provided back only hypervisor logs)

As suggested,  I tried the two kernel parameters. As a result, I was able 
to assign the devices (in this case a USB host controller and my PCIe GPU) I 
used 
before to the HVM domU.
What did not work was passing through the PCI device (Audigy2 soundcard).

Unbinding the devices worked for the first two, with the PCI card not so much.

This info won't help, but back in 2014 I tried Debian Jessie with (probably) 
Xen 4.4.
Back then this was working as it did on 4.0.1.

I will try Devuan Daedalus with Xen 4.17 next on my Z97 board with an i7-4771.
It probably will show the same errors because it also has an old PCI slot with 
an Audigy2 in it.

So I assume (from my limited knowlegde about all this) that this has something 
to do with 
older boards using PCI slots (or people like me using PCI devices for 
passthrough).


Denis