[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH] iommu/amd-vi: do not error if device referenced in IVMD is not behind any IOMMU



On Tue, Oct 08, 2024 at 04:01:28PM +0200, Jan Beulich wrote:
> On 08.10.2024 12:47, Roger Pau Monne wrote:
> > IVMD table contains restrictions about memory which must be mandatory 
> > assigned
> > to devices (and which permissions it should use), or memory that should be
> > never accessible to devices.
> > 
> > Some hardware however contains ranges in IVMD that reference devices 
> > outside of
> > the IVHD tables (in other words, devices not behind any IOMMU).  Such 
> > mismatch
> > will cause Xen to fail in register_range_for_device(), ultimately leading to
> > the IOMMU being disabled, and Xen crashing as x2APIC support might be 
> > already
> > enabled and relying on the IOMMU functionality.
> 
> I find it hard to believe that on x86 systems with IOMMUs some devices would
> be left uncovered by any IOMMU. Is it possible that IVHD is flawed there? In
> which case we might rightfully refuse to boot? (Can you share e.g. that
> "iommu=debug" output that results from parsing the tables on that system?)

I'm afraid I don't have any of such systems to test myself, however I
have the contents of IVRS:

  ACPI Table Header
------------------------------------------------------------------
Signature          : IVRS
Length             : 0x000001F8
Revision           : 0x02
Checksum           : 0x06
OEM ID             : AMD  
OEM Table ID       : AmdTable
OEM Revision       : 0x00000001
Creator ID         : AMD 
Creator Revision   : 0x00000001
IVinfo             : 0x00203043
          IVHD
        ----------------------------------------------------------------
        Type                  : 0x10
        Flags                 : 0xB0
        Length                : 0x0044
        IOMMU Device ID       : 0x0002
        Capability Offset     : 0x0040
        IOMMU Base Address    : 0x00000000FD200000
        Segment Group         : 0x0000
        IOMMU Info            : 0x0000
        IOMMU Feature Info    : 0x80048F6E
                  Range
                --------------------------------------------------
                Type                  : 0x03
                Start of Range        : 0x0003
                End of Range          : 0xFFFE
                DTE Setting           : 0x00
                  Alias Range
                --------------------------------------------------
                Type                  : 0x43
                Start of Range        : 0xFF00
                End of Range          : 0xFFFF
                DTE Setting           : 0x00
                Source Device ID      : 0x00A5
                  Special Device
                --------------------------------------------------
                Type                  : 0x48
                Device ID             : 0x0000
                DTE Setting           : 0x00
                Source Device ID      : 0x00A0
                Handle                : 0x00
                Variety               : HPET
                  Special Device
                --------------------------------------------------
                Type                  : 0x48
                Device ID             : 0x0000
                DTE Setting           : 0xD7
                Source Device ID      : 0x00A0
                Handle                : 0x21
                Variety               : IOAPIC
                  Special Device
                --------------------------------------------------
                Type                  : 0x48
                Device ID             : 0x0000
                DTE Setting           : 0x00
                Source Device ID      : 0x0001
                Handle                : 0x22
                Variety               : IOAPIC
          IVHD
        ----------------------------------------------------------------
        Type                  : 0x11
        Flags                 : 0x30
        Length                : 0x0054
        IOMMU Device ID       : 0x0002
        Capability Offset     : 0x0040
        IOMMU Base Address    : 0x00000000FD200000
        Segment Group         : 0x0000
        IOMMU Info            : 0x0000
        IOMMU Feature Info    : 0x00048000
                  Range
                --------------------------------------------------
                Type                  : 0x03
                Start of Range        : 0x0003
                End of Range          : 0xFFFE
                DTE Setting           : 0x00
                  Alias Range
                --------------------------------------------------
                Type                  : 0x43
                Start of Range        : 0xFF00
                End of Range          : 0xFFFF
                DTE Setting           : 0x00
                Source Device ID      : 0x00A5
                  Special Device
                --------------------------------------------------
                Type                  : 0x48
                Device ID             : 0x0000
                DTE Setting           : 0x00
                Source Device ID      : 0x00A0
                Handle                : 0x00
                Variety               : HPET
                  Special Device
                --------------------------------------------------
                Type                  : 0x48
                Device ID             : 0x0000
                DTE Setting           : 0xD7
                Source Device ID      : 0x00A0
                Handle                : 0x21
                Variety               : IOAPIC
                  Special Device
                --------------------------------------------------
                Type                  : 0x48
                Device ID             : 0x0000
                DTE Setting           : 0x00
                Source Device ID      : 0x0001
                Handle                : 0x22
                Variety               : IOAPIC
          IVMD
        ----------------------------------------------------------------
        Type                                 : 0x22
        Flags                                : 0x08
        Length                               : 0x0020
        DeviceID                             : 0x0000
        AuxiliaryData                        : 0x0FFF
        Reserved                             : 0x0000000000000000
        IVMD Start Address                   : 0x0000000096191000
        IVMD Memory Block Length             : 0x0000000000000022
          IVMD
        ----------------------------------------------------------------
        Type                                 : 0x22
        Flags                                : 0x08
        Length                               : 0x0020
        DeviceID                             : 0x0000
        AuxiliaryData                        : 0x0FFF
        Reserved                             : 0x0000000000000000
        IVMD Start Address                   : 0x0000000097D9E000
        IVMD Memory Block Length             : 0x0000000000000022
          IVMD
        ----------------------------------------------------------------
        Type                                 : 0x22
        Flags                                : 0x08
        Length                               : 0x0020
        DeviceID                             : 0x0000
        AuxiliaryData                        : 0x0FFF
        Reserved                             : 0x0000000000000000
        IVMD Start Address                   : 0x0000000097D9D000
        IVMD Memory Block Length             : 0x0000000000000022
          IVHD
        ----------------------------------------------------------------
        Type                  : 0x40
        Flags                 : 0x30
        Length                : 0x00D0
        IOMMU Device ID       : 0x0002
        Capability Offset     : 0x0040
        IOMMU Base Address    : 0x00000000FD200000
        Segment Group         : 0x0000
        IOMMU Info            : 0x0000
        IOMMU Feature Info    : 0x00048000
                  Range
                --------------------------------------------------
                Type                  : 0x03
                Start of Range        : 0x0003
                End of Range          : 0xFFFE
                DTE Setting           : 0x00
                  Alias Range
                --------------------------------------------------
                Type                  : 0x43
                Start of Range        : 0xFF00
                End of Range          : 0xFFFF
                DTE Setting           : 0x00
                Source Device ID      : 0x00A5
                  Special Device
                --------------------------------------------------
                Type                  : 0x48
                Device ID             : 0x0000
                DTE Setting           : 0x00
                Source Device ID      : 0x00A0
                Handle                : 0x00
                Variety               : HPET
                  Special Device
                --------------------------------------------------
                Type                  : 0x48
                Device ID             : 0x0000
                DTE Setting           : 0xD7
                Source Device ID      : 0x00A0
                Handle                : 0x21
                Variety               : IOAPIC
                  Special Device
                --------------------------------------------------
                Type                  : 0x48
                Device ID             : 0x0000
                DTE Setting           : 0x00
                Source Device ID      : 0x0001
                Handle                : 0x22
                Variety               : IOAPIC
                  Variable Length ACPI HID Device
                --------------------------------------------------
                Type                  : 0xF0
                Device ID             : 0x00A5
                DTE Setting           : 0x40
                Hardware ID           : AMDI0020
                Extended DTE Setting  : 
                Unique ID Format      : 2
                Unique ID Length      : 9
                Unique ID             : \_SB.FUR0
                  Variable Length ACPI HID Device
                --------------------------------------------------
                Type                  : 0xF0
                Device ID             : 0x00A5
                DTE Setting           : 0x40
                Hardware ID           : AMDI0020
                Extended DTE Setting  : 
                Unique ID Format      : 2
                Unique ID Length      : 9
                Unique ID             : \_SB.FUR1
                  Variable Length ACPI HID Device
                --------------------------------------------------
                Type                  : 0xF0
                Device ID             : 0x00A5
                DTE Setting           : 0x40
                Hardware ID           : AMDI0020
                Extended DTE Setting  : 
                Unique ID Format      : 2
                Unique ID Length      : 9
                Unique ID             : \_SB.FUR2
                  Variable Length ACPI HID Device
                --------------------------------------------------
                Type                  : 0xF0
                Device ID             : 0x00A5
                DTE Setting           : 0x40
                Hardware ID           : AMDI0020
                Extended DTE Setting  : 
                Unique ID Format      : 2
                Unique ID Length      : 9
                Unique ID             : \_SB.FUR3

FWIW, I've checked on one of the AMD server systems we have on the
lab, and the IVHD entries are fairly similar to the ones here, as
neither the PCI Host Bridge, nor the IOMMU are covered by any IVHD
block.  That system however doesn't have any IVMD blocks.

> > Relax IVMD parsing: allow IVMD blocks to reference devices not assigned to 
> > any
> > IOMMU.  It's impossible for Xen to fulfill the requirement in the IVMD 
> > block if
> > the device is not behind any IOMMU, but it's no worse than booting without
> > IOMMU support, and thus not parsing ACPI IVRS in the first place.
> > 
> > Reported-by: Willi Junga <xenproject@xxxxxx>
> > Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
> > ---
> >  xen/drivers/passthrough/amd/iommu_acpi.c | 5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> > 
> > diff --git a/xen/drivers/passthrough/amd/iommu_acpi.c 
> > b/xen/drivers/passthrough/amd/iommu_acpi.c
> > index 3f5508eba049..c416120326c9 100644
> > --- a/xen/drivers/passthrough/amd/iommu_acpi.c
> > +++ b/xen/drivers/passthrough/amd/iommu_acpi.c
> > @@ -248,8 +248,9 @@ static int __init register_range_for_device(
> >      iommu = find_iommu_for_device(seg, bdf);
> >      if ( !iommu )
> >      {
> > -        AMD_IOMMU_ERROR("IVMD: no IOMMU for Dev_Id %#x\n", bdf);
> > -        return -ENODEV;
> > +        AMD_IOMMU_WARN("IVMD: no IOMMU for device %pp - ignoring 
> > constrain\n",
> 
> I'm not a native speaker, but "constrain" to me can only be a verb (with
> "constraint" being the noun). IOW as worded I'm afraid I can't make sense
> of the message.

Indeed, sorry for the typo.

> > +                       &PCI_SBDF(seg, bdf));
> > +        return 0;
> >      }
> >      req = ivrs_mappings[bdf].dte_requestor_id;
> >  
> 
> Down from here in parse_ivmd_device_iommu() is somewhat similar code.
> Wouldn't that need adjusting similarly then? Or else shouldn't the
> adjustment above be accompanied by a comment clarifying that the
> behavior is just because of observations on certain hardware?

Hm, I think that one is bogus and should be removed, according to my
copy of the AMD-Vi spec (48882—Rev 3.08-PUB—Oct 2023), the IVMD type
can only be:

20h=all peripherals
21h=specified peripheral
22h=peripheral range

So type 23h (ACPI_IVRS_TYPE_MEMORY_IOMMU) is not a valid type for the
IVMD blocks.

Thanks, Roger.



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.