[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [patch V2 00/46] x86, PCI, XEN, genirq ...: Prepare for device MSI



Hi Thomas,

On 8/26/20 7:16 PM, Thomas Gleixner wrote:
This is the second version of providing a base to support device MSI (non
PCI based) and on top of that support for IMS (Interrupt Message Storm)
based devices in a halfways architecture independent way.

After applying this patch series, the dmar_alloc_hwirq() helper doesn't
work anymore during boot. This causes the IOMMU driver to fail to
register the DMA fault handler and abort the IOMMU probe processing.
Is this a known issue?

Best regards,
baolu


The first version can be found here:

     https://lore.kernel.org/r/20200821002424.119492231@xxxxxxxxxxxxx

It's still a mixed bag of bug fixes, cleanups and general improvements
which are worthwhile independent of device MSI.

There are quite a bunch of issues to solve:

   - X86 does not use the device::msi_domain pointer for historical reasons
     and due to XEN, which makes it impossible to create an architecture
     agnostic device MSI infrastructure.

   - X86 has it's own msi_alloc_info data type which is pointlessly
     different from the generic version and does not allow to share code.

   - The logic of composing MSI messages in an hierarchy is busted at the
     core level and of course some (x86) drivers depend on that.

   - A few minor shortcomings as usual

This series addresses that in several steps:

  1) Accidental bug fixes

       iommu/amd: Prevent NULL pointer dereference

  2) Janitoring

       x86/init: Remove unused init ops
       PCI: vmd: Dont abuse vector irqomain as parent
       x86/msi: Remove pointless vcpu_affinity callback

  3) Sanitizing the composition of MSI messages in a hierarchy
genirq/chip: Use the first chip in irq_chip_compose_msi_msg()
       x86/msi: Move compose message callback where it belongs

  4) Simplification of the x86 specific interrupt allocation mechanism

       x86/irq: Rename X86_IRQ_ALLOC_TYPE_MSI* to reflect PCI dependency
       x86/irq: Add allocation type for parent domain retrieval
       iommu/vt-d: Consolidate irq domain getter
       iommu/amd: Consolidate irq domain getter
       iommu/irq_remapping: Consolidate irq domain lookup

  5) Consolidation of the X86 specific interrupt allocation mechanism to be as 
close
     as possible to the generic MSI allocation mechanism which allows to get rid
     of quite a bunch of x86'isms which are pointless

       x86/irq: Prepare consolidation of irq_alloc_info
       x86/msi: Consolidate HPET allocation
       x86/ioapic: Consolidate IOAPIC allocation
       x86/irq: Consolidate DMAR irq allocation
       x86/irq: Consolidate UV domain allocation
       PCI/MSI: Rework pci_msi_domain_calc_hwirq()
       x86/msi: Consolidate MSI allocation
       x86/msi: Use generic MSI domain ops

   6) x86 specific cleanups to remove the dependency on arch_*_msi_irqs()

       x86/irq: Move apic_post_init() invocation to one place
       x86/pci: Reducde #ifdeffery in PCI init code
       x86/irq: Initialize PCI/MSI domain at PCI init time
       irqdomain/msi: Provide DOMAIN_BUS_VMD_MSI
       PCI: vmd: Mark VMD irqdomain with DOMAIN_BUS_VMD_MSI
       PCI/MSI: Provide pci_dev_has_special_msi_domain() helper
       x86/xen: Make xen_msi_init() static and rename it to xen_hvm_msi_init()
       x86/xen: Rework MSI teardown
       x86/xen: Consolidate XEN-MSI init
       irqdomain/msi: Allow to override msi_domain_alloc/free_irqs()
       x86/xen: Wrap XEN MSI management into irqdomain
       iommm/vt-d: Store irq domain in struct device
       iommm/amd: Store irq domain in struct device
       x86/pci: Set default irq domain in pcibios_add_device()
       PCI/MSI: Make arch_.*_msi_irq[s] fallbacks selectable
       x86/irq: Cleanup the arch_*_msi_irqs() leftovers
       x86/irq: Make most MSI ops XEN private
       iommu/vt-d: Remove domain search for PCI/MSI[X]
       iommu/amd: Remove domain search for PCI/MSI

   7) X86 specific preparation for device MSI

       x86/irq: Add DEV_MSI allocation type
       x86/msi: Rename and rework pci_msi_prepare() to cover non-PCI MSI

   8) Generic device MSI infrastructure
       platform-msi: Provide default irq_chip:: Ack
       genirq/proc: Take buslock on affinity write
       genirq/msi: Provide and use msi_domain_set_default_info_flags()
       platform-msi: Add device MSI infrastructure
       irqdomain/msi: Provide msi_alloc/free_store() callbacks

   9) POC of IMS (Interrupt Message Storm) irq domain and irqchip
      implementations for both device array and queue storage.

       irqchip: Add IMS (Interrupt Message Storm) driver - NOT FOR MERGING

Changes vs. V1:

    - Addressed various review comments and addressed the 0day fallout.
      - Corrected the XEN logic (Jürgen)
      - Make the arch fallback in PCI/MSI opt-in not opt-out (Bjorn)

    - Fixed the compose MSI message inconsistency

    - Ensure that the necessary flags are set for device SMI

    - Make the irq bus logic work for affinity setting to prepare
      support for IMS storage in queue memory. It turned out to be
      less scary than I feared.

    - Remove leftovers in iommu/intel|amd

    - Reworked the IMS POC driver to cover queue storage so Jason can have a
      look whether that fits the needs of MLX devices.

The whole lot is also available from git:

    git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git device-msi

This has been tested on Intel/AMD/KVM but lacks testing on:

     - HYPERV (-ENODEV)
     - VMD enabled systems (-ENODEV)
     - XEN (-ENOCLUE)
     - IMS (-ENODEV)

     - Any non-X86 code which might depend on the broken compose MSI message
       logic. Marc excpects not much fallout, but agrees that we need to fix
       it anyway.

#1 - #3 should be applied unconditionally for obvious reasons
#4 - #6 are wortwhile cleanups which should be done independent of device MSI

#7 - #8 look promising to cleanup the platform MSI implementation
        independent of #8, but I neither had cycles nor the stomach to
        tackle that.

#9      is obviously just for the folks interested in IMS

Thanks,

        tglx




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.