Re: Cannot pass through PCI/Intel IGD to Debian Bullseye

[C Stainer <chr.stainer@xxxxxxxxx>]

Hi Chuck,

According to Xen Project 4.15 Feature List [0]...

"Increase the maximum number of guests which can share a single IRQ
from 7 to 16, and make this configurable with irq-max-guests"

Compiling latest Xen 4.15 from source may be applicable to your use case? [1]

[0] https://wiki.xenproject.org/wiki/Xen_Project_4.15_Feature_List
[1] https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source


C

On Wed, 27 Oct 2021 at 00:28, Chuck Zmudzinski <brchuckz@xxxxxxxxxxxx> wrote:
>
> Hello,
>
> I am using Xen for Desktop virtualization of Windows and Linux
> on an older Intel box with a Haswell (4th generation) Intel
> Integrated Graphics Device (IGD) for graphics which is
> now fairly old hardware.
>
> I use Debian for the dom0 and I also use Xen's Intel IGD
> passthrough feature for accelerated graphics in an HVM domU.
>
> With up-to-date Debian stable version 11.1 (Bullseye) for dom0,
> PCI/IGD passthrough works well with Windows HVM domUs, but
> not with modern Linux HVM domUs. I reported this several
> months ago as a bug to the Debian BTS:
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=988333
>
> A quick synopsis of that bug report:
>
> In the dom0:
>
> Debian 11.1 Bullseye version of Xen: 4.14.3-1~deb11u1
> Debian 11.1 Bullseye version of Qemu: 5.2+dfsg-11+deb11u1
> Debian 11.1 Bullseye version of Linux kernel: 5.10.70-1
>
> Two kinds of guests configured for passthrough (a domain config
> I use for a Debian 11.1 HVM is added to the end of this message):
>
> 1. Windows 10 HVM domU: I can configure it to work well with
> passthrough of the Intel IGD, the on-board USB card, and the
> sound card to Windows 10, but only with the rdm_policy set
> to relaxed hack with my old hardware that, AFAICT, does not
> isolate passthrough devices optimally.
>
> 2. Bullseye Debian 11.1 HVM domU: Passthrough with Intel IGD, USB card
> and sound card does not work at all
>
> The main problem I see in the Debian 11.1 dom0 Journal when starting
> a Debian 11.1 HVM domU configured for PCI passthrough of the Intel
> IGD, the USB 3 controller, and the sound card is that IRQ 16 is
> disabled:
>
> ----------------- Start of Journal snippet from dom0 --------------
> dom0 kernel: irq 16: nobody cared ...
> ...
> dom0 kernel: Call Trace:
> dom0 kernel:  <IRQ>
> dom0 kernel:  dump_stack+0x6b/0x83
> dom0 kernel:  __report_bad_irq+0x35/0xa7
> dom0 kernel:  note_interrupt.cold+0xb/0x61
> dom0 kernel:  handle_irq_event+0xa8/0xb0
> ...
> dom0 kernel: handlers:
> dom0 kernel: [<00000000ad934388>] usb_hcd_irq [usbcore]
> dom0 kernel: [<000000003087e3ca>] ath_isr [ath9k]
> dom0 kernel: Disabling IRQ #16
> ---------------- End of Journal snippet from dom0 ---------------
>
>  From what I have read this is caused by an IRQ being assigned
> to a device that does not respond to the IRQ. In the dom0, IRQ
> #16 has been disabled according to the log, but also according
> to the logs, IRQ #16 is assigned to a PCI device that I do not
> passthrough. I am thinking perhaps this device assigned to
> IRQ #16, which is a USB 2 controller, belongs to the same
> IOMMU group (in VFIO terminology) the USB 3 controller that
> I do pass through belongs to:
>
> dom0 kernel: ehci-pci 0000:00:1a.0: irq 16, io mem 0xf053c000
>
> 0000:00:14.0 is the usb 3 controller (xhci) I do passthrough.
>
> So I think the problem with the Linux HVM domU is caused by the
> Linux device drivers in the domU HVM not being as able to deal
> with the poor isolation of the devices on this old box as well
> as the Windows drivers do. Am I on the right track? Can this be
> solved by passing through both this USB 2 controller and the
> USB 3 controller to the domU?
>
> A few more questions:
>
> 1. Does anyone report Intel IGD passthrough with a Debian 11
> dom0 and a Debian 11 domU working on Xen with newer Intel
> hardware, such as 8th generation or newer?
>
> 2. Since IGD passthrough works fine with a Windows HVM, and
> I also discovered [0] an unmodified Debian 11.1 HVM domU works
> fine with a slightly modified old version of Debian 8 for
> the dom0, which uses the old Xen 4.4 hypervisor, the old
> Linux 3.16 kernel, and the old qemu traditional device model
> for PCI/IGD passthrough, does anyone know if it is possible
> to configure or patch the modern Xen 4.14 hypervisor and/or
> the modern qemu upstream device model to get a modern Debian
> 11.1 HVM domU to enable PCI/IGD passthrough to work with
> newer software running in dom0 than what is available from
> the old Debian 8 release that is now way beyond EOL? I
> would prefer to be able to get PCI/IGD passthrough to Debian
> 11.1 to work on a supported dom0 configuration such as a
> Debian 11.1 dom0 with Xen 4.14 instead of unsupported
> Debian 8.x dom0 with unsupported Xen 4.4.
>
> 3. Is this problem likely to be only limited to certain hardware
> that does not isolate the devices as well as modern Linux-based
> software expects?
>
> Thanks in advance for any advice you can give me.
>
> Regards,
>
> Chuck Zmudzinski
>
> [0] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=988333#10
>
> My domain config file (private data such as mac and ip addresses
> are redacted):
>
> builder = 'hvm'
> memory = '3072'
> vcpus = '4'
> device_model_version = 'qemu-xen'
> # device_model_version = 'qemu-xen-traditional'
> # This is a bullseye system (Debian 11.1)
> disk = ['/dev/systems/linux,,xvda,w','/dev/data/linuxdata,,xvdb,w']
> name = 'bullseye-hvm'
> vif = [ 'mac=xx:xx:xx:xx:xx:xx,model=e1000,script=vif-route,ip=A.B.X.Y' ]
> on_poweroff = 'destroy'
> on_reboot = 'restart'
> on_crash = 'restart'
> boot = 'c'
> acpi = '1'
> apic = '1'
> viridian = '1'
> xen_platform_pci = '1'
> serial = 'pty'
> vga = 'none'
> sdl = '0'
> vnc = '0'
> gfx_passthru = '1'
> pci = [ '00:1b.0', '00:14.0,rdm_policy=relaxed', '00:02.0' ]
>