Restartinf a PVHVM domU with pci-passthrough hangs on Xen 4.13.1

[Chris <chris@xxxxxxxxx>]

Hi all,


I did several tests on different versions of Xen with these results concerning 
the reboot of a domU with pci-passthrough (hardware : Supermicro A2SDi-4C-HLN4F 
with one LSI 9211-4i dedicated to a PVHVM domU) :
* Xen 4.12.1 : OK with domU's reboot
* Xen 4.12.2 : OK with domU's reboot
* Xen 4.12.3 : Not tested
* Xen 4.13.0 : OK with domU's reboot
* Xen 4.13.1 : Not OK with domU's reboot (dom0 has to be rebooted ...)

The issue seems to have reappeared (cf thread on xen-users titled "PCI 
pass-through issue with Xen 4.11 under ubuntu 20.04") ... Annoying ...

By dichotomy, I tried with a custom compiled version of Qemu 4.1.1 (version 
used by Xen 4.13.1 instead of Qemu 4.1.0 with Xen 4.13.0) which allows to 
reboot a VM vith the HBA without issue on KVM (I compiled Qemu and the kernel - 
5.4.42 so far - with the support of both KVM and Xen. So I can switch between 
them for tests). But with Xen 4.13.1, no luck, the domU waits forever during 
restart on the PCI device. However, nothing seems wrong with the logs on the 
dom0.

Relevant logs on the domU's console (Freenas 11.3) : 
"mps0 : Calling Reinit from mps_wait_command, timeout=60, elapsed=60
mps0 : Reinitializing controller,"
At this stage, destroying the domU raises this warning :
"libxl: warning: libxl_pci.c:2141:pci_remove_timeout: Domain 4:timed out 
waiting for DM to remove pci-pt-02_00.0"

If I try to dedicate a NVME device to the PVHVM domU, the first boot is OK, 
but, as I reboot the VM, this log appears on the domU's console (still Freenas 
11.3) :
"nvme0: Missing interrupt"

I could make other tests if a dev or a Xen guru could guide me !


Regards,


Chris