[Xen-devel] [PATCH] xen/pt: fix some pass-thru devices don't work across reboot

[Chao Gao <chao.gao@xxxxxxxxx>]

I find some pass-thru devices don't work any more across guest
reboot. Assigning it to another domain also meets the same issue. And
the only way to make it work again is un-binding and binding it to
pciback. Someone reported this issue one year ago [1].

The root cause is that xen sets the maskall bit in MSI-x capability
during guest shutdown. The maskall bit will be kept even after the
device being assigned to another domain and no guest's operation can
unmask it.

If driver doesn't disable MSI-X during shutdown, the related pirq
won't be unmapped. Then xen will unmap all pirq. But pciback has
already disabled meory decoding before xen unmapping pirq. Then when
Xen is disabling a MSI of the device, it has to sets the maskall bit
to mask a MSI rather than sets maskbit in MSI-x table (seeing the
warning in msi_set_mask_bit()).

To fix this, host_maskall flag is reset and MSI-x maskall bit is
updated. Also 'msix->warned' is initialized to DOMID_INVALID to avoid
warnings missing for Dom0.

[1]: https://lists.xenproject.org/archives/html/xen-devel/2017-09/msg02520.html

Signed-off-by: Chao Gao <chao.gao@xxxxxxxxx>
---
 xen/drivers/passthrough/pci.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index e5b9602..10b8bed 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -327,6 +327,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 
bus, u8 devfn)
             return NULL;
         }
         spin_lock_init(&msix->table_lock);
+        msix->warned = DOMID_INVALID;
         pdev->msix = msix;
     }
 
@@ -1439,7 +1440,27 @@ static int assign_device(struct domain *d, u16 seg, u8 
bus, u8 devfn, u32 flag)
     }
 
     if ( pdev->msix )
+    {
+        uint8_t slot = PCI_SLOT(devfn), func = PCI_FUNC(devfn);
+        uint8_t pos = pci_find_cap_offset(seg, bus, slot, func,
+                                          PCI_CAP_ID_MSIX);
+        uint16_t control = pci_conf_read16(seg, bus, slot, func,
+                                           msix_control_reg(pos));
+        uint16_t new_control;
+
+        /* Reset status owned by Xen */
+        pdev->msix->host_maskall = false;
+        pdev->msix->warned = DOMID_INVALID;
+
+        /* Update 'maskall' bit in MSI-X Capability */
+        new_control = (control & ~PCI_MSIX_FLAGS_MASKALL) |
+                      pdev->msix->guest_maskall;
+        if ( new_control != control )
+            pci_conf_write16(seg, bus, slot, func, msix_control_reg(pos),
+                             control);
+
         msixtbl_init(d);
+    }
 
     pdev->fault.count = 0;
 
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel