Re: dom0 PVH: 'entry->arch.pirq != INVALID_PIRQ' failed at vmsi.c:843

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Thu, Nov 12, 2020 at 04:57:15PM +0100, Manuel Bouyer wrote:
> Hello,
> I'm trying to add dom0 PVH support to NetBSD. I'm tesing with Xen 4.13
> on a brand new Intel x86 server (Dell R440).

I would recommend using 4.14, PVH dom0 is still very much in
progress, and while I don't recall any specific fix going in 4.14 that
could be related to this there have been changes.

> While the dom0 kernel configures hardware, Xen panics with:
> (XEN) Xen call trace:
> (XEN)    [<ffff82d08031cc28>] R vpci_msix_arch_mask_entry+0x18/0x20
> (XEN)    [<ffff82d08025a38a>] S drivers/vpci/msix.c#msix_write+0x18a/0x2b0
> (XEN)    [<ffff82d08030d943>] S 
> arch/x86/hvm/intercept.c#hvm_mmio_write+0x23/0x3
> 0
> (XEN)    [<ffff82d08030dd19>] S hvm_process_io_intercept+0x1e9/0x260
> (XEN)    [<ffff82d08030ddad>] S hvm_io_intercept+0x1d/0x40
> (XEN)    [<ffff82d0802fe7ba>] S 
> arch/x86/hvm/emulate.c#hvmemul_do_io+0x26a/0x4d0
> (XEN)    [<ffff82d080259ef9>] S drivers/vpci/msix.c#msix_accept+0x9/0x20
> (XEN)    [<ffff82d0802fea56>] S 
> arch/x86/hvm/emulate.c#hvmemul_do_io_buffer+0x36
> /0x70
> (XEN)    [<ffff82d0802ff005>] S 
> arch/x86/hvm/emulate.c#hvmemul_linear_mmio_access+0x1e5/0x300
> (XEN)    [<ffff82d0802fff44>] S arch/x86/hvm/emulate.c#linear_write+0x84/0x160
> (XEN)    [<ffff82d080301ca8>] S 
> arch/x86/hvm/emulate.c#hvmemul_write+0xe8/0x100
> (XEN)    [<ffff82d0802de6cc>] S x86_emulate+0x289dc/0x2cfb0
> (XEN)    [<ffff82d08027c7ab>] S map_domain_page+0x4b/0x600
> (XEN)    [<ffff82d080340eaa>] S __get_gfn_type_access+0x6a/0x100
> (XEN)    [<ffff82d08034a367>] S 
> arch/x86/mm/p2m-ept.c#ept_next_level+0x107/0x150
> (XEN)    [<ffff82d0802e4961>] S x86_emulate_wrapper+0x21/0x60
> (XEN)    [<ffff82d08030024f>] S 
> arch/x86/hvm/emulate.c#_hvm_emulate_one+0x4f/0x220
> (XEN)    [<ffff82d0803004ed>] S hvmemul_get_seg_reg+0x4d/0x50
> (XEN)    [<ffff82d08030042e>] S hvm_emulate_one+0xe/0x10
> (XEN)    [<ffff82d08030e4ca>] S hvm_emulate_one_insn+0x3a/0xf0
> (XEN)    [<ffff82d0802e4af0>] S x86_insn_is_mem_access+0/0x260
> (XEN)    [<ffff82d08030e5c9>] S handle_mmio_with_translation+0x49/0x60
> (XEN)    [<ffff82d080305d78>] S hvm_hap_nested_page_fault+0x2c8/0x720
> (XEN)    [<ffff82d0802fea56>] S arch/x86/hvm/emulate.c#hv(XEN) 
> (XEN) ****************************************
> (XEN) Panic on CPU 13:
> (XEN) Assertion 'entry->arch.pirq != INVALID_PIRQ' failed at vmsi.c:843
> (XEN) ****************************************
> 
> This is when it configures the broadcom network interface, which interrupts
> at "msix3 vec 0". It is the first MSI-X device configured; the previous
> ones are MSI only.
> 
> Is it a bug on the Xen side, or something missing on the NetBSD side ?

Looks like a bug on the Xen side, do you see any relevant messages
before hitting the assert?

Can you give a try to the following debug patch and paste what you
get?

Thanks, Roger.
---8<---
diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c
index 64dd0a929c..7ff76b7f59 100644
--- a/xen/drivers/vpci/msix.c
+++ b/xen/drivers/vpci/msix.c
@@ -371,7 +371,12 @@ static int msix_write(struct vcpu *v, unsigned long addr, 
unsigned int len,
             entry->updated = false;
         }
         else
+        {
+            printk("%pp offset %u len %u new_masked %d enabled %d masked %d 
updated %d\n",
+                   &pdev->sbdf, offset, len, new_masked, msix->enabled, 
msix->masked,
+                   entry->updated);
             vpci_msix_arch_mask_entry(entry, pdev, entry->masked);
+        }
 
         break;
     }