Re: PIRQ handling and PVH dom0?

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Wed, Mar 02, 2022 at 05:49:14PM +0000, Andrew Cooper wrote:
> On 02/03/2022 17:27, Alex Olson wrote:
> > I further attempted to see how far PVH dom0 can get but had a general 
> > question regarding what is not yet implemented... 
> >
> > With an initial version of Roger's recent "vpci/msix: fix PBA access" 
> > patches and after refreshing his earlier 2018 patchset "vpci: add support 
> > for SR-IOV capability" regarding SR-IOV support for PVH dom0, I was able to 
> > get both physical functions and virtual functions of an SR-IOV network card 
> > to operate correctly in PVH dom0.
> >
> > However, it looks like any PCI-passthrough for HVM domUs with PVH dom0 is 
> > not yet implemented. I see the "PHYSDEVOP_map_pirq" call fails since the 
> > "emulation_flags" for dom0 do not include "XEN_X86_EMU_USE_PIRQ"...
> >
> >     libxl: error: libxl_pci.c:1461:pci_add_dm_done: Domain 
> > 1:xc_physdev_map_pirq irq=17 (error=-1): Function not implemented           
> >                                                                             
> >                                                                             
> >                                                                             
> >                                                           
> >     libxl: error: libxl_pci.c:1781:device_pci_add_done: Domain 
> > 1:libxl__device_pci_add failed for PCI device 0:5:0.1 (rc -3)               
> >                                                                             
> >                                                                             
> >                                                                             
> >                                                       
> >     libxl: error: libxl_create.c:1895:domcreate_attach_devices: Domain 
> > 1:unable to add pci devices                                              
> >
> >
> > What is PVH dom0 missing at a conceptual level for PCI passthrough to 
> > domUs?  I naively assumed that an HVM domU guest wouldn't care much whether 
> > dom0 was PV or PVH in terms of passthrough device IRQ handling...
> >
> > Thanks
> 
> Hmm.  xen/arch/x86/hvm/hypercall.c hvm_physdev_op() filters map/unmap
> pirq based on currd.
> 
> But this is buggy.  It should read the physdev_map_pirq_t parameter and
> look at the domid parameter.  What qemu here is doing is trying to map a
> pirq on behalf of the target domain, not on behalf of dom0.

Even doing the filtering based on the domid parameter would be wrong
given the current logic. PHYSDEVOP_{un,}map_pirq are used by both
domains that have physical interrupts routed over even channels and
domains that have interrupts delivered using the emulated interrupt
controllers.

I've posted a fix that should allow you to make further progress:

https://lore.kernel.org/xen-devel/20220303103057.49181-4-roger.pau@xxxxxxxxxx/

It's likely more work will be needed, and it's unsafe to use until we
sort out the issue around locking for PCI devices when used by vPCI.

Thanks, Roger.