[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: PIRQ handling and PVH dom0?


  • To: Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>
  • From: Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • Date: Thu, 3 Mar 2022 11:43:37 +0100
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UqWAYvfvGDjJAHRQH+uBn2KdGRf0bs5xU5VaKskiyI4=; b=Lbea8xShwn3Jo1lzi13vlPNe2XYMOXA/BJv3FrgoyOMkziFAZLAqBzwA271bq28hzS28/5IPi7NqoFH9B34oiHsGMf9z693Sm4HXxtlEaLiTShhpAi8s/927ZHifdF/j5lyzt8k9u02Ryu7C1Yldk7o7sS4uTTTN9rP4vAe+LuCOmD426/5YekT+8oTgIl9MiWj7i9ziGa0ljh0OAQ1asjxgrd5XalF01FHhHOOqPDYcXXtLFB62URuSKVwv2gx8IWhivi0fKwuj5boMzgiKqvetBG3Jz5wlMHLw8dZ2Z6iyFNjPzrnQ4t+cs25LH0eSwnPZWEl3oTxHqX8jv/Lnew==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Uv0FF1bQ+r5wSa6hRtqO/wySPA6/aGvFPAItTrM1ORVYv47fz6uETFzmGd9dSgxvsApcCoiU8BMbTNqrVcNEOahMOpbvNXeRFWbF6DcjPee1Kgp80l6vRYe8mm1wDLTYeaRRUps0Zd4ba0TcthR7nPIJEtQ6IAHngi5CuIt+5VKSx1N7mF8CmJWL58LvrJpXhlgKqGnmiRT5638++ewLXXdTy/WafiGw5HeozVWb5AutU1xIaKckaql2vc1Z3hKQV4nK2+ETMUi8+Wo9BBV1up0r/A55RJ11airOzJ8SKltGo30TpSs/Zl98ySR2xFSSTvEIoibQa617UP1cVq+2VA==
  • Authentication-results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
  • Cc: Alex Olson <this.is.a0lson@xxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Thu, 03 Mar 2022 10:44:00 +0000
  • Ironport-data: A9a23:g73L2a3p8j1Ea35AyvbD5dhxkn2cJEfYwER7XKvMYLTBsI5bpzRRx 2FJDWiFMviKYWTyct4iadni8k9QsZfRxtZkQFZvpC1hF35El5HIVI+TRqvS04J+DSFhoGZPt Zh2hgzodZhsJpPkjk7xdOCn9xGQ7InQLlbGILes1htZGEk1EE/NtTo5w7Rj2tUw2oDga++wk YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx /0VjrrvTg0WOpfN28FECD0HKiJvGYd/reqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr KJCbmpXN1ba2rzwkOnTpupE36zPKOHxO4wSoDd4xCzxBvc6W5HTBa7N4Le02R9u25ofTKiGN 6L1bxJQZhnkQwJzKmwRGZElkM72mCLhYQBH/Qf9Sa0fvDGIkV0ZPKLWGMrYfJmGSNtYmm6cp 3na5CLpDxcCLtudxDGZtHW2iYfngSP6Q8QTD/uxrvpxh1u7yWkaCRlQXly+ycRVkWbnBYgZc RZNvHNz8+5iryRHU+URQTXgm1jbuRQjX+BRUMhjsyXS86nFxyygUz1soiF6VPQqs8o/RDoP3 1CPns/0CTEHjIB5WU5x5Z/P82rsZHF9wXsqIHZdEFBbu4WLTJQb00qXJuuPBpJZmTEc9dvY5 zmR5BYziLwI5SLg//XqpAuX695AS3Wgc+LU2uk1dj/9hu+aTNT8D2BN1bQ9xawaRGp+ZgPc1 EXoY+DEsIgz4WilzURhutklErCz/OqiOzbBm1NpFJRJ323zpyD5Id4MsWoheR4B3iM4ldnBO hW7VeR5vsI7AZdXRfUvP9LZ5zoCl8AM6ugJptiLN4ETM/CdhSeM/T10ZF744oweuBNErE3LA r/CKZzEJS9DUcxPlWPqL89Age5D7n1vngv7GMGkpylLJJLDPRZ5v59eawDQBg34hYvZyDjoH yF3bJPbm00CC7SlOkE6M+c7dDg3EJTyPrivw+R/fe+fOAt2XmYnDv7a27Q6fIJ52a9Sk4/1E ruVACe0FHKXaaX7FDi3
  • Ironport-hdrordr: A9a23:7qHh9q+Y8zXBUet8P3duk+E8db1zdoMgy1knxilNoENuHPBwxv rAoB1E73PJYVYqOE3Jmbi7Sc+9qFfnhONICO4qTMuftWjdyRGVxeRZjLcKrAeQfhEWmtQtsZ uINpIOd+EYbmIK/foSgjPIa+rIqePvmMvD6Ja8vhVQpENRGtpdBm9Ce3em+yZNNXB77PQCZf 2hDp0tnUvfRZ1bVLXxOlA1G8z44/HbnpPvZhALQzYh9Qm1lDutrJr3CQKR0BsyWy5Ghe5Kyx mJryXJooGY992rwB7V0GHeq7xQhdva09NGQOiBkNIcJDnAghuhIK5hR7qBljYop/zH0idhrP D85zMbe+hj4XLYeW+45TPrxgnbyT4rr0TvzFeJ6EGT1/DRdXYfMY5slIhZehzW5w4Lp9dnyp 9G2Gqfqt5+EQ7AtD6V3amHazha0m6P5VYym+8aiHJSFaEEbqVKkIAZ9ERJVL8dASPB7pw9Gu UGNrCS2B9vSyLbU5nlhBgt/DT1NU5DXCtuA3Jy9vB96gIm3UyQlCAjtYkidnRpzuNLd3AL3Z WBDk1SrsA9ciYnV9MPOA4/e7rDNoXse2OEDIvAGyWuKEk4U0i936Ifpo9Fo92XRA==
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On Wed, Mar 02, 2022 at 05:49:14PM +0000, Andrew Cooper wrote:
> On 02/03/2022 17:27, Alex Olson wrote:
> > I further attempted to see how far PVH dom0 can get but had a general 
> > question regarding what is not yet implemented... 
> >
> > With an initial version of Roger's recent "vpci/msix: fix PBA access" 
> > patches and after refreshing his earlier 2018 patchset "vpci: add support 
> > for SR-IOV capability" regarding SR-IOV support for PVH dom0, I was able to 
> > get both physical functions and virtual functions of an SR-IOV network card 
> > to operate correctly in PVH dom0.
> >
> > However, it looks like any PCI-passthrough for HVM domUs with PVH dom0 is 
> > not yet implemented. I see the "PHYSDEVOP_map_pirq" call fails since the 
> > "emulation_flags" for dom0 do not include "XEN_X86_EMU_USE_PIRQ"...
> >
> >     libxl: error: libxl_pci.c:1461:pci_add_dm_done: Domain 
> > 1:xc_physdev_map_pirq irq=17 (error=-1): Function not implemented           
> >                                                                             
> >                                                                             
> >                                                                             
> >                                                           
> >     libxl: error: libxl_pci.c:1781:device_pci_add_done: Domain 
> > 1:libxl__device_pci_add failed for PCI device 0:5:0.1 (rc -3)               
> >                                                                             
> >                                                                             
> >                                                                             
> >                                                       
> >     libxl: error: libxl_create.c:1895:domcreate_attach_devices: Domain 
> > 1:unable to add pci devices                                              
> >
> >
> > What is PVH dom0 missing at a conceptual level for PCI passthrough to 
> > domUs?  I naively assumed that an HVM domU guest wouldn't care much whether 
> > dom0 was PV or PVH in terms of passthrough device IRQ handling...
> >
> > Thanks
> 
> Hmm.  xen/arch/x86/hvm/hypercall.c hvm_physdev_op() filters map/unmap
> pirq based on currd.
> 
> But this is buggy.  It should read the physdev_map_pirq_t parameter and
> look at the domid parameter.  What qemu here is doing is trying to map a
> pirq on behalf of the target domain, not on behalf of dom0.

Even doing the filtering based on the domid parameter would be wrong
given the current logic. PHYSDEVOP_{un,}map_pirq are used by both
domains that have physical interrupts routed over even channels and
domains that have interrupts delivered using the emulated interrupt
controllers.

I've posted a fix that should allow you to make further progress:

https://lore.kernel.org/xen-devel/20220303103057.49181-4-roger.pau@xxxxxxxxxx/

It's likely more work will be needed, and it's unsafe to use until we
sort out the issue around locking for PCI devices when used by vPCI.

Thanks, Roger.



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.