[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: memory overcomittment with sr-iov device assighment
- To: Alex Nln <alex.nlnnfn@xxxxxxxxx>
- From: Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>
- Date: Tue, 7 Jun 2022 19:47:02 +0000
- Accept-language: en-GB, en-US
- Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
- Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ccZ9Gby7PCU6na6QwRjpasAAHx9d+SrXzMkgZ3b6Xhg=; b=HZg/bMtVkCsyfCJmgaL0bRat1xqWncdKhr8GGbGPeqrH22OnOBNE9XHUnY1RLmzdG/4WrbejZZ3Sc638Add8CSB9/mVMwmKJmGXqWs8kKDVTqVnVtLeORUegEKz21/IPzKC+VeyFZqSU1mZd1t2kLkXE/GK4xUTdsVm24P0c+9omn/bUudxOWdOKzCqbV8pcNnE1OWs+7VrMQiEucHqPUFUpeRhVq/csmyGNx0i+VExZMCHqWVTp38alMfIBvsB71noozNR48dVpStJ8WxxvqRYOHWnJL9aR8qPaomllc6FtwBhP9zYJRA3IVPHqiafJDKkdEeZDcG2jkFLWLZt74A==
- Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kH/OvUob82g+X1Pov5fBcpCUBhsU8rsqd4h//My2ZzVoI4DnGHb4qUD+h3f9Zfms6V6IGCWbrPqzU/z9fYEvyl7AWu+Sa/Hb1rE6pCnSZkVDWPI3XYxsgvI/XJJMg6AQk/TsWj+hGao/Q3ExwMMVFxjA+/j2Ya0jY9Hqu6nrVuiK6InAHjWeF6rB9pjIGX0wuLnFcwIlWr3/62Jelvm1+i4N7CSCn8jZsX61iAt4PF1YDXio+ssdWX/bQrG1bG3MH/v+pb1iciqbbIn1gcDtLI2irKSJ1gSEi9+09oohBr5bLIdlJ6P9pyQs6sazGFpEz6zuwIrAlm4NnyJQ6ATPnQ==
- Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=citrix.com;
- Cc: "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
- Delivery-date: Tue, 07 Jun 2022 19:47:23 +0000
- Ironport-data: A9a23:zMqNraLrEnedYsRzFE+RuZQlxSXFcZb7ZxGr2PjKsXjdYENS0zMCy WEXXDzUOqmDY2ujKNx1YN60pE0Gv5aGxoRkSlBlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s q3yv/GZdJhcokf0/0vrav67xZVF/fngqoDUUYYoAQgsA149IMsdoUg7wbRh3NY32YLR7z6l4 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4 OtplrO9SRwQBbaWpb9GQT91Mg8uDIQTrdcrIVDn2SCS52vvViOwht9IXAQxN4Be/ftrC2ZT8 /BeMCoKch2Im+OxxvS8V/VogcMgasLsOevzuFk5lW2fUalgHsyFH/SiCdxwhV/cguhnG/rEa tVfQj1odBnaODVEO0sNCYJ4l+Ct7pX6W2IC+QjF+vRti4TV5B5SyZfLHfGKQOGTGeoJj2qnj GXm42usV3n2M/Tak1Jp6EmEj+vCjWX9XIQUGruQ7uRtnFqVgGkeYDUGWF3+rfSnh0qWX9NEN 1dS6icotbI19kGgUp/6RRLQnZKflhsVWt4VGetq7giIkvbQ+1zBWjlCSSNdYts7ssNwXSYty lKCg9LuA3poraGRTnWesLyTqFteJBQoEIPLXgdcJSNt3jUpiNhbYs7nJjq7LJOIsw==
- Ironport-hdrordr: A9a23:7dExyqsItahImT/j+mKJylsE7skCL4Aji2hC6mlwRA09TyXGra 2TdaUgvyMc1gx7ZJh5o6H6BEGBKUmslqKceeEqTPuftXrdyRGVxeZZnMTfKlzbamDDH4tmuZ uIHJIOb+EYYWIasS++2njBLz9C+qjIzEnLv5a5854Fd2gDBM9dBkVCe3+m+yZNNWt77O8CZf 6hD7181l+dkBosDviTNz0gZazuttfLnJXpbVotHBg88jSDijuu9frTDwWY9g12aUIO/Z4StU z+1yDp7KSqtP+2jjXG0XXI0phQkNz9jvNeGc23jNQPIDmEsHfqWG0hYczBgNkGmpDq1L8Yqq iKn/7mBbU015rlRBDxnfIq4Xi47N9h0Q679bbSuwqfnSWwfkNHNyMGv/MZTvKR0TtfgDk3up g7oF6xpt5ZCwjNkz/64MWNXxZ2llCsqX5niuILiWdDOLFuIIO5gLZvin+9Kq1wVR4SKbpXYt VGHYXZ/rJbYFmaZ3fWsi1mx8GtRG06GlODTlIZssKY3jBKlDQhpnFojvA3jzMF7tYwWpNE7+ PLPuBhk6xPVNYfaeZ4CP0aScW6B2TRSVbHMX6UI17gCKYbUki94KLf8fEw/qWnaZYIxJw9lN DIV05Zr3c7fwb0BciHzPRwg2fwqaWGLEDQI+1lluhEU+fHNcvW2AW4OSMTutrlpekDCcvGXP v2MI5KApbYXB7TJbo=
- List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
- Thread-index: AQHYeihemc0HW39+c0mUd/wlJkeuKK1Dt4iAgACcCYCAAAbTgA==
- Thread-topic: memory overcomittment with sr-iov device assighment
On 07/06/2022 20:22, Alex Nln wrote:
> ------- Original Message -------
> On Tuesday, June 7th, 2022 at 3:04 AM, Andrew Cooper
> <Andrew.Cooper3@xxxxxxxxxx> wrote:
>> But IOMMU violations are not restartable. We can't just take an IOMMU
>> fault, and shuffle the guests memory, because the PCIe protocol has
>> timeouts. These aren't generally long enough to even send an interrupt
>> to request software help, let alone page one part out and another part in.
>>
>> For an IOMMU mapping to exists, it must point at real RAM, because any
>> DMA targetting it cannot be paused and delayed for later.
>>
> Thanks for the information!
>
> Speaking about IOMMU. Can Xen emulate virtual IOMMU? Just thinking out loud.
> If Xen exposes a virtual IOMMU to a guest VM, wouldn't it be possible
> for the hypervisor to pin VM's pages that are mapped (dynamically)
> by the guest virtual IOMMU? But I guess it will eliminate the performance
> benefits of direct device assignment.
As Jan pointed out, there's no such thing as pinning like that. Xen is
not KVM. See
https://xenbits.xen.org/docs/latest/admin-guide/introduction.html
But "can you only deal with the subset mapped in a virtual IOMMU" is a
reasonable question. The answer is no, for two reasons.
First, what do you do if the guest maps it's whole guest physical
address space in the virtual IOMMU? There's no "sorry - not got enough
memory to arrange that" error available in either Intel or AMD's IOMMU
spec, because there's no such thing as memory overcommit in a real system.
Secondly and more importantly, the at-reset behaviour of an IOMMU is no
translation, which for a guest means that DMA can go anywhere in the
guest physical address space, meaning the whole guest physical address
space needs mapping into the IOMMU pagetables,
~Andrew
|
