[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: memory overcomittment with sr-iov device assighment
- To: "alex.nlnnfn@xxxxxxxxx" <alex.nlnnfn@xxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
- From: Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>
- Date: Tue, 7 Jun 2022 10:04:07 +0000
- Accept-language: en-GB, en-US
- Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
- Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=R9cTBjs/q4wVN5C/aVgkLZwsXfn2Agxuj5RywH0c0XI=; b=XC8+8VbDXGwj5i2+CoJXXzAyAeDZF6o/PGz8q0uGjQnK2fkvBVaR4KakApIxOIliHNjhJjRhCGiawxv3B97/W8OZMZ4BDWsPBl8NpRAAbR3Z3owYjO/TTYaEyXtL5fgs9Em4s8VqPmi1NAxazIKqsSWFelS/+yUafjPcUq9voaHhwMBo+IhX8Bivs9asVGUUlxW7WYwOxmg3yKpnizlgtQmMrEGnaQsRqJEft9CDLoN4hXOGrIQIo41C7/TqFYY/JN+1HsqBYb+DB0hyXmLVUtpqmJlDDc6pnFbNZr0c3n5IXKYr4M7MDxuAPil3I4KfS22An6cpdXWggpyR1Tftog==
- Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DosVwEvLZoq2PZbQzYpmdYa+a4nMTA3t8e0Vn/8JrMcGhiAaVV67/hpGt0VMi1k/NnjNJglgweejHF0fARDmPzZk0CI/M46gZv2lcBIaDR8g9h4xnd8ciEaGdeG9Zju0Y4oydwSeO6nW3dyLbdUdDzaM4x8rAsFZsga04QAGC1Rozuu6xNiD/NtyqvxIpsU4aqaDTJ1R2/S0NMZcwgF4gbzlZGTZ487yvbJH+IgEqFRrEYSTYn0ACzqvgLW+DivHmK9BXttRmqlsaYNdd/QaGLVHUsICkAH6eywzwcC1jxl4sVaWBAgLyF/qOmdeBVdw+xf8vqu3qqwygMZzEfGinw==
- Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=citrix.com;
- Delivery-date: Tue, 07 Jun 2022 10:04:14 +0000
- Ironport-data: A9a23:cMLKM6OmChRWKA3vrR3OlsFynXyQoLVcMsEvi/4bfWQNrUoq02NRz mtKWjuFPfuOYzDwLt13aI2x/EwAuZSHnIVjSQto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h ynLQoCYdKjYdleF+lH1dOKJQUBUjclkfJKlYAL/En03FFYMpBsJ00o5wbZn2tEw2LBVPivW0 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z1 MRxmq73ThgSEqzQteA5UAsIKTtXBPgTkFPHCSDXXc276WTjKiKp6NI3SUY8MMsf5/p9BnxI+ boAMjcRYxufhuWwhrWmVu1rgcdlJ87uVG8dkig4kXeFUrB7H9aaHP+iCdxwhV/cguhnG/rEa tVfQj1odBnaODVEO0sNCYJ4l+Ct7pX6W2ID9AnE/vBqi4TV5BVB8JTcINzKQOTQfuZHnBq5j GTGoE2sV3n2M/Tak1Jp6EmEj+vCjWX9XIQUGruQ7uRtnFqVgGkeYDUGWF3+rfSnh0qWX9NEN 1dS6icotbI19kGgUp/6RRLQnZKflhsVWt4VGOpj7giIk/PQ+1zAWTJCSSNdYts7ssNwXSYty lKCg9LuA3poraGRTnWesLyTqFteJBQoEIPLXgdcJSNt3jUpiN1bYs7nJjq7LJOIsw==
- Ironport-hdrordr: A9a23:vNjL+KDSbo05/w7lHegOsceALOsnbusQ8zAXPh9KJCC9I/bzqy nxpp8mPEfP+U4ssOlJo6HMBEDyewKmyXcT2/hcAV7CZnithILMFu1fBOTZslnd8kHFl9K1kJ 0QCpSWa+eAQmSS7/yKhzVQeuxIqLbozEnrv5a5854Hd3AIV0gU1XYdNu/tKDwVeOApP/oEPa vZwvACiyureHwRYMj+LGICRfL/q9rCk4+jSQIaBjY8gTP+zA+A2frfKVy1zx0eWzRAzfMJ6m 7eiTH04a2lrrWS1gLc7WnO9J5b8eGRiuerRfb8yfT9GA+czzpAV74RH4FqewpF591H3Wxa0u UkZS1QefibpUmhJ11d6iGdoTUImAxelkMKj2Xoz0cL6PaJOw7TQaB69P5kWwqc5Ew6sN5m1q VXm2qfqppMFBvF2D/w/t7SSnhR5wOJSFcZ4JkuZkZkIP0jgX5q3P8i1VIQFI1FEDPx6YghHu UrBMbA5OxOeVffa3zCpGFgzNGlQ3x2R369MwM/k93Q1yITkGFyzkMeysBalnAc9IglQ50B4+ jfKKxnmLxHU8dTZ6NgA+UKR9exFwX2MFrxGXPXJU6iGLAMOnrLpZKy6LIp5PuycJhN15c2kI SpaiItiYfzQTOaNSSj5uw7zvmWehTCYd3E8LAv27Fp/rvhWbHsLSqPDFgzjsrImYRsPvHm
- List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
- Thread-index: AQHYeihemc0HW39+c0mUd/wlJkeuKK1Dt4iA
- Thread-topic: memory overcomittment with sr-iov device assighment
On 07/06/2022 04:59, alex.nlnnfn@xxxxxxxxx wrote:
> Hello list,
>
> I looked into Xen documentation and also Xen wiki and I could't find a
> definitive answer if Xen supports memory over-commitment when VMs use SR-IOV
> device assignment (passthrough). Memory over-commitment I mean giving VMs
> more RAM than is available in the host.
>
> I know that ESX doesn't support it, and also QEMU/KVM pins all RAM when a
> device is directly assigned to a VM (VFIO_IOMMU_MAP_DMA ioctl). I have two
> questions:
>
> 1. Does Xen supports memory over commitment when all VMs are using direct
> device assignment e.g., SR-IOV?
No. Memory overcommit is fundamentally incompatible with having real
devices.
On the CPU side, EPT_VIOLATION/EPT_MISCONFIG are faults, just like #PF,
so we can play games with swapping out some other part of the guest and
paging in the part which is currently being accessed.
But IOMMU violations are not restartable. We can't just take an IOMMU
fault, and shuffle the guests memory, because the PCIe protocol has
timeouts. These aren't generally long enough to even send an interrupt
to request software help, let alone page one part out and another part in.
For an IOMMU mapping to exists, it must point at real RAM, because any
DMA targetting it cannot be paused and delayed for later.
~Andrew
|
