[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: [PATCH] Rework CACHE to use a FreeList
- To: Paul Durrant <xadimgnik@xxxxxxxxx>, "win-pv-devel@xxxxxxxxxxxxxxxxxxxx" <win-pv-devel@xxxxxxxxxxxxxxxxxxxx>
- From: Owen Smith <owen.smith@xxxxxxxxxx>
- Date: Mon, 22 Aug 2022 07:24:31 +0000
- Accept-language: en-GB, en-US
- Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
- Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w3eZlgyYjH9GuuHOC73KZe4lWtGUCZ4dspOAAKmXs4A=; b=Es/KcKNsflTq1uBQnv11dMyqCLEWyRGTnB29CqzhLrOtXlFAyc7/72tjsb8SO7pObXwTgqIfgB40TAWVgew74ZNeLC8sIuKciaM/WIPXCIGeAupeRs8kZZZY1uEocw8mtGP8hSkl7jWtYOVa/zPiP5McPijqDq8yVXLwHilvuvi61bX3zLx57TnNvBmppRcY+Mp3acNQ4tTEgPKMvj2CNct/8YoPLscsFucEcaWjrHqxLC4QlPZDav1JTm0RUFBpeqHMir6jxoyFsGmSPciKg4l5u6pHjKYUKUAvlyFLRDmr5nStAHLKT6dGVPpCeOQVZ+dofy0tYrt4213w+PeNsw==
- Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kX+6RNiL0hF9W+NsUTqWCuCeGTlmbEn8jA0L+nRNSG0d1Jeg5iD6gVJRUPhFMsK+2gCpZVTD2pptmyMYYWXMfILP9U+sjNsfEfje8riSFzyIMHroDQKvShSp8Jv7o2I0DfVG/tYkPmsNk2tunN6ATINSv/BnBJFJAXuJEc8ZzFrEcD5/fd38PGceqGjjc52z/iWazdMBRBKObMXBf4ggwVc+7IicE5qsKinIqCc/G69NvSV3PRJMQyCVEuHdPfPJ+vX8DN0OtZ8K1GV/ZlZFOJoLxoYvUbORfGp0s1QsDSQ6R2xt4tBLqOu0Ky0VeQToe14ctjzxx7XW+w/7a8l4PA==
- Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=citrix.com;
- Delivery-date: Mon, 22 Aug 2022 07:24:41 +0000
- Ironport-data: A9a23:X8k5NK9sRMDv1mT9nJ2gDrUDlH+TJUtcMsCJ2f8bNWPcYEJGY0x3m DFNW2qCO/7YMzHxc4p/O4Ti/UoA68eAndBgTgZsq3o8E34SpcT7XtnIdU2Y0wF+jyHgoOCLy +1EN7Es+ehtFie0Si+Fa+Sn9z8kvU2xbuKUIPbePSxsThNTRi4kiBZy88Y0mYctitWia++3k YqaT/b3ZRn0gFaYDkpOs/jZ8EM15qyr0N8llgdWic5j7Qe2e0Y9VPrzFYnpR1PkT49dGPKNR uqr5NlVKUuAon/Bovv8+lrKWhViroz6ZGBiuVIPM0SWuTBQpzRa70oOHKF0hXG7Kdm+t4sZJ N1l7fRcQOqyV0HGsLx1vxJwS0mSMUDakVNuzLfWXcG7liX7n3XQL/pGEhAVBr1H+NlNMWwJ+ eYXBCgnShCNvrfjqF67YrEEasULCuDOZdtallQ+iDbTALAhXIzJRLjM6ZlAxjAsi8tSHPHYI c0EdT5oaxeGaBpKUrsVIMtmwKH02T+iI3sB8A39SakfugA/yCR2z7XrdsHQe8CKbc5UglyZt iTN+GGR7hQya43HkmPUoy3EaunnxRrGZo0jSIyCyaBguG+W23QoDT8pWg7uyRW+ogvkMz5FE GQE8yYvqKc09U+DQdz0Xhn+q3mB1jYDWtwVC/N/5AyTx6785weCGnNCXjNHcMYhtsI9WXotz FDhoj/yLTlmsbnQRXfD8L6R9Gu2IXJMdTBEYjIYRwwY5dWluJs0kh/EUtdkFuiyk8HxHjbzh TuNqUDSmokusCLC7I3jlXivvt5mjsKhotIdjukPYl+Y0w==
- Ironport-hdrordr: A9a23:xditLKodJXTC3ZGgGVwvEvkaV5uVL9V00zEX/kB9WHVpm5Oj+v xGzc5w6farsl0ssSkb6Ki90dq7MAjhHPlOkMIs1NaZLUHbUQ6TQL2KgrGSpwEIdxeeygcZ79 YYT0EcMqy7MbEZt7ec3ODQKb9Jr7e6GeKT9IHjJhxWPGJXgtRbnmJE43GgYy9LrWd9ZaYRJd 653I5qtjCgcXMYYoCQHX8eRdXOoNXNidbPfQMGLwRP0njDsRqYrJrBVzSI1BYXVD1ChZ0493 LergD/7qK/99mm1x7n0XPJ5Zg+oqqv9jIDPr3DtiEmEESttu+aXvUjZ1REhkF2nAib0idqrD ALmWZkAy080QKUQoj/m2qW5+Cp6kdS15al8y7UvZKrm72EeBsqT8VGno5XaR3f9g4pu8x9yr tC2yaDu4NQFg6oplWL2zHkbWAeqqOPmwtXrccDy3hEFYcOYr5YqoISuEtTDZcbBSr/rIQqCv NnAs3Q7OtfNQryVQGRgkB/hNi3GngjFBaPRUYP/sSTzjhNhXh8i08V3tYWkHsM/I80D5NE++ PHOKJ1k6wmdL5hUYttQOMaBcenAG3ERhzBdGqUPFT8DakCf2nArpbmiY9Flt1CuKZ4v6fatK 6xLm+w71RCBH4GIff+raFj41TKXHi3Wyjrx4VX+4V504eMNIbWDQ==
- List-id: Developer list for the Windows PV Drivers subproject <win-pv-devel.lists.xenproject.org>
- Thread-index: AQHYs7Sg9Rrov4BadECgnNCp5OuXFq22aG4AgAQeHLA=
- Thread-topic: [PATCH] Rework CACHE to use a FreeList
-----Original Message-----
From: win-pv-devel <win-pv-devel-bounces@xxxxxxxxxxxxxxxxxxxx> On Behalf Of
Paul Durrant
Sent: 19 August 2022 17:23
To: win-pv-devel@xxxxxxxxxxxxxxxxxxxx
Subject: Re: [PATCH] Rework CACHE to use a FreeList
[CAUTION - EXTERNAL EMAIL] DO NOT reply, click links, or open attachments
unless you have verified the sender and know the content is safe.
On 19/08/2022 11:15, Owen Smith wrote:
> The slab allocation method will allogate about a PAGE worth of
> objects, and every object will be initialized. If the objects
> initializer allocates any resources, this can result in resource
> starvation. A particular bad example of this is the grant table cache, where
> a page of gnttab objects is 253 objects.
> This is highlighted by xenvif's queues, where the receiver requires
> 257 grant references (1 for the ring, and 256 for the ring slots)
> which results in 2 slabs, or 506 gnttab objects, reserving 506 grant
> references.
>
> Use a FreeList to contain individual objects that are not in use. This
> trades an increase in smaller allocations for reducing the wastage of unused
> objects.
>
> Signed-off-by: Owen Smith <owen.smith@xxxxxxxxxx>
Sorry I didn't comment on the RFC; I was on PTO and then snowed under with mail
etc.
I think this is a sledgehammer to crack a nut. I agree that XENVIF is being a
grant ref hog... but the correct thing to do there is to re-work the grant
table cache, not the underlying slab allocator; which I think is fine. The
problem is (ab)using the slab allocator's Ctor to get the reference. So a free
list implementation is fine... just in the gnttab code, rather than the cache
code.
Paul
This also hits any other use of the CACHE interface where the object's Ctor
allocates a resource and there are many objects in a slab. XenVbd's segment
cache allocates a page, which can lead to failures with large number of VBDs
(one of our automated test cases fails with a 0x4B NO_PAGES_AVAILABLE when run
with 256 VBDs - I cant remember exactly how many VBDs are required).
The alternative here would be to rework the GNTTAB interface to use a free list
rather than the CACHE interface, and rework any other uses where the Ctor could
allocate significant resources (possibly using a lazy allocation of memory, but
this could Fill without enough Spill to still lead to resource exhaustion)
Owen
|
