[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v4] x86/x2apic: introduce a mixed physical/cluster mode
- To: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
- From: Henry Wang <Henry.Wang@xxxxxxx>
- Date: Tue, 7 Nov 2023 00:09:46 +0000
- Accept-language: zh-CN, en-US
- Arc-authentication-results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com])
- Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
- Arc-message-signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BtqyCPIUyHaSeEENDJNns8yZ6jT+ClT1ngFWOl0+vTA=; b=I8fBuP5uvRUeTvXozCWCHMJZWar5eK1gBJlA2DvTBFNF98QN+YoW0JA/vumvhN381Obn9UhfUZ4WZ+4Asae3bbUh269nY18qq++aXcFjMYSn1LGYzZE27x6CaCCt1AqQoGTTzoX5L83zKmGdaG3gW/RSSF1ZAmPWsR2U/j3Sbt7t4emoyZuqzWkr58T2V01G5DV2cAW/4UUDn8OIOUvsYBNtn3MEDuTMQEBbAngfqq6EAgkYRvPzBS6B75RQydup9b1NzOioWNvEZRnBPs2AM+k9AgvCcIIoLCO7yzX30DZN2Ph2qvilu7MI0g4CNt40j5WWYJVbCo694aks6sjuVw==
- Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BtqyCPIUyHaSeEENDJNns8yZ6jT+ClT1ngFWOl0+vTA=; b=J13Fe8nBHbyFWZYZ6w0Zyj67Uv7h2Izi5O9gFUbSt6oIEuHADNnf1bO9+nNVuXRFTS3nHjzvkdQi08AzGXUUxBlrtwhL4fGcavwubTjlvqI6VH+jiIVKxEt+kVVSO5A4mgptIGsKlgi+JI0lcLm+2A8xVJPwfnQLxH+sA2+8nnLVHYGEKmUpI1m6f+2PCgYDDO7Ob0ZAxcaCxRZHPVA04ReKlPxrTq5tlgc8OqUJ4zjI4iWPsESlyXg2Q2XwJq8oNsnjX/V9BzuAxmcrbP8jx2yGToJN3yjDmgkKoX4/wTb6jJYSE5rowLBB9eK1/xw9HbaQqlT4rqV7/KBzsl7iWQ==
- Arc-seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=J4zxhEj3pB+GDFH7HlyvUvHv9vjZ1bJIu+FkOUjCVkn+ZTWvj4oALCR3b3PFJXpefEEA/Vqk75NXocQVGCy02Xh26+5+odYk8q8iQxxnisoGFpw/BLWx0N9ObEsBi1rWQphr+I/FbwqICXCUZ+Va/7/qnhBXU+Z4BcSL79wNtqZwmERd/AiXA8jKhuUi9QtavhGQ4i31BxD8G45eM4iLSdoOW7oS3XTc8aG7rhPx7eg3cKNM45tJwUJQKdWUD8m4mgUOI394Sbu+7eMEc8PeG2bpmdnlqrNITEz8ost5SqHhIBqsMQ0cHuXfxqttujgz5wgJMMVxOv4dY/LeWt+neg==
- Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UbhMhhPorF5kDeamQqre8W0ATeTPJJ84gCJdNkUrHn9KxHySI4e2jIJtgSuISCNvfWfpgVXUkr4Wvja3G7lUFIuFELVNPzvEsPOQUW0foxdDdPQScqZ7tVOdEE2sZ82+y+gGH30RPaVFc5o9hkCvPqNLDcC8XInMNrsLHDqku3NFX1L69tXHezOei1Dx4fEaeXa4Ad60RVAlLjYLAMpLKOkH6pZi8pV+mocwl5f6VFfMg9WtDSGC9dqzpcfBbe2R0holYhbe8gseF+FpGuxO9m/m9/VT9Geh/DtzAd5vXgo7Ox28s8WdL+4+uNJSR75NOGQQFLxQzys+n1754VsRxw==
- Authentication-results-original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
- Cc: Roger Pau Monne <roger.pau@xxxxxxxxxx>, Xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Community Manager <community.manager@xxxxxxxxxxxxxx>, George Dunlap <george.dunlap@xxxxxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>, Julien Grall <julien@xxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Wei Liu <wl@xxxxxxx>
- Delivery-date: Tue, 07 Nov 2023 00:10:30 +0000
- List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
- Nodisclaimer: true
- Original-authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
- Thread-index: AQHaEL2HdnKn60ZZBkKY+yRAm03LbrBtdQ4AgACG2QA=
- Thread-topic: [PATCH v4] x86/x2apic: introduce a mixed physical/cluster mode
Hi Roger,
> On Nov 7, 2023, at 00:06, Andrew Cooper <andrew.cooper3@xxxxxxxxxx> wrote:
>
> On 06/11/2023 2:27 pm, Roger Pau Monne wrote:
>> The current implementation of x2APIC requires to either use Cluster Logical
>> or
>> Physical mode for all interrupts. However the selection of Physical vs
>> Logical
>> is not done at APIC setup, an APIC can be addressed both in Physical or
>> Logical
>> destination modes concurrently.
>>
>> Introduce a new x2APIC mode called Mixed, which uses Logical Cluster mode for
>> IPIs, and Physical mode for external interrupts, thus attempting to use the
>> best method for each interrupt type.
>>
>> Using Physical mode for external interrupts allows more vectors to be used,
>> and
>> interrupt balancing to be more accurate.
>>
>> Using Logical Cluster mode for IPIs allows fewer accesses to the ICR register
>> when sending those, as multiple CPUs can be targeted with a single ICR
>> register
>> write.
>>
>> A simple test calling flush_tlb_all() 10000 times on a tight loop on AMD EPYC
>> 9754 with 512 CPUs gives the following figures in nano seconds:
>>
>> x mixed
>> + phys
>> * cluster
>> N Min Max Median Avg Stddev
>> x 25 3.5131328e+08 3.5716441e+08 3.5410987e+08 3.5432659e+08 1566737.4
>> + 12 1.231082e+09 1.238824e+09 1.2370528e+09 1.2357981e+09 2853892.9
>> Difference at 95.0% confidence
>> 8.81472e+08 +/- 1.46849e+06
>> 248.774% +/- 0.96566%
>> (Student's t, pooled s = 2.05985e+06)
>> * 11 3.5099276e+08 3.5561459e+08 3.5461234e+08 3.5415668e+08 1415071.9
>> No difference proven at 95.0% confidence
>>
>> So Mixed has no difference when compared to Cluster mode, and Physical mode
>> is
>> 248% slower when compared to either Mixed or Cluster modes with a 95%
>> confidence.
>>
>> Note that Xen uses Cluster mode by default, and hence is already using the
>> fastest way for IPI delivery at the cost of reducing the amount of vectors
>> available system-wide.
>>
>> Make the newly introduced mode the default one.
>>
>> Note the printing of the APIC addressing mode done in connect_bsp_APIC() has
>> been removed, as with the newly introduced mixed mode this would require more
>> fine grained printing, or else would be incorrect. The addressing mode can
>> already be derived from the APIC driver in use, which is printed by different
>> helpers.
>>
>> Suggested-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
>> Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
>
> Reviewed-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
Acked-by: Henry Wang <Henry.Wang@xxxxxxx> # CHANGELOG
Kind regards,
Henry
|
