Re: [PATCH v8 0/6] Device tree based NUMA support for Arm - Part#2

[Jan Beulich <jbeulich@xxxxxxxx>]

On 14.11.2022 07:34, Wei Chen wrote:
> (The Arm device tree based NUMA support patch set contains 35
> patches. In order to make stuff easier for reviewers, I split
> them into 3 parts:
> 1. Preparation. I have re-sorted the patch series. And moved
>    independent patches to the head of the series - merged in [1]
> 2. Move generically usable code from x86 to common - this series.
> 3. Add new code to support Arm.
> 
> This series only contains the second part patches. As the whole NUMA
> series has been reviewed for 1 round in [2], so this series would
> be v8)
> 
> Xen memory allocation and scheduler modules are NUMA aware.
> But actually, on x86 has implemented the architecture APIs
> to support NUMA. Arm was providing a set of fake architecture
> APIs to make it compatible with NUMA awared memory allocation
> and scheduler.
> 
> Arm system was working well as a single node NUMA system with
> these fake APIs, because we didn't have multiple nodes NUMA
> system on Arm. But in recent years, more and more Arm devices
> support multiple nodes NUMA system.
> 
> So now we have a new problem. When Xen is running on these Arm
> devices, Xen still treat them as single node SMP systems. The
> NUMA affinity capability of Xen memory allocation and scheduler
> becomes meaningless. Because they rely on input data that does
> not reflect real NUMA layout.
> 
> Xen still think the access time for all of the memory is the
> same for all CPUs. However, Xen may allocate memory to a VM
> from different NUMA nodes with different access speeds. This
> difference can be amplified in workloads inside VM, causing
> performance instability and timeouts.
> 
> So in this patch series, we implement a set of NUMA API to use
> device tree to describe the NUMA layout. We reuse most of the
> code of x86 NUMA to create and maintain the mapping between
> memory and CPU, create the matrix between any two NUMA nodes.
> Except ACPI and some x86 specified code, we have moved other
> code to common. In next stage, when we implement ACPI based
> NUMA for Arm64, we may move the ACPI NUMA code to common too,
> but in current stage, we keep it as x86 only.
> 
> This patch serires has been tested and booted well on one
> Arm64 NUMA machine and one HPE x86 NUMA machine.
> 
> [1] https://lists.xenproject.org/archives/html/xen-devel/2022-06/msg00499.html
> [2] https://lists.xenproject.org/archives/html/xen-devel/2021-09/msg01903.html
> 
> ---
> v7 -> v8:
>  1. Rebase code to resolve merge conflict.

You mention this here but not in any of the patches. Which leaves
reviewers guessing where the re-base actually was: Re-bases, at
least sometimes, also need (re-)reviewing.

Jan