Re: [PATCH v6 00/17] Device tree based NUMA support for Arm

[Julien Grall <julien@xxxxxxx>]

Hi Henry,

On 20/11/2023 02:54, Henry Wang wrote:
> The preparation work to support NUMA on Arm has been merged
> and can be found at [1] and [2]. The initial discussions of
> the Arm NUMA support can be found at [3].
>
> - Background of this series:
>
> Xen memory allocation and scheduler modules are NUMA aware.
> But actually, on x86 has implemented the architecture APIs
> to support NUMA. Arm was providing a set of fake architecture
> APIs to make it compatible with NUMA awared memory allocation
> and scheduler.
>
> Arm system was working well as a single node NUMA system with
> these fake APIs, because we didn't have multiple nodes NUMA
> system on Arm. But in recent years, more and more Arm devices
> support multiple nodes NUMA system.
>
> So now we have a new problem. When Xen is running on these Arm
> devices, Xen still treat them as single node SMP systems. The
> NUMA affinity capability of Xen memory allocation and scheduler
> becomes meaningless. Because they rely on input data that does
> not reflect real NUMA layout.
>
> Xen still think the access time for all of the memory is the
> same for all CPUs. However, Xen may allocate memory to a VM
> from different NUMA nodes with different access speeds. This
> difference can be amplified in workloads inside VM, causing
> performance instability and timeouts.
>
> So in this patch series, we implement a set of NUMA API to use
> device tree to describe the NUMA layout. We reuse most of the
> code of x86 NUMA to create and maintain the mapping between
> memory and CPU, create the matrix between any two NUMA nodes.
> Except ACPI and some x86 specified code, we have moved other
> code to common. In next stage, when we implement ACPI based
> NUMA for Arm64, we may move the ACPI NUMA code to common too,
> but in current stage, we keep it as x86 only.
>
> This patch serires has been tested and booted well on FVP in
> Arm64 mode with NUMA configs in device tree and one HPE x86
> NUMA machine.
This series doesn't seem to apply cleanly to staging. Do you have a tree 
with the series applied? This would help to check some of the final 
behavior.
Cheers,

--
Julien Grall