Re: [Xen-users] NUMA nodes and cpu layout

[George Dunlap <dunlapg@xxxxxxxxx>]

On Thu, Jun 14, 2018 at 11:19 PM, Stefan Kadow <stefan.kadow@xxxxxxxxx> wrote:
> Hello,
>
> I have installed Debian 9.4 Stretch (stable) with Xen Hypervisor 4.8
> (package from repository) on a NUMA machine with two AMD Epyc processors.
>
> When booting the default Debian kernel, the command "numactl --hardware"
> returns the following:
> available: 8 nodes (0-7)
> node 0 cpus: 0 1 2 3 32 33 34 35
> node 0 size: 0 MB
> node 0 free: 0 MB
> node 1 cpus: 4 5 6 7 36 37 38 39
> node 1 size: 32170 MB
> node 1 free: 31758 MB
> node 2 cpus: 8 9 10 11 40 41 42 43
> node 2 size: 0 MB
> node 2 free: 0 MB
> ...
>
> But when booting the Xen Kernel, the command "xl cpupool-numa-split"
> returns another layout for cpus and nodes:
> Name               CPU list
> Pool-node0         0,1,2,3,4,5,6,7
> Pool-node1         8,9,10,11,12,13,14,15
> Pool-node2         16,17,18,19,20,21,22,23
> ...
>
> Shouldn't the layout be the same?

No, because the numbers don't mean the same thing.  numactl seems to
be reporting the ACPI IDs of the various logical processors, but ACPI
often gives you nonsensical numbers like you see there -- nodes as
high as 43 even though there are actually only 24 logical cpus,
disjoint sets of numbers for the same node, &c.

Xen uses its own numbering scheme, which is designed to be a bit more
rational: hyperthreads are always next to each other, cores are
contiguous, no holes in the number sequence.

The long and the short of it is: When using Xen tools, use the numbers
reported by `xl info`.  When using numactl, use numbers reported by
numactl.

 -George

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-users