RE: [PATCH v2 01/17] xen/arm: use NR_MEM_BANKS to override default NR_NODE_MEMBLKS

[Wei Chen <Wei.Chen@xxxxxxx>]

Hi Jan,

> -----Original Message-----
> From: Jan Beulich <jbeulich@xxxxxxxx>
> Sent: 2023年1月10日 18:00
> To: Wei Chen <Wei.Chen@xxxxxxx>
> Cc: nd <nd@xxxxxxx>; Stefano Stabellini <sstabellini@xxxxxxxxxx>; Julien
> Grall <julien@xxxxxxx>; Bertrand Marquis <Bertrand.Marquis@xxxxxxx>;
> Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>; Andrew Cooper
> <andrew.cooper3@xxxxxxxxxx>; George Dunlap <george.dunlap@xxxxxxxxxx>; Wei
> Liu <wl@xxxxxxx>; xen-devel@xxxxxxxxxxxxxxxxxxxx
> Subject: Re: [PATCH v2 01/17] xen/arm: use NR_MEM_BANKS to override
> default NR_NODE_MEMBLKS
> 
> On 10.01.2023 09:49, Wei Chen wrote:
> > --- a/xen/arch/arm/include/asm/numa.h
> > +++ b/xen/arch/arm/include/asm/numa.h
> > @@ -3,9 +3,26 @@
> >
> >  #include <xen/mm.h>
> >
> > +#include <asm/setup.h>
> > +
> >  typedef u8 nodeid_t;
> >
> > -#ifndef CONFIG_NUMA
> > +#ifdef CONFIG_NUMA
> > +
> > +/*
> > + * It is very likely that if you have more than 64 nodes, you may
> > + * need a lot more than 2 regions per node. So, for Arm, we would
> > + * just define NR_NODE_MEMBLKS as an alias to NR_MEM_BANKS.
> > + * And in the future NR_MEM_BANKS will be bumped for new platforms,
> > + * but for now leave NR_MEM_BANKS as it is on Arm. This avoid to
> > + * have different way to define the value based NUMA vs non-NUMA.
> > + *
> > + * Further discussions can be found here:
> > + * https://lists.xenproject.org/archives/html/xen-devel/2021-
> 09/msg02322.html
> > + */
> > +#define NR_NODE_MEMBLKS NR_MEM_BANKS
> 
> But isn't NR_MEM_BANKS a system-wide setting, which doesn't really
> make sense to use as a per-node one? IOW aren't you now allowing
> NR_MEM_BANKS regions on each node, which taken together will be
> much more than NR_MEM_BANKS that you can deal with on the whole
> system?
> 

Thanks of pointing out this. Yes NR_MEM_BANKS a system-wide setting,
but we just use it to define the MAX memory banks for single node,
this does not mean that there are really so many banks on these nodes.
When a system only contains one node NR_MEM_BANKS equals to
NR_NODE_MEMBLKS. Our idea is that, if the total memory banks of
all nodes exceed the NR_MEM_BANKS, we will bump NR_MEM_BANKS.

But I am open to this question, if there are more suggestions from
maintainers.

Cheers,
Wei Chen


> Jan