[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [PATCH v4 2/6] xen/x86: move generically usable NUMA code from x86 to common


  • To: Jan Beulich <jbeulich@xxxxxxxx>
  • From: Wei Chen <Wei.Chen@xxxxxxx>
  • Date: Thu, 8 Sep 2022 14:52:47 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com])
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
  • Arc-message-signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eYwq38ij0WcLT7MnjaCpNaXYlUNRMMsOg1q+J7cC/Xk=; b=P0URd0EmomavJzOYyDOVS1sTbgkOjNTt4Q+GAa66CWG0qTcwARqKGN7k6O5bKPB8AMOUSvmv8a622n2hvE/GS8n8ahJ0DdktiFf/dui9JNUQwIzflro26LxFCnJnwLFdFsolqQsnBqLa7fJ9fzaAM2uc+fnh+4o0z7zor1gebkrQGSNAC8rzv3yHnYQooGVxo44r2BYxLK1znk7oatgNGDpGS5eGSeFDENrLeago2l2RS8yu6D66inc4CXoCXVCR4F5FbVGCLUdhAHM1E8IMoWGuQPF8ONFDrHkjT0lqT0YL7t9VC4fDqk1qFQU2B8q+3XtT4pjRXhLtBCKJCWilkA==
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eYwq38ij0WcLT7MnjaCpNaXYlUNRMMsOg1q+J7cC/Xk=; b=Rdeg89KEQIFCyHUVy601H8UtwcTJi7i+yO6zeLBxycpBVmf8SkM/wSYQ6wynED1Bf+HitC/VLAZolev7rDOZl0jfuFFT2Y7RbQcPTneD8u5/VMR33UjUCftFt9xdxde2GH/WvEy4khKnAnftmiXmex7A5g4ojHkmJGz/9mqKN+rNpfTL1Mp+WYnRhSNhxXme5xG90193Jj+Sduda9gMlqzLwBEfQiJTXzasFu7fsEkRXKUaZhMzCrz84CrxTbRX74RmK+oTykGuAoVjyJ31JeetU6nkzuk8bGBFJcOII+7164OKz+JZQilbu4VAY+qACV4uXu5cap7k/eveM0r6aZQ==
  • Arc-seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=K7IDXYl2M1Cruu98GmZi1ZhJ0Qy68u8VZmcpIDoppdtvEk+/mJ7Xf92MI6oehWd9lp4BTFM4BGpdnYjnQbhGK7bN6IS/ur6xrvIdE12qabloGNMy+t/FesTIBZXryqYrG/0lvAq5kB/PwPzwv1yhhQbuRgogiq7DOzSJW6eqUU6p2k8jSSUkQ0vb7v3PRGVYeBYP/g/XdRFxFQ03C8P4k9AI/Ebblky3qLyMvOnlNiwDTEnmgBnw0kRBVwWOP1N26A+/oEwFYjTgEli1PwV1KUogSmvNAO6o2QClUd+cltULKGh0/r7c1l/ZmMhZo8xVccJeGWOkXrDpnaY/4Q8+7A==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Gbh6TVkjthrRpGfHNoZOc48QQauRyGBiKpRyEY+Ql+qt72TBLks7eKWspU3BFk0irmPqOvxWFx+r+K0qUaVkX+3VxhV7dExsaEG/CtE9JA/rq3VkMXnMN8xNVsnvwOi2Ng4jSYQCX9PwT6w9N7zPeCM8/aRD/qznkptl2ui7WTZJYUo7iTp2/dQKzma4OovCBQNGAVkgwDBZKUxH3lF3pPXXpIWM43Z0/6Kdf5nbVh9tTzOtOZIAUD5goP1jaAMsWQ/28tHckoiiqCMeGouHiR4rgMET3fZzu3NgC0PZp+hZOzcF8rd2Nnxt9lu3LG5F5x9ObTeEImXc9PwNWHcygg==
  • Authentication-results-original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
  • Cc: nd <nd@xxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Roger Pau Monné <roger.pau@xxxxxxxxxx>, Wei Liu <wl@xxxxxxx>, George Dunlap <george.dunlap@xxxxxxxxxx>, Julien Grall <julien@xxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Thu, 08 Sep 2022 16:39:07 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Nodisclaimer: true
  • Original-authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
  • Thread-index: AQHYvnyKHl6xJ/9ji0S8MAXJnXBUfq3VSGGAgACdXwD//41NgIAANFtg
  • Thread-topic: [PATCH v4 2/6] xen/x86: move generically usable NUMA code from x86 to common

Hi Jan,

> -----Original Message-----
> From: Jan Beulich <jbeulich@xxxxxxxx>
> Sent: 2022年9月8日 19:42
> To: Wei Chen <Wei.Chen@xxxxxxx>
> Cc: nd <nd@xxxxxxx>; Andrew Cooper <andrew.cooper3@xxxxxxxxxx>; Roger Pau
> Monné <roger.pau@xxxxxxxxxx>; Wei Liu <wl@xxxxxxx>; George Dunlap
> <george.dunlap@xxxxxxxxxx>; Julien Grall <julien@xxxxxxx>; Stefano
> Stabellini <sstabellini@xxxxxxxxxx>; xen-devel@xxxxxxxxxxxxxxxxxxxx
> Subject: Re: [PATCH v4 2/6] xen/x86: move generically usable NUMA code
> from x86 to common
> 
> On 08.09.2022 12:32, Wei Chen wrote:
> > On 2022/9/8 17:09, Jan Beulich wrote:
> >> On 02.09.2022 05:31, Wei Chen wrote:
> >>> --- /dev/null
> >>> +++ b/xen/common/numa.c
> >>> @@ -0,0 +1,442 @@
> >>> +/*
> >>> + * Generic VM initialization for NUMA setups.
> >>> + * Copyright 2002,2003 Andi Kleen, SuSE Labs.
> >>> + * Adapted for Xen: Ryan Harper <ryanh@xxxxxxxxxx>
> >>> + */
> >>> +
> >>> +#include <xen/init.h>
> >>> +#include <xen/keyhandler.h>
> >>> +#include <xen/mm.h>
> >>> +#include <xen/nodemask.h>
> >>> +#include <xen/numa.h>
> >>> +#include <xen/param.h>
> >>> +#include <xen/sched.h>
> >>> +#include <xen/softirq.h>
> >>> +
> >>> +struct node_data __ro_after_init node_data[MAX_NUMNODES];
> >>> +
> >>> +/* Mapping from pdx to node id */
> >>> +unsigned int __ro_after_init memnode_shift;
> >>> +unsigned long __ro_after_init memnodemapsize;
> >>> +uint8_t *__ro_after_init memnodemap;
> >>> +static uint8_t __ro_after_init _memnodemap[64];
> >>
> >> These last two want to use nodeid_t instead of uint8_t. Originally
> >> the latter used typeof(*memnodemap) for (I think - iirc it was me who
> >> made it that way) this reason: That way correcting memnodemap's type
> >> would have propagated without the need for further adjustments.
> >>
> >
> > Thanks for this info, should I need to restore it to use
> > "typeof(*memnodemap)" in next version ?
> 
> That would be more in line with the original code, but it's not
> strictly necessary once nodeid_t if properly used for these variables.
> I'd leave it up to you as long as you switch to nodeid_t.
> 

Ok, I will think more about it in next version.

> >>> +nodeid_t __read_mostly cpu_to_node[NR_CPUS] = {
> >>> +    [0 ... NR_CPUS-1] = NUMA_NO_NODE
> >>> +};
> >>> +
> >>> +cpumask_t __read_mostly node_to_cpumask[MAX_NUMNODES];
> >>> +
> >>> +nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
> >>> +
> >>> +bool __read_mostly numa_off;
> >>
> >> The v3 review discussing this possibly becoming __ro_after_init didn't
> >> really finish (you didn't reply to my latest request there), but you
> >> also didn't change the attribute. Please clarify.
> >>
> >
> > I think I had answered your question by:
> >  >> I think yes, it will be used in numa_disabled and numa_disabled will
> >  >> be called in cpu_add."
> >
> > And you replied me with:
> >  > In the original code I cannot spot such a path - can you please point
> >  > out how exactly you see numa_disabled() reachable from cpu_add()? I'm
> >  > clearly overlooking something ..."
> >
> > But there is a time difference here, your reply was sent after I sent
> > v3, maybe you didn't notice it
> 
> Which suggests you might better have waited with sending v3 until the
> discussion had settled.
> 
> > About the new question:
> > cpu_add will call srat_disabled, srat_disabled will access numa_off.
> > srat_disabled is a function without __init.
> 
> But the request wasn't to make the variable __initdata. That would be
> wrong of course. Since srat_disabled() only reads numa_off,
> __ro_after_init does look usable to me.
> 

Oh, yes, you're right. I had thought wrong. I will correct this.

Cheers,
Wei Chen.

> Jan

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.