Re: xentrace buffer size, maxcpus and online cpus

[Jan Beulich <jbeulich@xxxxxxxx>]

On 30.05.2023 22:06, Olaf Hering wrote:
> Tue, 30 May 2023 10:41:07 +0200 Jan Beulich <jbeulich@xxxxxxxx>:
> 
>> Using this N would be correct afaict, but that N isn't num_online_cpus().
>> CPUs may have been offlined by the time trace buffers are initialized, so
>> without looking too closely I think it would be num_present_cpus() that
>> you're after.
> 
> In my testing num_online_cpus returns N, while num_present_cpus returns
> all available pcpus. There is also num_possible_cpus, but this appears to
> be an ARM thing.
> 
> If Xen is booted with maxcpus=, is there a way to use the remaining cpus?

In general no, because then nr_cpu_ids will be too constrained. But
note that CPU parking also comes into play here, leading to nr_cpu_ids
being set to all possible (present + hotplug) CPUs. Iirc parked CPUs
can be brought online even beyond what "maxcpus=" says (albeit I think
that's more a side effect of the parking implementation than an
intended goal).

> In case this is possible, the code needs adjustment to reinitialize the
> trace buffers. This is not an easy change. But if the remaining cpus
> will remain offline, then something like this may work:
> 
> +++ b/xen/common/trace.c
> @@ -110,7 +110,8 @@ static int calculate_tbuf_size(unsigned int pages, 
> uint16_t t_info_first_offset)
>      struct t_info dummy_pages;
>      typeof(dummy_pages.tbuf_size) max_pages;
>      typeof(dummy_pages.mfn_offset[0]) max_mfn_offset;
> -    unsigned int max_cpus = nr_cpu_ids;
> +    unsigned int nr_cpus = num_online_cpus();
> +    unsigned int max_cpus = nr_cpus;

As said before, num_online_cpus() will under-report for the purpose
here, as CPUs may have been brought offline, and may be brought online
again later (independent of the use of "maxcpus=").

Re-initializing trace buffers when more CPUs come online might of
course be an option, but it would need doing in a race free manner.

Jan