Re: [PATCH v1 2/3] xen/smp: Rewrite on_selected_cpus() to be lockless

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Wed, Apr 01, 2026 at 05:35:20PM +0100, Ross Lagerwall wrote:
> on_selected_cpus() holds a global lock even if the function is to be
> called on non-overlapping CPUs. This is a scalability bottleneck so to
> avoid that:
> 
> 1. Remove the global lock.
> 2. Make call_data_struct per-CPU.
> 3. Track which CPUs are currently running on_selected_cpus() using a
>    global CPU mask. This tells CPUs running the interrupt which per-CPU
>    call_data_structs to look at.
> 
> Since the call data is now per-CPU, skip waiting for CPUs to "check in"
> for async calls. Instead, delay it until the next time
> on_selected_cpus() is called by which point there should be nothing to
> wait for.

One concern I have about this approach is loosing the fairness that
the usage of the spin lock provided.  Without the spinlock, and with
the usage of for_each_cpu ( i, &tasks ) to process any pending tasks,
it means the tasks from lower CPUs will always get preference over
the tasks from higher CPUs, regardless of when they have been
queued.

Ideally we need some kind of mechanism that can do a FIFO style
processing of the pending tasks, otherwise we are introducing a
performance penalty to high CPU IDs compared to low ones, and that's
likely to get worse as systems get bigger.

Thanks, Roger.