Re: [patch v3 08/36] x86/smpboot: Split up native_cpu_up() into separate phases and document them

[Peter Zijlstra <peterz@xxxxxxxxxxxxx>]

And since I'm commenting on existing things anyway, let me continue...

On Mon, May 08, 2023 at 09:43:39PM +0200, Thomas Gleixner wrote:

> +static int wait_cpu_cpumask(unsigned int cpu, const struct cpumask *mask)
> +{
> +     unsigned long timeout;
>  
> +     /*
> +      * Wait up to 10s for the CPU to report in.
> +      */
> +     timeout = jiffies + 10*HZ;
> +     while (time_before(jiffies, timeout)) {
> +             if (cpumask_test_cpu(cpu, mask))
> +                     return 0;
> +
> +             schedule();
>       }
> +     return -1;
> +}

> +/*
> + * Bringup step three: Wait for the target AP to reach smp_callin().
> + * The AP is not waiting for us here so we don't need to parallelise
> + * this step. Not entirely clear why we care about this, since we just
> + * proceed directly to TSC synchronization which is the next sync
> + * point with the AP anyway.
> + */
> +static void wait_cpu_callin(unsigned int cpu)
> +{
> +     while (!cpumask_test_cpu(cpu, cpu_callin_mask))
> +             schedule();
> +}
> +
> +/*
> + * Bringup step four: Synchronize the TSC and wait for the target AP
> + * to reach set_cpu_online() in start_secondary().
> + */
> +static void wait_cpu_online(unsigned int cpu)
>  {
>       unsigned long flags;
> +
> +     /*
> +      * Check TSC synchronization with the AP (keep irqs disabled
> +      * while doing so):
> +      */
> +     local_irq_save(flags);
> +     check_tsc_sync_source(cpu);
> +     local_irq_restore(flags);
> +
> +     /*
> +      * Wait for the AP to mark itself online, so the core caller
> +      * can drop sparse_irq_lock.
> +      */
> +     while (!cpu_online(cpu))
> +             schedule();
> +}

These schedule() loops make me itch... this is basically Ye Olde yield()
loop with all it's known 'benefits'.

Now, I don't think it's horribly broken, we're explicitly waiting on
another CPU and can't have priority inversions, but yuck!

It could all be somewhat cleaned up with wait_var_event{_timeout}() and
wake_up_var(), but I'm really not sure that's worth it. But at least it
requires a comment to justify.