Re: [PATCH] cpu/hotplug: Allow the CPU in CPU_UP_PREPARE state to be brought up again.

[Dongli Zhang <dongli.zhang@xxxxxxxxxx>]

Tested-by: Dongli Zhang <dongli.zhang@xxxxxxxxxx>


The bug fixed by commit 53fafdbb8b21 ("KVM: x86: switch KVMCLOCK base to
monotonic raw clock") may leave the cpu_hotplug_state at CPU_UP_PREPARE. As a
result, to online this CPU again (even after removal) is always failed.

I have tested that this patch works well to workaround the issue, by introducing
either a mdeley(11000) or while(1); to start_secondary(). That is, to online the
same CPU again is successful even after initial do_boot_cpu() failure.

1. add mdelay(11000) or while(1); to the start_secondary().

2. to online CPU is failed at do_boot_cpu().

3. to online CPU again is failed without this patch.

# echo 1 > /sys/devices/system/cpu/cpu4/online
-su: echo: write error: Input/output error

4. to online CPU again is successful with this patch.

Thank you very much!

Dongli Zhang

On 11/22/21 7:47 AM, Sebastian Andrzej Siewior wrote:
> From: "Longpeng(Mike)" <longpeng2@xxxxxxxxxx>
> 
> A CPU will not show up in virtualized environment which includes an
> Enclave. The VM splits its resources into a primary VM and a Enclave
> VM. While the Enclave is active, the hypervisor will ignore all requests
> to bring up a CPU and this CPU will remain in CPU_UP_PREPARE state.
> The kernel will wait up to ten seconds for CPU to show up
> (do_boot_cpu()) and then rollback the hotplug state back to
> CPUHP_OFFLINE leaving the CPU state in CPU_UP_PREPARE. The CPU state is
> set back to CPUHP_TEARDOWN_CPU during the CPU_POST_DEAD stage.
> 
> After the Enclave VM terminates, the primary VM can bring up the CPU
> again.
> 
> Allow to bring up the CPU if it is in the CPU_UP_PREPARE state.
> 
> [bigeasy: Rewrite commit description.]
> 
> Signed-off-by: Longpeng(Mike) <longpeng2@xxxxxxxxxx>
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>
> Link: 
> https://urldefense.com/v3/__https://lore.kernel.org/r/20210901051143.2752-1-longpeng2@huawei.com__;!!ACWV5N9M2RV99hQ!d4sCCXMQV7ekFwpd21vo1_9K-m5h4VZ-gE8Z62PLL58DT4VJ6StH57TR_KpBdbwhBE0$
>  
> ---
> 
> For XEN: this changes the behaviour as it allows to invoke
> cpu_initialize_context() again should it have have earlier. I *think*
> this is okay and would to bring up the CPU again should the memory
> allocation in cpu_initialize_context() fail.
> 
>  kernel/smpboot.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/kernel/smpboot.c b/kernel/smpboot.c
> index f6bc0bc8a2aab..34958d7fe2c1c 100644
> --- a/kernel/smpboot.c
> +++ b/kernel/smpboot.c
> @@ -392,6 +392,13 @@ int cpu_check_up_prepare(int cpu)
>                */
>               return -EAGAIN;
>  
> +     case CPU_UP_PREPARE:
> +             /*
> +              * Timeout while waiting for the CPU to show up. Allow to try
> +              * again later.
> +              */
> +             return 0;
> +
>       default:
>  
>               /* Should not happen.  Famous last words. */
>