Re: [PATCH] x86/spec-ctrl: Skip RSB overwriting when safe to do so

[Jan Beulich <jbeulich@xxxxxxxx>]

On 19.08.2021 18:26, Andrew Cooper wrote:
> In some configurations, it is safe to not overwrite the RSB on entry to Xen.
> Both Intel and AMD have guidelines in this area, because of the performance
> difference it makes for native kernels.

I don't think I've come across AMD's guidelines - would you happen to
have a pointer? Nevertheless ...

> A simple microperf test, measuring the amount of time a XENVER_version
> hypercall takes, shows the following improvements:
> 
>   KabyLake:     -13.9175% +/- 6.85387%
>   CoffeeLake-R:  -9.1183% +/- 5.04519%
>   Milan:        -17.7803% +/- 1.29808%
> 
> This is best case improvement, because no real workloads are making
> XENVER_version hypercalls in a tight loop.  However, this is the hypercall
> used by PV kernels to force evtchn delivery if one is pending, so it is a
> common hypercall to see, especially in dom0.
> 
> The avoidance of RSB-overwriting speeds up all interrupts, exceptions and
> system calls from PV or Xen context.  RSB-overwriting is still required on
> VMExit from HVM guests for now.
> 
> In terms of more realistic testing, LMBench in dom0 on an AMD Rome system
> shows improvements across the board, with the best improvement at 8% for
> simple syscall and simple write.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>

Reviewed-by: Jan Beulich <jbeulich@xxxxxxxx>
albeit with one further remark / request:

> @@ -992,18 +1021,36 @@ void __init init_speculation_mitigations(void)
>          default_xen_spec_ctrl |= SPEC_CTRL_SSBD;
>  
>      /*
> -     * PV guests can poison the RSB to any virtual address from which
> -     * they can execute a call instruction.  This is necessarily outside
> -     * of the Xen supervisor mappings.
> +     * PV guests can create RSB entries for any linear address they control,
> +     * which are outside of Xen's mappings.
> +     *
> +     * SMEP inhibits speculation to any user mappings, so in principle it is
> +     * safe to not overwrite the RSB when SMEP is active.
> +     *
> +     * However, some caveats apply:
> +     *
> +     * 1) CALL instructions push the next sequential linear address into the
> +     *    RSB, meaning that there is a boundary case at the user=>supervisor
> +     *    split.  This can be compensated for by having an unmapped or NX
> +     *    page, or an instruction which halts speculation.
>       *
> -     * With SMEP enabled, the processor won't speculate into user mappings.
> -     * Therefore, in this case, we don't need to worry about poisoned entries
> -     * from 64bit PV guests.
> +     *    For Xen, the next sequential linear address is the start of M2P
> +     *    (mapped NX), or a zapped hole (unmapped).

IIUC you mean the compat M2P here - perhaps worth being explicit? I'm
also not sure why you use "zapped": Nothing can ever be mapped into
the non-canonical hole.

Jan