Re: [PATCH] x86/spec-ctrl: Skip RSB overwriting when safe to do so

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 24/08/2021 14:04, Jan Beulich wrote:
> On 19.08.2021 18:26, Andrew Cooper wrote:
>> In some configurations, it is safe to not overwrite the RSB on entry to Xen.
>> Both Intel and AMD have guidelines in this area, because of the performance
>> difference it makes for native kernels.
> I don't think I've come across AMD's guidelines - would you happen to
> have a pointer?

APM Vol2 3.2.9 "Speculation Control MSRs"

The information about SMEP is in the final paragraph before describing
MSR_SPEC_CTRL.STIBP.

>  Nevertheless ...
>
>> A simple microperf test, measuring the amount of time a XENVER_version
>> hypercall takes, shows the following improvements:
>>
>>   KabyLake:     -13.9175% +/- 6.85387%
>>   CoffeeLake-R:  -9.1183% +/- 5.04519%
>>   Milan:        -17.7803% +/- 1.29808%
>>
>> This is best case improvement, because no real workloads are making
>> XENVER_version hypercalls in a tight loop.  However, this is the hypercall
>> used by PV kernels to force evtchn delivery if one is pending, so it is a
>> common hypercall to see, especially in dom0.
>>
>> The avoidance of RSB-overwriting speeds up all interrupts, exceptions and
>> system calls from PV or Xen context.  RSB-overwriting is still required on
>> VMExit from HVM guests for now.
>>
>> In terms of more realistic testing, LMBench in dom0 on an AMD Rome system
>> shows improvements across the board, with the best improvement at 8% for
>> simple syscall and simple write.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
> Reviewed-by: Jan Beulich <jbeulich@xxxxxxxx>

Thanks.

> albeit with one further remark / request:
>
>> @@ -992,18 +1021,36 @@ void __init init_speculation_mitigations(void)
>>          default_xen_spec_ctrl |= SPEC_CTRL_SSBD;
>>  
>>      /*
>> -     * PV guests can poison the RSB to any virtual address from which
>> -     * they can execute a call instruction.  This is necessarily outside
>> -     * of the Xen supervisor mappings.
>> +     * PV guests can create RSB entries for any linear address they control,
>> +     * which are outside of Xen's mappings.
>> +     *
>> +     * SMEP inhibits speculation to any user mappings, so in principle it is
>> +     * safe to not overwrite the RSB when SMEP is active.
>> +     *
>> +     * However, some caveats apply:
>> +     *
>> +     * 1) CALL instructions push the next sequential linear address into the
>> +     *    RSB, meaning that there is a boundary case at the user=>supervisor
>> +     *    split.  This can be compensated for by having an unmapped or NX
>> +     *    page, or an instruction which halts speculation.
>>       *
>> -     * With SMEP enabled, the processor won't speculate into user mappings.
>> -     * Therefore, in this case, we don't need to worry about poisoned 
>> entries
>> -     * from 64bit PV guests.
>> +     *    For Xen, the next sequential linear address is the start of M2P
>> +     *    (mapped NX), or a zapped hole (unmapped).
> IIUC you mean the compat M2P here - perhaps worth being explicit? I'm
> also not sure why you use "zapped": Nothing can ever be mapped into
> the non-canonical hole.

The non-canonical hole doesn't behave as "a unmapped gap" when the
microarchitecture uses 48 bit pointers internally.

~Andrew