Re: -mno-tls-direct-seg-refs support in glibc for i386 PV Xen

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 27/05/2020 14:03, Florian Weimer wrote:
> I'm about to remove nosegneg support from upstream glibc, special builds
> that use -mno-tls-direct-seg-refs, and the ability load different
> libraries built in this mode automatically, when the Linux kernel tells
> us to do that.  I think the intended effect is that these special builds
> do not use operands of the form %gs:(%eax) when %eax has the MSB set
> because that had a performance hit with paravirtualization on 32-bit
> x86.  Instead, the thread pointer is first loaded from %gs:0, and the
> actual access does not use a segment prefix.
>
> Before doing that, I'd like to ask if anybody is still using this
> feature?
>
> I know that we've been carrying nosegneg libraries for many years, in
> some cases even after we stopped shipping 32-bit kernels. 8-/ The
> feature has always been rather poorly documented, and the way the
> dynamic loader selects those nosegneg library variants is still very
> bizarre.

I wasn't even aware of this feature, or that there was a problem wanting
fixing.

That said, I have found:

# 32-bit x86 does not perform well with -ve segment accesses on Xen.
CFLAGS-$(CONFIG_X86_32) += $(call cc-option,$(CC),-mno-tls-direct-seg-refs)

in one of our makefiles.

Why does the MSB make any difference?  %gs still needs to remain intact
so the thread pointer can be pulled out, so there is nothing that Xen or
Linux can do in the way of lazy loading.

Beyond that, its straight up segment base semantics in x86.  There will
be a 1-cycle AGU delay from a non-zero base, but that nothing to do with
Xen and applies to all segment based TLS accesses on x86, and you'll win
that back easily through reduced register pressure.

Are there any further details on the perf problem claim?  I find it
suspicious.


Either way, 32bit PV is on its last legs (not too bad, for something
which was essentially killed by the AMD64 spec).

Ring 1 counting as supervisor mode as far as pagetables goes has already
caused guests to suffer a major performance hit on hardware with
SMAP/SMEP (IvyBridge and later), as well as various speculative
mitigations (we can't rely on SMEP preventing the CPU from speculating
back into Ring 1, etc), and the forthcoming CET Shadow Stack feature
totally kills Ring1/2 as usable concepts in the architecture.

Linux is threatening to drop PV32 support, and I've recently added an
option to Xen to compile out and/or disable PV32 (both for attack
surface reduction purposes, and as a necessary consequence of using
Shadow Stacks).

With both my XenServer and upstream x86 maintainers hats on, PV32 is
solely for legacy workloads now.  People currently using PV32 obviously
don't care about performance, or haven't been taking security updates. 
I severely doubt they'll notice any change from this.

~Andrew