[Xen-devel] A better fix for the /lib/tls problems

[Keir Fraser <Keir.Fraser@xxxxxxxxxxxx>]

Further to the email below from a few weeks ago, I can announce that
Xen/Linux 2.6 handles the TLS problem (-ve accesses into segments) by
dynamically rewriting the offending instructions.

This is enabled by default in the 2.6 kernels, and reduces the
overhead of handling those instructions to almost zero (compared with
around 100% overhead when using pure emulation in Xen, when running
library-intensive programs).

Although the rewriting works well for me, it may not be bulletproof
for a number of reasons:
 1. We assume that the base address of GS is stored at GS:0x0, as
    dictated by the NPTL standard. This is probably a safe assumption
    therefore. 
 2. We assume that the program will not read or write its own
    instructions, and get confused by the rewrite. Once again,
    probably safe. 
 3. The patch occasionally must touch two or three adjacent
    instructions. We currently assume that these do not straddle a
    basic-block boundary (i.e., noone will JMP into the middle of
    them). I'm less happy about this, but in practise it seems we get
    away with it. :-)

If you find some programs fail oddly (e.g., segfault) then try booting
your Linux 2.6 kernel with the 'nosegfixup' command-line option. This
will disable dynamic rewriting and fall back to pure emulation.
If this fixes your problem then let me know about it!

 -- Keir

> More info on the TLS problems. I've added limited instruction
> emulation to Xen so that we can use the TLS libraries that come with
> e.g., Redhat 9 and Debian. Basically we take teh general protection
> fault, then decode the instruction and interpret it if it's trying to
> access a -ve segment offset.
> 
> Unfortunately the emulation triggers _a lot_! So it will affect
> performance...
> 
> However, I intend to add dynamic patching of the unexecutable
> instructions to Linux -- it will detect instructions of a particular
> limited class, and dynamically create fixup code, and patch the
> original execution site to call the fixup code.
> 
> I originally thought this would be very hard, but I think I have a way
> of doing this in a reasonably simple manner (fingers crossed). This
> should get almost every drop of performance back.
> 
>  -- Keir
> 
> > I'm not an expert or anything like that but I investigated this same
> > problem because UML (user mode linux) doesn't like (read: support) tls
> > too - either 2.4 and 2.6 umls.
> > 
> > "various different tls implementations" -> No such thing. NPTL/TLS
> > support is stable and much the same in every linux distro. So if you
> > have problems with RH you'll have problems with debian unstable, i686
> > and a 2.6 kernel (read below for explanation), etc...
> > 
> > /lib/ld.so checks `uname` in 2.4 kernels. If extraversion begins with
> > "-ntpl" than this 2.4 kernel has nptl support. If extraversion doesn't
> > have that string it's assumed that the kernel doesn't have nptl support
> > (and tls support).
> > 
> > If the kernel is 2.6+ then ld.so assumes that nptl support is present.
> > The way to remove it is to "mv /lib/tls /lib/tls.off". Other way to deal
> > with it is to use the environment var LD_ASSUME_KERNEL and set it to 2.4
> > globally, wich may cause havoc :)
> > 
> > I think that nptl requires at least i486, so if you install for i386,
> > nptl support won't be included in the installation. Another way is to
> > downgrade the VMM from i686 (or whatever processor you have) to plain
> > i386. I don't know if it's possible in xen...
> > 
> > In my uml setups i choose the KISS method: mv /lib/tls /lib/tls.off
> > 
> > With debian you can instruct dpkg (man dpkg-divert) to relocate
> > everything that would be installed to /lib/tls to another location
> > automagically. This saves troubles when apt-get upgrad'ing ;)
> > 
> > Hope this helps,
> > Nuno Silva
> > 
> > P.S. Great work! Keep going!! :)
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.2.4 (GNU/Linux)
> > Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
> > 
> > iD8DBQFA+KHUOPig54MP17wRAmQ6AKDJt+S34QrrzXN0Bm4hFlBkNpW4dgCgw55G
> > EFOTQDye2YnPA/foE9R0OUs=
> > =L05g
> > -----END PGP SIGNATURE-----
> > 
> > 
> > -------------------------------------------------------
> > This SF.Net email is sponsored by BEA Weblogic Workshop
> > FREE Java Enterprise J2EE developer tools!
> > Get your free copy of BEA WebLogic Workshop 8.1 today.
> > http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxxxx
> > https://lists.sourceforge.net/lists/listinfo/xen-devel
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by BEA Weblogic Workshop
> FREE Java Enterprise J2EE developer tools!
> Get your free copy of BEA WebLogic Workshop 8.1 today.
> http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxxxx
> https://lists.sourceforge.net/lists/listinfo/xen-devel



-------------------------------------------------------
This SF.Net email is sponsored by OSTG. Have you noticed the changes on
Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now,
one more big change to announce. We are now OSTG- Open Source Technology
Group. Come see the changes on the new OSTG site. www.ostg.com
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/xen-devel