Re: Async-signal-safe access to __thread variables from dlopen()ed libraries?

[Roland McGrath <roland at hack dot frob dot com>]

> When a library is dlopen()ed, it would be reasonable for it to force such
> access from its initializer.
> 
> Unfortunately, that would require iterating over all pre-existing threads
> and calling some routine in each thread, which is near impossible (as far
> as I can tell). If in fact there is an existing mechanism to do this,
> I'll be happy to use it.

There is no such generic mechanism.  In any given program that's
concerned, there may well be some specific method you can use.

> > Or else, the DSO can use __attribute__ ((tls_model ("initial-exec"))) for
> > the variable it wants to use in a signal handler.
> 
> That doesn't generally work: as far as I can tell, a single "initial-exec"
> TLS variable in a DSO makes *all* TLS variables in the DSO use that model.

I was afraid that might be the case, but didn't bother to think it
through.  (To be pedantic, I don't think it makes code generation use IE
model for all __thread variables.  But it is the case that the
granularity of allocating the storage is the whole PT_TLS segment of the
module.)

> If there are a couple, very quickly you end up with a DSO that fails to
> dlopen with "cannot load any more object with static TLS" due to DTV_SURPLUS
> exhaustion.

Understood.  So it hinges on how many modules there are in practice that
really need to use __thread variables in signal handlers, how much total
TLS space such a module uses, and how many such modules are in practice
all loaded into the same process.

> > Only if those are really not practical in a real-world example should we
> > consider adding bizarre new features for this corner case.
> 
> That's one question I have: how much of a corner case is this really?
> 
> At Google we are hitting that corner case with all of our Java programs. I'd
> be somewhat surprised if it turns out that nobody else needs this feature.

There are many Java programs, sure, but in terms of a libc feature, a
JVM and some dlopen'd JNI modules really do seem like something of a
corner case.

I'm not entirely averse to finding a way to address this, of course.
But it's going to be extremely fiddly no matter what and the only
proposal made so far involves adding to the dynamic linker's ABI, so we
need to be extremely circumspect.  I am especially dubious about any
supposedly general-purpose solutions that are still only half-measures.
Ian's proposal is only adequate if the signal handler's use of a
__thread variable can cope with that variable being unavailable; i.e. it
only reads it and can fall back to a default value, or if it wants to
write it, it can somehow cope with being unable to do so.

While POSIX does not require that pthread_getspecific be
async-signal-safe, in practice it is.  However, pthread_setspecific is
not async-signal-safe in the general case.  Whether it calls malloc or
not depends on how many pthread_key_t values are already in use and
whether the thread has already used the one in question.  So if you only
need to read a value in a signal handler, then using pthread_getspecific
instead of a __thread variable is an approach that works today, works at
least with all past versions of NPTL, and most likely works on most or
all other POSIX systems too.  (Or, if you need to write too, but can be
sure that your pthread_key_create was among the first 32 calls
made--actually the 32 lowest-numbered still in use now, which you can
actually just test with KEY<32--and you're not concerned with bogons
from interrupting other calls to pthread_setspecific with your key.)

Hmm, it turns out we already have a _dl_tls_get_addr_soft internally.
(I actually have no recollection of adding this, but the ChangeLog says
I did, and a mere six years ago!)  But it is not part of the public ABI.
If you wanted to be entirely un-kosher about it, you could call this
(it's a GLIBC_PRIVATE symbol).  This isn't an exact analogue to
__tls_get_addr; it takes a struct link_map * argument and returns the
address of the whole PT_TLS segment's incarnation in the calling thread,
or a null pointer if the module has none or the thread hasn't allocated
its yet.  So, if the module's initializer acquires its own struct
link_map * somehow, its signal handlers could then call this as a test
of whether __thread variable access is safe (i.e., won't have to
allocate).  It looks to me like it's thoroughly async-signal-safe, even
though it was never particularly intended to be.  It exists to implement
dlinfo (,RTLD_DI_TLS_DATA,).  dlinfo itself is not async-signal-safe,
but only because of the _dl_catch_error machinery (so in practice it's
safe if you're sure you're not interrupting some <dlfcn.h> function or
the libc-internal equivalents such as nsswitch and iconv module loading,
and "safe enough" just if none of those things throws an error from
inside the dynamic linker).


Thanks,
Roland