Re: [PATCH] Async signal safe TLS accesses

[OndÅej BÃlka <neleai at seznam dot cz>]

On Fri, Dec 06, 2013 at 02:50:21PM -0800, Andrew Hunter wrote:
> I'm looking into fixing TLDDESC.  Is this the best available documentation:
> 
> http://www.fsfla.org/~lxoliva/writeups/TLS/RFC-TLSDESC-x86.txt
> 
> ?
> 
> I note with some interest that using mask_all_signals whenever we take
> dl_load_lock would (very nearly, except for STB_GNU_UNIQUE symbols)
> solve the problem outright.  I welcome comment from others as to
> whether this carries acceptable overhead.
> 
> If not, I see two obvious possibilities
> 
> - deprecate lazy relocations; I'm unclear if the gains from not doing
> a single dl_lookup_symbol_x per reloc at dlopen time are actually
> significant (though maybe I'm missing the point of lazy relocations?)
> AFAICT, the current scheme is more than capable of supporting the
> TLSDESC indirections that actually use __tls_get_addr and friends.  I
> favor this scheme unless someone can tell me a good reason not to do
> so.
>
I asked several times about if these actually do save anything. See
following bug:

https://sourceware.org/bugzilla/show_bug.cgi?id=11787

For laziness to pay itself you need lot of big allocations that happen
infrequently. As said in bug report most libraries just use small amount
of tls so lazy initialization overhead is bigger than doing
initialization eagerly. Combination of big and infrequent tls segment
does not make much sense, as user often would get better result by
calculating what is actually needed.


If we do not improve tls somebody else will. For dynamic tls adding a
function call is useless unless you access variable from different dso.

It could be also done as userspace library with clunky
syntax.

Idea is that each dso gets assigned static number that determines its
index in userspace dtv. These are assigned on dlopen.

A dtv is thread-local pointer to array of actual tls data.
When it needs to be resized we create for each thread its copy and keep
old array.

Here I need to solve technical problem to convince compiler to emit
static access for this variable in shared objects.

Now tls access just becomes a simple lookup in dtv. There are three ways
how allocate memory
1. at dlopen/pthread_create. When tls segment is less than 64 bytes then
this looks like fastest way.
2. Adding a null check after lookup and calling allocating routine when
check fails. This has nonzero runtime overhead.
3. With gcc support we could just emit ifunc for each routine that
access tls which causes allocator to be called, avoiding runtime
overhead of 2.


 

extern pthread_mutex_t mutex;
extern __thread void **dtv;
static size_t tls_idx;
struct thread
{
  void **dtv;
  struct thread *next;
}

static void
__attribute__ ((constructor)) init_tls()
{
  /* disable signals */
  pthread_mutex_lock (mutex);
  tls_idx = get_dtv_idx ();
  if (need_realloc ())
    for (t = first_thread; t; t=t->next)
      {
        m = malloc (2 * size);
        memcpy (m, t->dtv, size);
        memset (m + size, 0, size);
        t->dtv = m;
      }
  pthread_mutex_unlock (mutex);
}

void
on_pthread_create ()
{
  pthread_mutex_lock (mutex);
  struct thread t = malloc (sizeof (struct thread));
  t->next = first_thread;
  dtv = t->dtv = malloc (get_dtv_size);
  first_thread = t;
  pthread_mutex_unlock (mutex);
}

static void
__attribute__((destructor)) fini_tls()
{
  pthread_mutex_lock (mutex);
  free_dtv_idx (tls_idx);
  pthread_mutex_unlock (mutex);
}

struct tls {
  int foo;
  int bar[20];
}

#define TLS3(var) ((struct tls *) dtv[tls_idx])->var

#define LAZY_TLS3(var) ({ \
  if (__glibc_unlikely (dtv[tls_idx] == NULL)) \
    dtv[tls_idx] = malloc (sizeof (struct tls)); \
  TLS3(var); \
})