This is the mail archive of the mailing list for the glibc project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Proposal: Slow dynamic TLS for internal glibc use

Could anyone be relying on the de facto shared state across threads
with the non-reentrant versions? While strtok, say, isn't thread-safe,
it can be used safely with external synchronization. It would be a
little odd, but you could imagine the following being in use:

static char *str;

void *thread_a(void *) {
  char *first_line = strtok(str, "\n");
  // ...

void *thread_b(void *) {
  char *second_line = strtok(NULL, "\n");
  // ...

int main(void) {
  str = read_file();

If  I understand correctly the strtok implementation you are
suggesting (see below), thread_a and thread_b would not be sharing
state anymore, and thread_b would (hopefully) segfault (calling
strtok(NULL, "\n") without a prior call with a non-null first

char *strtok(char *str, const char *delim) {
  static __thread char *state;
  return strtok_r(str, delim, &state);

Is the argument that using strtok() across threads is undefined
behavior? From the 3p manpage, it doesn't quite read like that to me:
"The strtok() function need not be thread-safe.", allowing for
something like this:

char *strtok(char *str, const char *delim) {
  static char *state;
  static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
  char *ret;

  ret = strtok_r(str, delim, &state);
  return ret;

Which would behave the same as a naive strtok() for thread_a and thread_b above.

The same observation would apply to other functions in this family
where the callers were relying on the shared state and otherwise
enforced a single caller at a time.
On Tue, Oct 23, 2018 at 5:19 AM Florian Weimer <> wrote:
> We have a lot of legacy interfaces which are not thread-safe: strtok,
> many, many NSS functions such as gethostbyname, wide character functions
> when invoked with a NULL state parameter, strerror, getpass, rpmatch,
> localtime and so on.
> Currently, most of these functions use global variables which take up
> space in .bss.  This introduces overhead even if they are never used,
> and these functions are of course not thread-safe.
> I would like to suggest that we add very slow TLS (slower than
> global-dynamic) to support these functions in a thread-safe manner.
> Memory usage will be larger if these interfaces are actually used, but
> if they are not, there is no overhead whatsoever (beyond a slight
> increase in code size), lower than what we have today.
> The interface I have in mind looks like this.  The keys should be
> addresses of global glibc functions.
> - void *__libc_slow_tls_get (void *key);
> Return an existing TLS allocation for KEY if one exists, or NULL
> otherwise.
> - void *__libc_slow_tls_get_or_allocate (void *key, size_t size);
> If a TLS allocation for KEY exists, return it, otherwise create a new
> allocation on the heap for SIZE bytes, and return the pointer for that.
> Return NULL on allocation error, with errno set appropriately.
> - void *__libc_slow_tls_reallocate (void *key, size_t new_size);
> Create a new TLS allocation for KEY of NEW_SIZE if none exists yet, or
> resize an existing allocation.  Return a pointer to the allocation, or
> NULL on allocation error, with errno set appropriately.
> These allocations are automatically freed if a thread exits.  A single
> per-thread pointer is used to point to the root of the data structure.
> The rest of the memory will be heap-allocated.
> I'm working to get clarification for C2X that this is a valid
> implementation for the functions where C11 wasn't updated accordingly
> when objects of thread storage duration were introduced.
> Using these new functions, c32rtomb would look like this:
> size_t
> c32rtomb (char *s, char32_t c32, mbstate_t *ps)
> {
>   if (ps == NULL)
>     {
>       ps = __libc_slow_tls_get_or_allocate (c32rtomb, sizeof (*ps));
>       if (ps == NULL)
>         return;
>     }
>   return wcrtomb (s, c32, ps);
> }
> And “static mbstate_t state;” would be gone.
> In contrast, for strtok, we should simply use regular TLS state because
> this function is actually used in inner loops, and there is no good way
> to report errors.
> Comments?
> Thanks,
> Florian

Eric Rannaud <>
Nanocritical, CEO

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]