[PATCH 0/3] elf: allow dl_iterate_phdr() to be called concurrently

[Caleb Sander Mateos]

dl_iterate_phdr() internally acquires the dl_load_write_lock mutex,
which means it will block if called concurrently on a second thread.
This mutual exclusion is surprising for a function that's read-only in
spirit. And it can cause deadlocks in applications calling it from
multiple places if the user-supplied callback acquires a nested mutex.
For example, an application compiled with ThreadSanitizer that uses
dl_iterate_phdr() to build a symbol table can deadlock if TSAN detects a
race condition concurrently. This is because any memory access inside
the dl_iterate_phdr() callback may be intercepted by TSAN and block on a
TSAN-internal semaphore. At the same time, TSAN's race reporting may
already hold the semaphore and call into dl_iterate_phdr() to symbolize
the stacktrace, blocking on dl_load_write_lock.

Add support for (recursive) reader-writer locks in rtld, using
pthread_rwlock internally. Change dl_load_write_lock from a mutex to a
reader-writer lock to allow concurrent calls to dl_iterate_phdr().
_dl_add_to_namespace_list() and _dl_close_worker() acquire a write lock
as they modify the loaded object list, while __dl_iterate_phdr()
acquires a read lock as it only reads from the list.

Example backtraces from threads in the dl_iterate_phdr-TSAN deadlock:
0  0x00000000043dacee in __sanitizer::FutexWait(__sanitizer::atomic_uint32_t*, unsigned int) ()
1  0x00000000043dc23a in __sanitizer::Semaphore::Wait() ()
2  0x000000000446cf90 in __tsan::TraceSwitchPartImpl(__tsan::ThreadState*) ()
3  0x000000000446eadd in void __tsan::RestartMemoryAccessRange<true>(__tsan::ThreadState*, unsigned long, unsigned long, unsigned long) ()
4  0x000000000440d7ab in __interceptor_strcmp ()
5  0x00000000068bb33f in elf_add (state=0x7f8270031000, filename=filename@entry=0x7f8270777de0 "/lib64/libresolv.so.2", descriptor=42, memory=memory@entry=0x0, memory_size=memory_size@entry=0, base_address=140198202634240, error_callback=0x4c3abe2 <backtrace_error_cb(void*, char const*, int)>, data=0x7f822a965e60, fileline_fn=0x7f822a965b48, found_sym=0x7f822a965d34, found_dwarf=0x7f822a965b54, fileline_entry=0x0, exe=0, debuginfo=0, with_buildid_data=0x0, with_buildid_size=0, so_filename=0x7f8270777de0 "/lib64/libresolv.so.2") at ./tpc/backtrace/elf.c:4264
6  0x00000000068bd405 in phdr_callback (info=0x7f822a965be0, size=<optimized out>, pdata=0x7f822a965cd0) at ./tpc/backtrace/elf.c:4896
7  0x000000000440cbe2 in dl_iterate_phdr_cb(__sanitizer::__sanitizer_dl_phdr_info*, unsigned long, void*) ()
8  0x00007f826fb56874 in __GI___dl_iterate_phdr (callback=0x440caf0 <dl_iterate_phdr_cb(__sanitizer::__sanitizer_dl_phdr_info*, unsigned long, void*)>, data=0x7f822a965c88) at dl-iteratephdr.c:75
9  0x000000000440ca52 in __interceptor_dl_iterate_phdr ()
10 0x00000000068ba9d2 in backtrace_initialize (state=state@entry=0x7f8270031000, filename=filename@entry=0x30c2219 <.L.str.189> "/proc/self/exe", descriptor=<optimized out>, error_callback=error_callback@entry=0x4c3abe2 <backtrace_error_cb(void*, char const*, int)>, data=data@entry=0x7f822a965e60, fileline_fn=fileline_fn@entry=0x7f822a965db8) at ./tpc/backtrace/elf.c:4940

0  futex_wait (private=0, expected=2, futex_word=0x7f827093a9b0 <_rtld_local+2480>) at ../sysdeps/nptl/futex-internal.h:146
1  __GI___lll_lock_wait (futex=futex@entry=0x7f827093a9b0 <_rtld_local+2480>, private=0) at lowlevellock.c:50
2  0x00007f826fa8d5ad in lll_mutex_lock_optimized (mutex=0x7f827093a9b0 <_rtld_local+2480>) at pthread_mutex_lock.c:49
3  ___pthread_mutex_lock (mutex=0x7f827093a9b0 <_rtld_local+2480>) at pthread_mutex_lock.c:129
4  0x00007f826fb5671c in __GI___dl_iterate_phdr (callback=0x43e6ae0 <__sanitizer::dl_iterate_phdr_cb(dl_phdr_info*, unsigned long, void*)>, data=0x7f822a8c34f0) at dl-iteratephdr.c:40
5  0x000000000440c9cd in __interceptor_dl_iterate_phdr ()
6  0x00000000043e6acd in __sanitizer::ListOfModules::init() ()
7  0x00000000043f0645 in __sanitizer::Symbolizer::FindModuleForAddress(unsigned long) ()
8  0x00000000043f083b in __sanitizer::Symbolizer::SymbolizeData(unsigned long, __sanitizer::DataInfo*) ()
9  0x0000000004482486 in __tsan::SymbolizeData(unsigned long) ()
10 0x000000000447d457 in __tsan::ScopedReportBase::AddLocation(unsigned long, unsigned long) ()
11 0x000000000447f82f in __tsan::ReportRace(__tsan::ThreadState*, __tsan::RawShadow*, __tsan::Shadow, __tsan::Shadow, unsigned long) ()

Caleb Sander Mateos (3):
  mach: remove unused rtld_lock macros
  sysdeps: add rtld_rwlock_recursive
  elf: allow dl_iterate_phdr() to be called concurrently

 elf/dl-close.c                |  4 ++--
 elf/dl-iteratephdr.c          |  6 +++---
 elf/dl-object.c               |  4 ++--
 elf/dl-support.c              |  2 +-
 elf/rtld.c                    |  2 +-
 sysdeps/generic/ldsodefs.h    |  5 ++++-
 sysdeps/generic/libc-lock.h   |  5 +++++
 sysdeps/mach/libc-lock.h      | 10 +++++++---
 sysdeps/nptl/dl-mutex.c       | 24 ++++++++++++++++++++++++
 sysdeps/nptl/dl-tls_init_tp.c |  9 +++++++++
 sysdeps/nptl/libc-lockP.h     | 26 ++++++++++++++++++++++++++
 11 files changed, 84 insertions(+), 13 deletions(-)

-- 
2.43.0