[RFC][PATCH v5 00/16] Proof-of-Concept implementation of RTLD_SHARED for dlmopen

[Vivek Das Mohapatra]

This is a revision of a previous patchset that I posted here
regarding https://sourceware.org/bugzilla/show_bug.cgi?id=22745 

I will recap the previous introduction here:

=======================================================================
  As discussed in the URL above dlmopen requires a mechanism for
  [optionally] sharing some objects between more than one namespace.

  The following patchset attempts an implementation for this: If an
  object is loaded with the new RTLD_SHARED flag we instead ensure
  that a "master" copy exists (and is flagged as no-delete) in the
  main namespace and a thin wrapper or clone is placed in the target
  namespace.

  This patch series should address all the comments received on the
  earlier (v1) series, and fixes a bug in the previous (v2) series
  which left the r_debug struct in an inconsistent state when creating
  a proxy triggered the initial load of a DSO into the main namespace.
=======================================================================

In addition this patch series now mplements the following:

 - dlmopen will implicitly apply RTLD_SHARED to the libc/libpthread group
   (requires a patched binutils/ld so that the libc family of DSOs can
   be flagged as requiring this behaviour)

 - LD_AUDIT paths will not apply this implict sharing rule, so audit libraries
   will continue to be completely isolated.

 - The mechanism for tagging DSOs as implicitly shared has been changed
   from a DT_FLAGS_1 flag to a DT_VALRNGHI/LO range dynamic tag.
   (Based on feedback on the binutils side of this patch series).

 - A flag RTLD_ISOLATE which is used inernally to suppress RTLD_SHARED
   behaviour when audit libraries are being loaded, and is also made available
   to users who really want a completely separate copy of glibc in their new
   namespace.

 - Tests for most of the new dlmopen behaviour (to be added to if
   this patch series is acceptable)
 
I have not yet implemented, but plan to address once this series is
accepted/acceptable:

 - Sensible RTLD_GLOBAL semantics for dlmopened DSOs in non-base namespaces

 - dl_iterate_ns_phdr (cf dl_iterate_phdr but taking a namespace argument)

Vivek Das Mohapatra (16):
  Declare and describe the dlmopen RTLD_SHARED flag
  include/link.h: Update the link_map struct to allow proxies
  elf/dl-object.c: Implement a helper function to proxy link_map entries
  elf/dl-load.c, elf-dl-open.c: Implement RTLD_SHARED dlmopen proxying
  elf/dl-fini.c: Handle proxy link_map entries in the shutdown path
  elf/dl-init.c: Skip proxied link map entries in the dl init path
  Define a new dynamic section tag - DT_GNU_UNIQUE
  Abstract the loaded-DSO search code into a private helper function
  Compare loaded DSOs by file ID and check for DF_1_UNIQUE
  Use the new DSO finder helper function since we have it
  Use the DSO search helper to check for preloaded DT_GNU_UNIQUE DSOs
  When loading DSOs into alternate namespaces check for DT_GNU_UNIQUE
  Suppress inter-namespace DSO sharing for audit libraries
  dlsym, dlvsym should be able to look up symbols via DSO proxies
  Add a DT_GNU_UNIQUE dynamic section to the glibc cluster DSOs
  Add dlmopen / RTLD_SHARED tests

 Makeconfig                       |   1 +
 Makerules                        |   2 +-
 bits/dlfcn.h                     |  10 +
 elf/Makefile                     |  42 +-
 elf/dl-close.c                   |  43 +-
 elf/dl-fini.c                    |   6 +-
 elf/dl-init.c                    |   4 +-
 elf/dl-load.c                    | 204 +++++++--
 elf/dl-object.c                  |  84 ++++
 elf/dl-open.c                    |  99 ++++-
 elf/dl-sym.c                     |  14 +
 elf/elf.h                        |   3 +-
 elf/rtld.c                       |   2 +-
 elf/tst-dlmopen-common.h         |  15 +
 elf/tst-dlmopen-main.h           | 899 +++++++++++++++++++++++++++++++++++++++
 elf/tst-dlmopen-modules.h        |   3 +
 elf/tst-dlmopen-rtld-shared1.c   |  74 ++++
 elf/tst-dlmopen-rtld-shared2.c   |  76 ++++
 elf/tst-dlmopen-rtld-shared3.c   |  53 +++
 elf/tst-dlmopen-rtld-shared4.c   |  24 ++
 elf/tst-dlmopen-rtld-shared5.c   |  35 ++
 elf/tst-dlmopen-rtld-shared6.c   |  46 ++
 elf/tst-dlmopen-rtld-unique1.c   |  96 +++++
 elf/tst-dlmopen-rtld-unique2.c   |  35 ++
 elf/tst-dlmopen-rtld-unique3.c   |  23 +
 elf/tst-dlmopen-rtld-unique4.c   |  24 ++
 elf/tst-dlmopen-sharedmod-norm.c |  11 +
 elf/tst-dlmopen-sharedmod-uniq.c |  11 +
 elf/tst-dlmopen-std-do-test.h    |  11 +
 iconvdata/Makefile               |   1 +
 include/link.h                   |   6 +-
 nptl/Makefile                    |   2 +-
 sysdeps/generic/ldsodefs.h       |   9 +
 sysdeps/mips/bits/dlfcn.h        |  10 +
 34 files changed, 1922 insertions(+), 56 deletions(-)
 create mode 100644 elf/tst-dlmopen-common.h
 create mode 100644 elf/tst-dlmopen-main.h
 create mode 100644 elf/tst-dlmopen-modules.h
 create mode 100644 elf/tst-dlmopen-rtld-shared1.c
 create mode 100644 elf/tst-dlmopen-rtld-shared2.c
 create mode 100644 elf/tst-dlmopen-rtld-shared3.c
 create mode 100644 elf/tst-dlmopen-rtld-shared4.c
 create mode 100644 elf/tst-dlmopen-rtld-shared5.c
 create mode 100644 elf/tst-dlmopen-rtld-shared6.c
 create mode 100644 elf/tst-dlmopen-rtld-unique1.c
 create mode 100644 elf/tst-dlmopen-rtld-unique2.c
 create mode 100644 elf/tst-dlmopen-rtld-unique3.c
 create mode 100644 elf/tst-dlmopen-rtld-unique4.c
 create mode 100644 elf/tst-dlmopen-sharedmod-norm.c
 create mode 100644 elf/tst-dlmopen-sharedmod-uniq.c
 create mode 100644 elf/tst-dlmopen-std-do-test.h

-- 
2.11.0