[PATCH v3] manual: Recommendations for dynamic linker hardening

[Florian Weimer]

This new section in the manual provides recommendations for
use of glibc in environments with higher integrity requirements.
It's reflecting both current implementation shortcomings, and
challenges we inherit from ELF and psABI requirements.

(snip)

This version incorporates a lot of feedback from Jason and Jonathan.  In
particular, I reworded things so that we now acknowledge the need for
symbol interposition when using C++.  I corrected the information about
__cxa_atexit error handling, and that STB_GNU_UNIQUE does not bypass
dlmopen boundaries.

Thanks,
Florian

---
 manual/dynlink.texi | 557 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 557 insertions(+)

diff --git a/manual/dynlink.texi b/manual/dynlink.texi
index d71f7a30d6..385166a36f 100644
--- a/manual/dynlink.texi
+++ b/manual/dynlink.texi
@@ -15,6 +15,7 @@ Dynamic linkers are sometimes called @dfn{dynamic loaders}.
 @menu
 * Dynamic Linker Invocation::   Explicit invocation of the dynamic linker.
 * Dynamic Linker Introspection::    Interfaces for querying mapping information.
+* Dynamic Linker Hardening::    Avoiding unexpected issues with dynamic linking.
 @end menu
 
 @node Dynamic Linker Invocation
@@ -535,6 +536,562 @@ information is processed.
 This function is a GNU extension.
 @end deftypefun
 
+@node Dynamic Linker Hardening
+@section Avoiding Unexpected Issues With Dynamic Linking
+
+This section details recommendations for increasing application
+robustness, by avoiding potential issues related to dynamic linking.
+The recommendations have two main aims: reduce the involvement of the
+dynamic linker in application execution after process startup, and
+restrict the application to a dynamic linker feature set whose behavior
+is more easily understood.
+
+Key aspects of limiting dynamic linker usage after startup are: no use
+of the @code{dlopen} function, disabling lazy binding, and using the
+static TLS model.  More easily understood dynamic linker behavior
+requires avoiding name conflicts (symbols and sonames) and highly
+customizable features like the audit subsystem.
+
+Note that while these steps can be considered a form of application
+hardening, they do not guard against potential harm from accidental or
+deliberate loading of untrusted or malicious code.  There is only
+limited overlap with traditional security hardening for applications
+running on GNU systems.
+
+@subsection Restricted Dynamic Linker Features
+
+Avoiding certain dynamic linker features can increase predictability of
+applications and reduce the risk of running into dynamic linker defects.
+
+@itemize @bullet
+@item
+Do not use the functions @code{dlopen}, @code{dlmopen}, or
+@code{dlclose}.  Dynamic loading and unloading of shared objects
+introduces substantial complications related to symbol and thread-local
+storage (TLS) management.
+
+@item
+Without the @code{dlopen} function, @code{dlsym} and @code{dlvsym}
+cannot be used with shared object handles.  Minimizing the use of both
+functions is recommended.  If they have to be used, only the
+@code{RTLD_DEFAULT} pseudo-handle should be used.
+
+@item
+Use the local-exec or initial-exec TLS models.  If @code{dlopen} is not
+used, there are no compatibility concerns for initial-exec TLS.  This
+TLS model avoids most of the complexity around TLS access.  In
+particular, there are no TLS-related run-time memory allocations after
+process or thread start.
+
+If shared objects are expected to be used more generally, outside the
+hardened, feature-restricted context, lack of compatibility between
+@code{dlopen} and initial-exec TLS could be a concern.  In that case,
+the second-best alternative is to use global-dynamic TLS with GNU2 TLS
+descriptors, for targets that fully implement them, including the fast
+path for access to TLS variables defined in the initially loaded set of
+objects.  Like initial-exec TLS, this avoids memory allocations after
+thread creation, but only if the @code{dlopen} function is not used.
+
+@item
+Do not use lazy binding.  Lazy binding may require run-time memory
+allocation, is not async-signal-safe, and introduces considerable
+complexity.
+
+@item
+Make dependencies on shared objects explicit.  Do not assume that
+certain libraries (such as @code{libc.so.6}) are always loaded.
+Specifically, if a main program or shared object references a symbol,
+create an ELF @code{DT_NEEDED} dependency on that shared object, or on
+another shared object that is documented (or otherwise guaranteed) to
+have the required explicit dependency.  Referencing a symbol without a
+matching link dependency results in underlinking, and underlinked
+objects cannot always be loaded correctly: Initialization of objects may
+not happen in the required order.
+
+@item
+Do not create dependency loops between shared objects (@code{libA.so.1}
+depending on @code{libB.so.1} depending on @code{libC.so.1} depending on
+@code{libA.so.1}).  @Theglibc{} has to initialize one of the objects in
+the cycle first, and the choice of that object is arbitrary and can
+change over time.  The object which is initialized first (and other
+objects involved in the cycle) may not run correctly because not all of
+its dependencies have been initialized.
+
+Underlinking (see above) can hide the presence of cycles.
+
+@item
+Limit the creation of indirect function (IFUNC) resolvers.  These
+resolvers run during relocation processing, when @theglibc{} is not in
+a fully consistent state.  If you write your own IFUNC resolvers, do
+not depend on external data or function references in those resolvers.
+
+@item
+Do not use the audit functionality (@code{LD_AUDIT}, @code{DT_AUDIT},
+@code{DT_DEPAUDIT}).  Its callback and hooking capabilities introduce a
+lot of complexity and subtly alter dynamic linker behavior in corner
+cases even if the audit module is inactive.
+
+@item
+Do not use symbol interposition.  Without symbol interposition, the
+exact order in which shared objects are searched are less relevant.
+
+Exceptions to this rule are copy relocations (see the next item), and
+vague linkage, as used by the C++ implementation (see below).
+
+@item
+One potential source of symbol interposition is a combination of static
+and dynamic linking, namely linking a static archive into multiple
+dynamic shared objects.  For such scenarios, the static library should
+be converted into its own dynamic shared object.
+
+A different approach to this situation uses hidden visibility for
+symbols in the static library, but this can cause problems if the
+library does not expect that multiple copies of its code coexist within
+the same process, with no or partial sharing of state.
+
+@item
+If you use shared objects that are linked with @option{-Wl,-Bsymbolic}
+(or equivalent) or use protected visibility, the code for the main
+program must be built as @option{-fpic} or @option{-fPIC} to avoid
+creating copy relocations (and the main program must not use copy
+relocations for other reasons).  Using @option{-fpie} or @option{-fPIE}
+is not an alternative to PIC code in this context.
+
+@item
+Be careful about explicit section annotations.  Make sure that the
+target section matches the properties of the declared entity (e.g., no
+writable objects in @code{.text}).
+
+@item
+Ensure that all assembler or object input files have the recommended
+security markup, particularly for non-executable stack.
+
+@item
+Avoid using non-default linker flags and features.  In particular, do
+not use the @code{DT_PREINIT_ARRAY} dynamic tag, and do not flag
+objects as @code{DF_1_INITFIRST}.  Do not change the default linker
+script of BFD ld.  Do not override ABI defaults, such as the dynamic
+linker path (with @option{--dynamic-linker}).
+
+@item
+Some features of @theglibc{} indirectly depend on run-time code loading
+and @code{dlopen}.  Use @code{iconv_open} with built-in converters only
+(such as @code{UTF-8}).  Do not use NSS functionality such as
+@code{getaddrinfo} or @code{getpwuid_r} unless the system is configured
+for built-in NSS service modules only (see below).
+@end itemize
+
+Several considerations apply to ELF constructors and destructors.
+
+@itemize @bullet
+@item
+The dynamic linker does not take constructor and destructor priorities
+into account when determining their execution order.  Priorities are
+only used by the link editor for ordering execution within a
+completely linked object.  If a dynamic shared object needs to be
+initialized before another object, this can be expressed with a
+@code{DT_NEEDED} dependency on the object that needs to be initialized
+earlier.
+
+@item
+The recommendations to avoid cyclic dependencies and symbol
+interposition make it less likely that ELF objects are accessed before
+their ELF constructors have run.  However, using @code{dlsym} and
+@code{dlvsym}, it is still possible to access uninitialized facilities
+even with these restrictions in place.  (Of course, access to
+uninitialized functionality is also possible within a single shared
+object or the main executable, without resorting to explicit symbol
+lookup.)  Consider using dynamic, on-demand initialization instead.  To
+deal with access after de-initialization, it may be necessary to
+implement special cases for that scenario, potentially with degraded
+functionality.
+
+@item
+Be aware that when ELF destructors are executed, it is possible to
+reference already-deconstructed shared objects.  This can happen even in
+the absence of @code{dlsym} and @code{dlvsym} function calls, for
+example if client code using a shared object has registered callbacks or
+objects with another shared object.  The ELF destructor for the client
+code is executed before the ELF destructor for the shared objects that
+it uses, based on the expected dependency order.
+
+@item
+If @code{dlopen} and @code{dlmopen} are not used, @code{DT_NEEDED}
+dependency information is complete, and lazy binding is disabled, the
+execution order of ELF destructors is expected to be the reverse of the
+ELF constructor order.  However, two separate dependency sort operations
+still occur.  Even though the listed preconditions should ensure that
+both sorts produce the same ordering, it is recommended not to depend on
+the destructor order being the reverse of the constructor order.
+@end itemize
+
+The following items provide C++-specific guidance for preparing
+applications.  If another programming language is used and it uses these
+toolchain features targeted at C++ to implement some language
+constructs, these restrictions and recommendations still apply in
+analogous ways.
+
+@itemize @bullet
+@item
+C++ inline functions, templates, and other constructs may need to be
+duplicated into multiple shared objects using vague linkage, resulting
+in symbol interposition.  This type of symbol interposition is
+unproblematic, as long as the C++ one definition rule (ODR) is followed,
+and all definitions in different translation units are equivalent
+according to the language C++ rules.
+
+@item
+Be aware that under C++ language rules, it is unspecified whether
+evaluating a string literal results in the same address for each
+evaluation.  This also applies to anonymous objects of static storage
+duration that GCC creates, for example to implement the compound
+literals C++ extension.  As a result, comparing pointers to such
+objects, or using them directly as hash table keys, may give unexpected
+results.
+
+By default, variables of block scope of static storage have consistent
+addresses across different translation units, even if defined in
+functions that use vague linkage.
+
+@item
+If C++ is used with symbol visibility and version management are used
+(the GCC @samp{visibility} attribute, the GCC @option{-fvisibility}, or
+a linker version script with the linker option
+@option{--version-script}), it is necessary to ensure that the symbol
+management remains consistent with how the symbols are used.  This is
+complicated by the fact that some C++ constructs are implemented with
+the help of ancillary symbols.  For example, an inline function that is
+always inlined into its callers has no symbol footprint for the function
+itself, but if the function contains a variable of static storage
+duration, this variable may result in the creation of one or more global
+symbols.  For correctness, such symbols must be visible and bound in all
+other places where the inline function may be called.  This requirement
+is not met if the symbol visibility is set to hidden, or if symbols are
+assigned a textually different symbol version (effectively creating two
+distinct sets of symbols).
+
+Due to the complex interaction between ELF symbol management and C++
+symbol generation, it is recommended to use C++ language features for
+symbol management, in particular inline namespaces.
+
+@item
+The toolchain and dynamic linker have multiple mechanisms that bypass
+the usual symbol binding procedures.  This means that the C++ one
+definition rule (ODR) still holds even if certain symbol-based isolation
+mechanisms are used, and object addresses are not shared across
+translation units with incompatible type definitions.
+
+This does not matter if the original (language-independent) advice
+regarding symbol interposition is followed.  However, as the advice may
+be difficult to implement for C++ applications, it is recommended to
+avoid ODR violations across the entire process image.  Inline namespaces
+can be helpful in this context because they can be used to create
+distinct ELF symbols while maintaining source code compatibility at the
+C++ level.
+
+@item
+Be aware that as a special case of interposed symbols, symbols with the
+@code{STB_GNU_UNIQUE} binding type do not follow the usual ELF symbol
+namespace isolation rules: such symbols bind across @code{RTLD_LOCAL}
+boundaries.  Furthermore, symbol versioning is ignored for such symbols;
+they are bound by symbol name only.  All their definitions and uses must
+therefore be compatible.  Hidden visibility still prevents the creation
+of @code{STB_GNU_UNIQUE} symbols and can achieve isolation of
+incompatible definitions.
+
+@item
+C++ constructor priorities only affect constructor ordering within one
+shared object.  Global constructor order across shared objects is
+consistent with ELF dependency ordering if there are no ELF dependency
+cycles.
+
+@item
+C++ exception handling and run-time type information (RTTI), as
+implemented in the GNU toolchain, is not address-significant, and
+therefore is not affected by the symbol binding behaviour of the dynamic
+linker.  This means that types of the same fully-qualified name (in
+non-anonymous namespaces) are always considered the same from an
+exception-handling or RTTI perspective.  This is true even if the type
+information object or vtable has hidden symbol visibility, or the
+corresponding symbols are versioned under different symbol versions, or
+the symbols are not bound to the same objects due to the use of
+@code{RTLD_LOCAL} or @code{dlmopen}.
+
+This can cause issues in applications that contain multiple incompatible
+definitions of the same type.  Inline namespaces can be used to create
+distinct symbols at the ELF layer, avoiding this type of issue.
+
+@item
+C++ exception handling across multiple @code{dlmopen} namespaces may
+not work, particular with the unwinder in GCC versions before 12.
+Current toolchain versions are able to process unwinding tables across
+@code{dlmopen} boundaries.  However, note that type comparison is
+name-based, not address-based (see the previous item), so exception
+types may still be matched in unexpected ways.  An important special
+case of exception handling, invoking destructors for variables of block
+scope, is not impacted by this RTTI type-sharing.  Likewise, regular
+virtual member function dispatch for objects is unaffected (but still
+requires that the type definitions match in all directly involved
+translation units).
+
+Once more, inline namespaces can be used to create distinct ELF symbols
+for different types.
+
+@item
+Although the C++ standard requires that destructors for global objects
+run in the opposite order of their constructors, the Itanium C++ ABI
+requires a different destruction order in some cases.  As a result, do
+not depend on the precise destructor invocation order in applications
+that use @code{dlclose}.
+
+@item
+Registering destructors for later invocation allocates memory and may
+silently fail if insufficient memory is available.  As a result, the
+destructor is never invoked.  This applies to all forms of destructor
+registration, with the exception of thread-local variables (see the next
+item).  To avoid this issue, ensure that such objects merely have
+trivial destructors, avoiding the need for registration, and deallocate
+resources using a different mechanism (for example, from an ELF
+destructor).
+
+@item
+A similar issue exists for @code{thread_local} or @code{__thread}
+variables with thread storage duration of types that have non-trivial
+destructors.  However, in this case, memory allocation failure during
+registration leads to process termination.  If process termination is
+not acceptable, use @code{thread_local} variables with trivial
+destructors only.  Functions for per-thread cleanup can be registered
+using @code{pthread_key_create} (globally for all threads) and
+activated using @code{pthread_setspecific} (on each thread).  Note
+that a @code{pthread_key_create} call may still fail (and
+@code{pthread_create} keys are a limited resource in @theglibc{}), but
+this failure can be handled without terminating the process.
+@end itemize
+
+@subsection Producing Matching Binaries
+
+This subsection recommends tools and build flags for producing
+applications that meet the recommendations of the previous subsection.
+
+@itemize @bullet
+@item
+Use BFD ld (@command{bfd.ld}) from GNU binutils to produce binaries,
+invoked through a compiler driver such as @command{gcc}.  The version
+should be not too far ahead of what was current when the version of
+@theglibc{} was first released.
+
+@item
+Do not use a binutils release that is older than the one used to build
+@theglibc{} itself.
+
+@item
+Compile with @option{-ftls-model=initial-exec} to force the initial-exec
+TLS model.
+
+@item
+Link with @option{-Wl,-z,now} to disable lazy binding.
+
+@item
+Link with @option{-Wl,-z,relro} to enable RELRO (which is the default on
+most targets).
+
+@item
+Specify all direct shared objects dependencies using @option{-l} options
+to avoid underlinking.  Rely on @code{.so} files (which can be linker
+scripts) and searching with the @option{-l} option.  Do not specify the
+file names of shared objects on the linker command line.
+
+@item
+Consider using @option{-Wl,-z,defs} to treat underlinking as an error
+condition.
+
+@item
+When creating a shared object (linked with @option{-shared}), use
+@option{-Wl,-soname,lib@dots{}} to set a soname that matches the final
+installed name of the file.
+
+@item
+Do not use the @option{-rpath} linker option.  (As explained below, all
+required shared objects should be installed into the default search
+path.)
+
+@item
+Use @option{-Wl,--error-rwx-segments} and @option{-Wl,--error-execstack} to
+instruct the link editor to fail the link if the resulting final object
+would have read-write-execute segments or an executable stack.  Such
+issues usually indicate that the input files are not marked up
+correctly.
+
+@item
+Ensure that for each @code{LOAD} segment in the ELF program header, file
+offsets, memory sizes, and load addresses are multiples of the largest
+page size supported at run time.  Similarly, the start address and size
+of the @code{GNU_RELRO} range should be multiples of the page size.
+
+Avoid creating gaps between @code{LOAD} segments.  The difference
+between the load addresses of two subsequent @code{LOAD} segments should
+be the size of the first @code{LOAD} segment.  (This may require linking
+with @option{-Wl,-z,noseparate-code}.)
+
+This may not be possible to achieve with the currently available link
+editors.
+
+@item
+If the multiple-of-page-size criterion for the @code{GNU_RELRO} region
+cannot be achieved, ensure that the process memory image right before
+the start of the region does not contain executable or writable memory.
+@c https://sourceware.org/pipermail/libc-alpha/2022-May/138638.html
+@end itemize
+
+@subsection Checking Binaries
+
+In some cases, if the previous recommendations are not followed, this
+can be determined from the produced binaries.  This section contains
+suggestions for verifying aspects of these binaries.
+
+@itemize @bullet
+@item
+To detect underlinking, examine the dynamic symbol table, for example
+using @samp{readelf -sDW}.  If the symbol is defined in a shared object
+that uses symbol versioning, it must carry a symbol version, as in
+@samp{pthread_kill@@GLIBC_2.34}.
+
+@item
+Examine the dynamic segment with @samp{readelf -dW} to check that all
+the required @code{NEEDED} entries are present.  (It is not necessary to
+list indirect dependencies if these dependencies are guaranteed to
+remain during the evolution of the explicitly listed direct
+dependencies.)
+
+@item
+The @code{NEEDED} entries should not contain full path names including
+slashes, only @code{sonames}.
+
+@item
+For a further consistency check, collect all shared objects referenced
+via @code{NEEDED} entries in dynamic segments, transitively, starting at
+the main program.  Then determine their dynamic symbol tables (using
+@samp{readelf -sDW}, for example).  Ideally, every symbol should be
+defined at most once, so that symbol interposition does not happen.
+
+If there are interposed data symbols, check if the single interposing
+definition is in the main program.  In this case, there must be a copy
+relocation for it.  (This only applies to targets with copy relocations.)
+
+Function symbols should only be interposed in C++ applications, to
+implement vague linkage.  (See the discussion in the C++ recommendations
+above.)
+
+@item
+Using the previously collected @code{NEEDED} entries, check that the
+dependency graph does not contain any cycles.
+
+@item
+The dynamic segment should also mention @code{BIND_NOW} on the
+@code{FLAGS} line or @code{NOW} on the @code{FLAGS_1} line (one is
+enough).
+
+@item
+For shared objects (not main programs), if the program header has a
+@code{PT_TLS} segment, the dynamic segment (as shown by @samp{readelf
+-dW}) should contain the @code{STATIC_TLS} flag on the @code{FLAGS}
+line.
+
+If @code{STATIC_TLS} is missing in shared objects, ensure that the
+appropriate relocations for GNU2 TLS descriptors are used (for example,
+@code{R_AARCH64_TLSDESC} or @code{R_X86_64_TLSDESC}).
+
+@item
+There should not be a reference to the symbols @code{__tls_get_addr},
+@code{__tls_get_offset}, @code{__tls_get_addr_opt} in the dynamic symbol
+table (in the @samp{readelf -sDW} output).  Thread-local storage must be
+accessed using the initial-exec (static) model, or using GNU2 TLS
+descriptors.
+
+@item
+Likewise, the functions @code{dlopen}, @code{dlmopen}, @code{dlclose}
+should not be referenced from the dynamic symbol table.
+
+@item
+For shared objects, there should be a @code{SONAME} entry that matches
+the file name (the base name, i.e., the part after the slash).  The
+@code{SONAME} string must not contain a slash @samp{/}.
+
+@item
+For all objects, the dynamic segment (as shown by @samp{readelf -dW})
+should not contain @code{RPATH} or @code{RUNPATH} entries.
+
+@item
+Likewise, the dynamic segment should not show any @code{AUDIT},
+@code{DEPAUDIT}, @code{AUXILIARY}, @code{FILTER}, or
+@code{PREINIT_ARRAY} tags.
+
+@item
+If the dynamic segment contains a (deprecated) @code{HASH} tag, it
+must also contain a @code{GNU_HASH} tag.
+
+@item
+The @code{INITFIRST} flag (undeer @code{FLAGS_1}) should not be used.
+
+@item
+The program header must not have @code{LOAD} segments that are writable
+and executable at the same time.
+
+@item
+All produced objects should have a @code{GNU_STACK} program header that
+is not marked as executable.  (However, on some newer targets, a
+non-executable stack is the default, so the @code{GNU_STACK} program
+header is not required.)
+@end itemize
+
+@subsection Run-time Considerations
+
+In addition to preparing program binaries in a recommended fashion, the
+run-time environment should be set up in such a way that problematic
+dynamic linker features are not used.
+
+@itemize @bullet
+@item
+Install shared objects using their sonames in a default search path
+directory (usually @file{/usr/lib64}).  Do not use symbolic links.
+@c This is currently not standard practice.
+
+@item
+The default search path must not contain objects with duplicate file
+names or sonames.
+
+@item
+Do not use environment variables (@code{LD_@dots{}} variables such as
+@code{LD_PRELOAD} or @code{LD_LIBRARY_PATH}, or @code{GLIBC_TUNABLES})
+to change default dynamic linker behavior.
+
+@item
+Do not install shared objects in non-default locations.  (Such locations
+are listed explicitly in the configuration file for @command{ldconfig},
+usually @file{/etc/ld.so.conf}, or in files included from there.)
+
+@item
+In relation to the previous item, do not install any objects it
+@code{glibc-hwcaps} subdirectories.
+
+@item
+Do not configure dynamically-loaded NSS service modules, to avoid
+accidental internal use of the @code{dlopen} facility.  The @code{files}
+and @code{dns} modules are built in and do not rely on @code{dlopen}.
+
+@item
+Do not truncate and overwrite files containing programs and shared
+objects in place, while they are used.  Instead, write the new version
+to a different path and use @code{rename} to replace the
+already-installed version.
+
+@item
+Be aware that during a component update procedure that involves multiple
+object files (shared objects and main programs), concurrently starting
+processes may observe an inconsistent combination of object files (some
+already updated, some still at the previous version).  For example,
+this can happen during an update of @theglibc{} itself.
+@end itemize
 
 @c FIXME these are undocumented:
 @c dladdr

base-commit: e7ac92e6ca9784b397189df0b2e1fb34f425bab8