[PATCH 00/30] RFC: elf: glibc-hwcaps support

Florian Weimer fweimer@redhat.com
Mon Jun 22 15:12:58 GMT 2020

This series implements searching for alternative library implementations
under glibc-hwcaps subdirectories on the library search path.  It's
lacking tests, documentation, and a NEWS file update, but I wanted to
submit it early for discussion (and perhaps review of some of the
cleanup patches).

For background, it is best reference first the discussion about the
current hwcaps subdirectory processing:

  hwcaps subdirectory selection in the dynamic loader

The new approach uses a special subdirectory, called "glibc-hwcaps",
which contains nested subdirectories with all the alternative library
implementations. The glibc-hwcaps subdirectory can be present in every
subdirectory of the library search path.

ldconfig recognizes the "glibc-hwcaps" name, and it receives special
treatment if it exists: subdirectories of the directory (but the
directory itself) are scanned for libraries.  If a library is found, it
is put into the cache, and associated with the name of the subdirectory
(of glibc-hwcaps).  ldconfig does this for all subdirectories (whose
name does not begin with '.'), so it does not have to be updated if new
subdirectories are added.  There is currently no symbolic link
processing for these subdirectories.  In the future, they might store
multiple implementations with different GNU property notes, and we might
want to select those through the cache as well.  (I'm open to changing
this part to align more with traditional cache processing.)

At run time, the dynamic loader has a built-in list of HWCAP
subdirectory names (which are different from the legacy HWCAP
directories).  The new “ld.so --help” option shows these subdirectory
names.  For LD_LIBRARY_PATH processing, ld.so simply attempts to open
libraries under their soname in the active/supported glibc-hwcaps
subdirectories.  (This part is quite similar to the original HWCAP
subdirectory handling, except that the new subdirectories do not nest.)
For ld.so.cache processing, ld.so sorts the active/supported HWCAP
subdirectory names and merges this list against the list of subdirectory
names as stored in a new ld.so.cache extension section.  Each
glibc-hwcaps library listed in the cache has a HWCAP field that contains
an index into the HWCAP subdirectory array of the cache file.  (A
previously unused HWCAP bit is set so that these entries are ignored by
existing dynamic linkers.)  The dynamic linker loads the library with
has the lowest positive preference value (essentially from the
subdirectory that comes first in its consolidated HWCAP name list).

There are new ld.so options, --glibc-hwcaps-prepend and
--glibc-hwcaps-mask, to change the list of glibc-hwcaps subdirectories
searched.  Due to the way ldconfig has been modified, it is possible to
pick up completely new subdirectories (not known to the implementation)
using --glibc-hwcaps-prepend even if ld.so.cache is used.

With these changes, on a current x86-64 machine (with AVX2-level CPU
features), you can drop a shared object into the directory


and the dynamic linker will load it, either via ld.so.cache or
LD_LIBRARY_PATH processing.

Under strace, it look like this if the cache is used:

| openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
| openat(AT_FDCWD, "/lib64/glibc-hwcaps/x86-102/libz.so.1", O_RDONLY|O_CLOEXEC) = 3

Diagnostics output from “ld.so --help” for this machine is this:

| Shared library search path:
|   (libraries located via /etc/ld.so.cache)
|   /lib64 (system search path)
|   /usr/lib64 (system search path)
| Subdirectories of glibc-hwcaps directories, in priority order:
|   x86-103
|   x86-102 (supported, searched)
|   x86-101 (supported, searched)
|   x86-100 (supported, searched)
| Legacy HWCAP subdirectories under library search path directories:
|   haswell (AT_PLATFORM)
|   tls (supported, searched)
|   avx512_1
|   x86_64 (supported, searched)

I've also included support for a "power9" subdirectory on powerpc64le,
some subdirectories on s390x, and "atomics" on aarch64.  The exact
definitions of the directories are still subject to change.  They are
there to that there is at least something new to be searched.  For the
x86-64 part, I'm going to start a separate, cross-project discussion
about the desired CPU feature flags for each micro-architecture baseline

The series does not yet check if the glibc-hwcaps directory itself
exists.  If the directory does not exist, we can skip searching all its
subdirectories.  This is another motivation for having the magic
glibc-hwcaps subdirectory.  But I have not implemented this yet because
it tends to hide the search activity from strace.

Patches up to (but excluding) “elf: Add glibc-hwcaps support for
LD_LIBRARY_PATH” are more or less cleanups, and so are these patches:

  elf: Unify old and new format cache handling code in ld.so
  elf: Implement a string table for ldconfig, with tail merging
  elf: Implement tail merging of strings in ldconfig
  elf: In ldconfig, extract the new_sub_entry function from search_dir

These cleanups could be reviewed and installed separately.


Florian Weimer (30):
  elf: Include <stdbool.h> in <dl-tunables.h> because bool is used
  elf: Include <stddef.h> (for size_t), <sys/stat.h> in <ldconfig.h>
  elf: Do not search HWCAP subdirectories in statically linked binaries
  elf: Implement __rtld_malloc_is_full
  elf: Implement _dl_write
  elf: Extract command-line/environment variables state from rtld.c
  elf: Move ld.so error/help output to _dl_usage
  elf: Record whether paths come from LD_LIBRARY_PATH or --library-path
  elf: Implement ld.so --help
  elf: Implement ld.so --version
  scripts/update-copyrights: Update csu/version.c, elf/dl-usage.c
  elf: Use the term "program interpreter" in the ld.so help message
  elf: Print the full name of the dynamic loader in the ld.so help
  elf: Make __rtld_env_path_list and __rtld_search_dirs global variables
  elf: Add library search path information to ld.so --help
  elf: Enhance ld.so --help to print HWCAP subdirectories
  elf: Do not pass GLRO(dl_platform), GLRO(dl_platformlen) to
  elf: Add glibc-hwcaps support for LD_LIBRARY_PATH
  x86_64: Add glibc-hwcaps support
  powerpc64le: Add glibc-hwcaps support
  s390x: Add Add glibc-hwcaps support
  aarch64: Add glibc-hwcaps support
  elf: Add endianness markup to ld.so.cache
  elf: Add extension mechanism to ld.so.cache
  elf: Unify old and new format cache handling code in ld.so
  elf: Implement a string table for ldconfig, with tail merging
  elf: Implement tail merging of strings in ldconfig
  elf: In ldconfig, extract the new_sub_entry function from search_dir
  elf: Process glibc-hwcaps subdirectories in ldconfig
  elf: Add glibc-hwcaps subdirectory support to ld.so cache processing

 NEWS                                          |   4 +
 elf/Makefile                                  |  22 +-
 elf/cache.c                                   | 400 +++++++++++++--
 elf/dl-cache.c                                | 483 ++++++++++++------
 elf/dl-hwcaps-subdirs.c                       |  29 ++
 elf/dl-hwcaps.c                               | 225 +++++++-
 elf/dl-hwcaps.h                               | 102 ++++
 elf/dl-hwcaps_split.c                         |  77 +++
 elf/dl-load.c                                 |  75 +--
 elf/dl-main.h                                 | 120 +++++
 elf/dl-minimal.c                              |   8 +
 elf/dl-support.c                              |   5 +-
 elf/dl-tunables.h                             |   2 +
 elf/dl-usage.c                                | 267 ++++++++++
 elf/dl-write.c                                |  56 ++
 elf/ldconfig.c                                | 187 +++++--
 elf/rtld.c                                    | 248 ++++-----
 elf/stringtable.c                             | 201 ++++++++
 elf/stringtable.h                             |  61 +++
 elf/stringtable_free.c                        |  32 ++
 elf/tst-dl-hwcaps_split.c                     | 139 +++++
 elf/tst-stringtable.c                         | 140 +++++
 include/link.h                                |   4 +
 include/rtld-malloc.h                         |   4 +
 scripts/update-copyrights                     |   6 +
 sysdeps/aarch64/dl-hwcaps-subdirs.c           |  31 ++
 sysdeps/generic/dl-cache.h                    | 232 ++++++++-
 sysdeps/generic/ldconfig.h                    |  20 +-
 sysdeps/generic/ldsodefs.h                    |  34 +-
 .../powerpc/powerpc64/le/dl-hwcaps-subdirs.c  |  31 ++
 sysdeps/s390/s390-64/dl-hwcaps-subdirs.c      |  54 ++
 sysdeps/unix/sysv/linux/dl-write.c            |  30 ++
 sysdeps/x86_64/dl-hwcaps-subdirs.c            |  73 +++
 33 files changed, 2980 insertions(+), 422 deletions(-)
 create mode 100644 elf/dl-hwcaps-subdirs.c
 create mode 100644 elf/dl-hwcaps_split.c
 create mode 100644 elf/dl-main.h
 create mode 100644 elf/dl-usage.c
 create mode 100644 elf/dl-write.c
 create mode 100644 elf/stringtable.c
 create mode 100644 elf/stringtable.h
 create mode 100644 elf/stringtable_free.c
 create mode 100644 elf/tst-dl-hwcaps_split.c
 create mode 100644 elf/tst-stringtable.c
 create mode 100644 sysdeps/aarch64/dl-hwcaps-subdirs.c
 create mode 100644 sysdeps/powerpc/powerpc64/le/dl-hwcaps-subdirs.c
 create mode 100644 sysdeps/s390/s390-64/dl-hwcaps-subdirs.c
 create mode 100644 sysdeps/unix/sysv/linux/dl-write.c
 create mode 100644 sysdeps/x86_64/dl-hwcaps-subdirs.c


More information about the Libc-alpha mailing list