Are the pthread "compatibility" copies of symbols in libc still necessary?

[Carlos O'Donell]

On 03/16/2018 08:41 AM, Florian Weimer wrote:
> On 03/16/2018 03:30 PM, Zack Weinberg wrote:
>> A number of source files that properly belong to libc.so are also
>> compiled as part of libpthread, with a note that this is for
>> "compatibility for old binaries".  The exact set varies based on
>> architecture, but includes basic things like read, write, and fork - I
>> _think_ there was a difference in semantics in the distant past,
>> having something to do with thread cancellation.
> 
> They are still needed because versioned symbols also embed a DSO name, and the dynamic linker checks that if it has not been interposed.
> 
> I think it's this code in elf/dl-lookup.c:
> 
> Â Â Â Â Â  if (__glibc_unlikely (res < 0) && skip_map == NULL)
> Â Â Â Â {
>       /* Oh, oh.  The file named in the relocation entry does not
>          contain the needed symbol.  This code is never reached
>          for unversioned lookups.  */
> Â Â Â Â Â  assert (version != NULL);
> Â Â Â Â Â  const char *reference_name = undef_map ? undef_map->l_name : "";
> Â Â Â Â Â  struct dl_exception exception;
>       /* XXX We cannot translate the message.  */
> Â Â Â Â Â  _dl_exception_create_format
> Â Â Â Â Â Â Â  (&exception, DSO_FILENAME (reference_name),
> Â Â Â Â Â Â Â Â  "symbol %s version %s not defined in file %s"
> Â Â Â Â Â Â Â Â  " with link time reference%s",
> Â Â Â Â Â Â Â Â  undef_name, version->name, version->filename,
> Â Â Â Â Â Â Â Â  res == -2 ? " (no version symbols)" : "");
> Â Â Â Â Â  _dl_signal_cexception (0, &exception, N_("relocation error"));
> Â Â Â Â Â  _dl_exception_free (&exception);
> Â Â Â Â Â  *ref = NULL;
> Â Â Â Â Â  return 0;
> Â Â Â Â }
> 
> This is required by the GNU symbol versioning spec, but I don't know why.

The specification says the data must be recorded, but it doesn't say what you
have to *do* with the data?

I am of the opinion that this is simply to produce a reasonable error message
if the versioned symbol is missing completely.

We should probably reach out to Ulrich to see if has any opinion on this,
as the primary author of the specification he might have an input here.

My own opinion is that the check is overly restrictive, and that we could relax
it to allow versioned symbols to move to other shared objects.

> Even new binaries use these symbols:
> 
> $ LD_DEBUG=bindings systemctl |& grep binding.*system.*libpthread.*[^_]fork
> Â Â Â Â  11905:Â Â Â  binding file /usr/lib/systemd/libsystemd-shared-234.so [0] to /lib64/libpthread.so.0 [0]: normal symbol `fork' [GLIBC_2.2.5]
> Â Â Â Â  11905:Â Â Â  binding file systemctl [0] to /lib64/libpthread.so.0 [0]: normal symbol `fork' [GLIBC_2.2.5]
> 
> This may be a linker bug because at least fork is a compat symbol.

Is it though? libc.so.6 has fork@@GLIBC_2.2.5, which causes the binary to have
a reference to fork@GLIBC_2.2.5, and that's correct. However, now we have both
libc.so.6 and libpthread.so.0 with definitions of fork at GLIBC_2.2.5.

Also libsystemd-shared-234.so has:

000000000043f580  0000019a00000007 R_X86_64_JUMP_SLOT     0000000000000000 fork@GLIBC_2.2.5 + 0
   410: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND fork@GLIBC_2.2.5 (2)

  0x0170: Version: 1  File: libc.so.6  Cnt: 16
...
  0x0270:   Name: GLIBC_2.2.5  Flags: none  Version: 2

Is it a dynamic loader bug that the compat symbol was selected?

systemctl has libpthread.so.0 as the *first* DT_NEEDED entry, and we process them in order.

 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libsystemd-shared-234.so]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]

So libpthread.so.0 interposes the versioned symbol from libc.so.6.

     26067:     Initial object scopes
     26067:     object=systemctl [0]
     26067:      scope 0: systemctl /lib64/libpthread.so.0 /lib64/libc.so.6 /usr/lib/systemd/libsystemd-shared-234.so /lib64/libgcc_s.so.1 /lib64/ld-linux-x86-64.so.2 /lib64/librt.so.1 /lib64/libcap.so.2 /lib64/libacl.so.1 /lib64/libcryptsetup.so.4 /lib64/libgcrypt.so.20 /lib64/libip4tc.so.0 /lib64/libseccomp.so.2 /lib64/libselinux.so.1 /lib64/libidn.so.11 /lib64/liblzma.so.5 /lib64/liblz4.so.1 /lib64/libblkid.so.1 /lib64/libattr.so.1 /lib64/libuuid.so.1 /lib64/libdevmapper.so.1.02 /lib64/libdl.so.2 /lib64/libgpg-error.so.0 /lib64/libpcap.so.1 /lib64/libpcre2-8.so.0 /lib64/libsepol.so.1 /lib64/libudev.so.1 /lib64/libm.so.6
     26067:

So the search scope is libpthread *first*.

The above error never triggers because libc.so.6 *does* have the named versioned symbol,
but libpthread.so.0 also has it too.

My conclusion is this:

* We can remove fork@GLIBC_2.2.5 from libpthread.so, the shared object doesn't
  encode it as being needed and won't cause a failure per the rules that
  require the referenced shared object to have the versioned symbol that was
  bound at static link time.

Cheers,
Carlos.