Bug 26018 - --dynamic-list* doesn't work before -Bsymbolic and -Bsymbolic-functions
Summary: --dynamic-list* doesn't work before -Bsymbolic and -Bsymbolic-functions
Status: RESOLVED FIXED
Alias: None
Product: binutils
Classification: Unclassified
Component: ld (show other bugs)
Version: 2.35
: P2 normal
Target Milestone: 2.35
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-05-21 01:26 UTC by Fangrui Song
Modified: 2021-06-14 14:36 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed: 2020-05-23 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Fangrui Song 2020-05-21 01:26:56 UTC
The interaction is currently quite complex. To make the behavior easier to reasonable, let's make --dynamic-list* override -Bsymbolic and -Bsymbolic-functions.

This also helps PR ld/25910, which would make the situation more difficult to understand.
Comment 1 H.J. Lu 2020-05-22 13:02:20 UTC
--dynamic-list=DYNAMIC-LIST-FILE'
     Specify the name of a dynamic list file to the linker.  This is
     typically used when creating shared libraries to specify a list of
     global symbols whose references shouldn't be bound to the
     definition within the shared library, or creating dynamically
     linked executables to specify a list of symbols which should be
     added to the symbol table in the executable.  This option is only
     meaningful on ELF platforms which support shared libraries.

The --dynamic-list* options are intended for shared libraries.  The goal
IS NOT put them in dynamic symbol table since all global symbols are in
dynamic symbol table already in a shared library.  The goal is to make
them PREEMPTIBLE.  So making the --dynamic-list* options override -Bsymbolic
and -Bsymbolic-functions is incorrect since there is NOTHING to override.
Comment 2 H.J. Lu 2020-05-23 14:39:47 UTC
Nothing to fix.
Comment 3 Fangrui Song 2020-05-23 16:10:26 UTC
I don't agree with the resolution. Copying my reply https://sourceware.org/pipermail/binutils/2020-May/111223.html here

IIRC, currently -Bsymbolic --dynamic-list a.list != --dynamic-list a.list
 -Bsymbolic

This order dependency is error-prone. It is still fixable because I don't think any project makes uses of the combination. Order dependency also makes it unclear how --dynamic-list associates with other symbol exporting options.


--dynamic-list* work for both executables and shared libraries.
For an executable, export some symbols.
For a shared library, specify preemptible symbols.


--dynamic-list* have function overlay
with -Bsymbolic. When two options overlap in functionality, many users
expect the more fine-grained option to win, thus it is preferable for 
--dynamic-list* to override -Bsymbolic and -Bsymbolic-functions.

We could also emit a warning and say -Bsymbolic is ignored, if that matters.
Comment 4 H.J. Lu 2020-05-23 18:59:56 UTC
--dynamic-list* should work -Bsymbolic and -Bsymbolic-functions.
Comment 5 Sourceware Commits 2020-05-24 03:56:46 UTC
The master branch has been updated by H.J. Lu <hjl@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=309cae1f7bbb6c4de0234e9f592d7dad3d6c61ba

commit 309cae1f7bbb6c4de0234e9f592d7dad3d6c61ba
Author: H.J. Lu <hjl.tools@gmail.com>
Date:   Sat May 23 20:49:16 2020 -0700

    ld: Add -Bsymbolic-functions tests
    
            PR ld/26018
            * testsuite/ld-i386/i386.exp: Add a -Bsymbolic-functions test.
            * testsuite/ld-x86-64/x86-64.exp: Likewise.
            * testsuite/ld-i386/pr26018.d: New file.
            * testsuite/ld-x86-64/pr26018.d: Likewise.
            * testsuite/ld-x86-64/pr26018.s: Likewise.
Comment 6 Sourceware Commits 2020-05-24 11:51:11 UTC
The master branch has been updated by H.J. Lu <hjl@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=bb68f22c8e648032a0d1c1d17353eec599ff5e6a

commit bb68f22c8e648032a0d1c1d17353eec599ff5e6a
Author: Fangrui Song <maskray@google.com>
Date:   Wed May 20 18:31:39 2020 -0700

    ld: Handle --dynamic-list* before -Bsymbolic -Bsymbolic-functions
    
    --dynamic-list* should work both before and after -Bsymbolic and
    -Bsymbolic-functions.
    
            PR ld/26018
            * lexsup.c (parse_args): Simplify.
            * testsuite/ld-elf/dl4e.out: New.
            * testsuite/ld-elf/shared.exp: Updated for PR ld/26018 tests.
Comment 7 H.J. Lu 2020-05-24 11:54:11 UTC
Fixed for 2.35.
Comment 8 Michael Matz 2021-06-09 20:06:12 UTC
This might have broken usage in the wild.  In libqt4 this feature is used as
follows:

g++ ... -shared -Wl,-Bsymbolic-functions -Wl,--dynamic-list,QtCore.dynlist ...

with QtCore.dynlist only containing function names:

-----------------
{
     extern "C" {
         "qt_startup_hook";
         "qt_addObject";
         "qt_removeObject";
     };
     extern "C++" {
         "QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>)";
     };
};
-----------------

The intent is that all functions are bound locally, except for the few ones
explicitely mentioned.  That worked fine with binutils 2.32, an excerpt from the
relocations:

...
00000000004e4bc8  0000047e00000006 R_X86_64_GLOB_DAT      00000000004e5900 _ZN7QString11shared_nullE + 0
...

I've singled out this one because it's important: this shared_null member _must
not_ be bound symbolically, there must be only one in the process image, and
there exist (non-PIE) binaries that contain COPY relocations on it.  So relocations to this symbol need to stay in the shared lib, i.e. it must remain
dynamic.

With binutils 2.35 _ZN7QString11shared_nullE is bound symbolically, there's
no trace of it in the relocation of libQtCore.so.4 anymore, breaking stuff
in interesting ways (e.g. some executables hang at start).

I think what this (old) libqt4 link command is doing is reasonable: it wants
default behaviour for symbol binding, then overrides this for functions (to be
bound symbolically) and then overrides _that_ for some further functions (to
again be bound as per default, i.e. late).  I.e. the overall expectation that
data objects remain bound late with that command line seem sensible.

I haven't yet verified if this commit is really at fault, but it looks at least
related and I wanted to leave a trace in case I can't continue with analysis 
tomorrow.  If it's not this, then it's some other change since 2.32.
Comment 9 Michael Matz 2021-06-14 14:36:44 UTC
So, this commit is at fault, but it's already reported as PR26928 and that is a
duplicate of PR26407.  I was even involved in analyzing it there but obviously
forgot :-/

Then let me at least note that I disagree with the resolution of these bugs that
all is well, and as intended.  I still think what I said here in comment #8,
namely that what libqt4 does here is sensible, and that --dynamic-list-data
should not be needed to make this case work.  (It does work-around the problem
here).