Bug 22751 - LTO broken for libgcc libcalls
Summary: LTO broken for libgcc libcalls
Status: RESOLVED FIXED
Alias: None
Product: binutils
Classification: Unclassified
Component: ld (show other bugs)
Version: 2.30
: P2 normal
Target Milestone: ---
Assignee: Alan Modra
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-01-26 16:57 UTC by James Cowgill
Modified: 2018-03-31 12:42 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed: 2018-01-26 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description James Cowgill 2018-01-26 16:57:47 UTC
From:
https://bugs.debian.org/888478

LD 2.30 seems to mishandle libcalls into libgcc when LTO is enabled.

Testcase for 64-bit architectures (test.c):
volatile  __int128 a = 42;
volatile  __int128 b = 1;

int main(void)
{
        return (int) (a / b);
}

For 32-bit:
volatile unsigned long long a = 42;
volatile unsigned long long b = 1;

int main(void)
{
        return (int) (a / b);
}

Compile with:
gcc -flto -O2 -c test.c
ar rcs libtest.a test.o
gcc -flto -Wl,--whole-archive libtest.a -Wl,--no-whole-archive

[The --whole-archive is not needed in general, but is for this test case]

On x86_64 this gives:
/usr/bin/ld: /tmp/cc8wkDcQ.ltrans0.ltrans.o(.text.startup+0x21): unresolvable R_X86_64_PLT32 relocation against symbol `__divti3'
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status

On i386:
/usr/bin/ld: /tmp/cc5ZOEfW.ltrans0.ltrans.o(.text.startup+0x3d): unresolvable R_386_PLT32 relocation against symbol `__udivdi3'
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status

On mipsel (o32 little endian):
/usr/bin/ld: BFD (GNU Binutils for Debian) 2.29.90.20180122 assertion fail ../../bfd/elflink.c:9757
collect2: error: ld returned 1 exit status

I managed to bisect this (on x86_64) to:
1fa4ec6ae707402c6b61cde33cfe4bdeafd53f82 is the first bad commit
commit 1fa4ec6ae707402c6b61cde33cfe4bdeafd53f82
Author: Alan Modra <amodra@gmail.com>
Date:   Sat Sep 2 11:08:05 2017 +0930

    LTO rescan archives
    
    ld ought to be more clever about where it puts LTO recompiled objects.
    Ideally the recompiled objects ought to be ordered to the same place
    their IR objects were, and files extracted from archives on the second
    pass ought to go in the same place as they would if extracted on the
    first pass.  This patch addresses the archive problem.  Without this
    fix, objects extracted from archives might be placed after the crt
    files intended to go at the end of an executable or shared library,
    possibly causing exception handling failures.
    
            * ldlang.h (lang_input_statement_type): Expand comments.
            (LANG_FOR_EACH_INPUT_STATEMENT): Rewrite without casts.
            * ldlang.c (lang_for_each_input_file): Likewise.
            (load_symbols): Set usrdata for archives.
            (find_rescan_insertion): New function.
            (lang_process): Trim off and reinsert entries added to file chain
            when rescanning archives for LTO.
            * ldmain.c (add_archive_element): Set my_archive input_statement
            next pointer to last element added.

:040000 040000 933c913523934e7984e5f291b407014374cc5aa8 41866e5eae1ab88e83ba47561d9dbbede3ff4231 M	ld
Comment 1 H.J. Lu 2018-01-26 21:41:16 UTC
The problem is

--whole-archive libx.a --no-whole-archive -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed 

ld rescan order is wrong.  It rescans -lgcc_s before x.ltrans0.ltrans.o
from libx.a.
Comment 2 Alan Modra 2018-01-26 22:25:42 UTC
Uggh.  I guess we need to implement the comment I made.  "Ideally the recompiled objects ought to be ordered to the same place their IR objects were."
Comment 3 Alan Modra 2018-01-27 04:10:47 UTC
Reverting 1fa4ec6ae7 makes no difference.  Indeed, now that I've refreshed my memory on what that patch does, it should not since all it is doing (or at least supposed to be doing!) is reordering the layout of object files.  Opening objects, adding their symbols and deciding whether they need including happens before ld gets to the new 1fa4ec6ae7 code..
Comment 4 Alan Modra 2018-01-27 13:01:41 UTC
Sigh, so it seems reverting the patch does fix the problem.  I don't know what happened when I attempted the revert earlier today..

Now that I've had a bit more time to look at the problem, it seems the patch is causing _divdi3.o extracted from libgcc.a to not be on the file_chain list.  This is the list used by LANG_FOR_EACH_INPUT_STATEMENT in output_section_callback, so the .text section from that file is not seen by map_input_to_output_sections and thus has no output section.  Symbols defined in a section without output section are unresolvable (elf-bfd.h:RELOC_FOR_GLOBAL_SYMBOL).
Comment 5 Sourceware Commits 2018-01-27 14:54:47 UTC
The binutils-2_30-branch branch has been updated by Nick Clifton <nickc@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=cad6449ec5efdb8cf7adb3f44e67712effebb05f

commit cad6449ec5efdb8cf7adb3f44e67712effebb05f
Author: Nick Clifton <nickc@redhat.com>
Date:   Sat Jan 27 14:53:19 2018 +0000

    Revert the optimization of the placement of LTO objects as a temporary solution for PR 22751.
    
    	PR 22751
    	Revert this change as a temporary solution for this PR:
    
    	2017-09-02  Alan Modra  <amodra@gmail.com>
    
    	* ldlang.h (lang_input_statement_type): Expand comments.
    	(LANG_FOR_EACH_INPUT_STATEMENT): Rewrite without casts.
    	* ldlang.c (lang_for_each_input_file): Likewise.
    	(load_symbols): Set usrdata for archives.
    	(find_rescan_insertion): New function.
    	(lang_process): Trim off and reinsert entries added to file chain
    	when rescanning archives for LTO.
    	* ldmain.c (add_archive_element): Set my_archive input_statement
    	next pointer to last element added.
Comment 6 Sourceware Commits 2018-01-27 22:34:40 UTC
The master branch has been updated by Alan Modra <amodra@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=d580dcc7aac21dc8396e8e90ea6d05ec32d6cbb9

commit d580dcc7aac21dc8396e8e90ea6d05ec32d6cbb9
Author: Alan Modra <amodra@gmail.com>
Date:   Sun Jan 28 08:03:26 2018 +1030

    PR22751, LTO broken for libgcc libcalls
    
    So what was happening was that the file added from libgcc.a during the
    rescan was not put on file_chain.  map_input_to_output_sections then
    doesn't see the file and its sections are treated as discarded.
    
    The file_chain list pointer bug was caused by that fact that an
    archive element claimed by the plugin does not have my_archive set.
    Or more correctly, the actual archive element does have my_archive
    set, but this bfd is replaced with a dummy bfd that doesn't have
    my_archive set.
    
    	PR 22751
    	* ldlang.c (find_rescan_insertion): Look past bfds with claim_archive
    	set.
Comment 7 Sourceware Commits 2018-01-28 00:05:25 UTC
The master branch has been updated by H.J. Lu <hjl@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=fef75122c0fe3abafb99d79a63545c1531f4107c

commit fef75122c0fe3abafb99d79a63545c1531f4107c
Author: H.J. Lu <hjl.tools@gmail.com>
Date:   Sat Jan 27 16:04:34 2018 -0800

    Add a testcase for PR ld/22751
    
    Since dummy.o must be placed before
    
    -Wl,--whole-archive tmpdir/pr22751.a -Wl,--no-whole-archive
    
    to trigger the bug, this patch adds an optional trailing ld options to
    run_ld_link_exec_tests.
    
    	PR ld/22751
    	* testsuite/config/default.exp (INT128_CFLAGS): New.
    	* testsuite/ld-plugin/lto.exp (INT128_CFLAGS): New.
    	Run ld/22751 tests.
    	* testsuite/ld-plugin/pr22751.c: New file.
    	* testsuite/lib/ld-lib.exp (run_ld_link_exec_tests): Add
    	ld trailing options.
Comment 8 Alan Modra 2018-01-28 00:50:31 UTC
Fixed