Bug 23869 - gem5.debug link with -fuse-ld=gold fails with /usr/bin/ld.gold: internal error in read_cie, at ../../gold/ehframe.cc:943 if partial link is done initially with ld and final link with gold
Summary: gem5.debug link with -fuse-ld=gold fails with /usr/bin/ld.gold: internal erro...
Status: UNCONFIRMED
Alias: None
Product: binutils
Classification: Unclassified
Component: gold (show other bugs)
Version: 2.32
: P2 normal
Target Milestone: ---
Assignee: Cary Coutant
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-11-07 07:28 UTC by Ciro Santilli
Modified: 2024-02-28 22:32 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ciro Santilli 2018-11-07 07:28:34 UTC
Reproducible 18.04 amd64 with GCC 7.3.0 and Binutils 2.30 from the repos.

Steps to reproduce:

    git clone https://gem5.googlesource.com/public/gem5
    cd gem5
    git checkout d4b3e064adeeace3c3e7d106801f95c14637c12f
    scons -j16 --ignore-style --verbose build/ARM/gem5.debug |& tee build.log

Build takes about 10 miutes. Note that this is not cross compiling for ARM, ARM just shows in the command beause this is a system simulator.

Now open build.log and find the final link command near the very end, the one that generates the `-o .*gem5.debug` executable.

Put that command alone on a shell script file by itself, and add to it:

    -fuse-ld=gold

before the object files, and then the link fails with:

    /usr/bin/ld.gold: internal error in read_cie, at ../../gold/ehframe.cc:943

A quick look at the sources shows that the line is:

    gold_assert(r == 1);

in `Eh_frame::read_cie`.

Interestingly, if you do the same procedure but on a non-debug build:

    scons -j16 --verbose build/ARM/gem5.opt |& tee build.log

then it works with `-fuse-ld=gold`, and reduces the link time by half, which is awesome.

The only difference I can see there is that the build was done with `-O0` (including for object files of course) instead of `-O3`.

Modifying the link command to use `g++-8` GCC 8.2.0 from the repos made no difference.

Checking file sizes with `If ls -hlrS` shows that in both builds, there is one huge ~1Gb file, and several other much smaller Mb sized files.

The large file contains a lot of pybind11 boilerplate, the project is written in C++.

The final failing link command looks like this:

    #!/usr/bin/env bash
    g++ \
    -fuse-ld=gold \
    -o \
    build/ARM/gem5.debug \
    -pthread \
    -L/usr/lib/python2.7/config-x86_64-linux-gnu \
    -L/usr/lib \
    -Xlinker \
    -export-dynamic \
    -Wl,-O1 \
    -Wl,-Bsymbolic-functions \
    -O0 \
    build/ARM/sim/main.do \
    build/ARM/systemc/dt/fx/lib.do.partial \
    build/ARM/sim/probe/lib.do.partial \
    build/ARM/cpu/testers/traffic_gen/lib.do.partial \
    build/ARM/mem/ruby/filters/lib.do.partial \
    build/ARM/mem/ruby/slicc_interface/lib.do.partial \
    build/ARM/systemc/utils/lib.do.partial \
    build/ARM/cpu/testers/memtest/lib.do.partial \
    build/ARM/systemc/dt/misc/lib.do.partial \
    build/ARM/systemc/dt/int/lib.do.partial \
    build/ARM/mem/cache/lib.do.partial \
    build/ARM/mem/cache/replacement_policies/lib.do.partial \
    build/ARM/mem/ruby/network/simple/lib.do.partial \
    build/ARM/sim/lib.do.partial \
    build/ARM/mem/ruby/structures/lib.do.partial \
    build/ARM/cpu/lib.do.partial \
    build/ARM/learning_gem5/part2/lib.do.partial \
    build/ARM/systemc/dt/bit/lib.do.partial \
    build/ARM/dev/arm/lib.do.partial \
    build/ARM/base/lib.do.partial \
    build/ARM/kern/lib.do.partial \
    build/ARM/base/vnc/lib.do.partial \
    build/ARM/sim/power/lib.do.partial \
    build/ARM/unittest/lib.do.partial \
    build/ARM/mem/probes/lib.do.partial \
    build/ARM/cpu/simple/probes/lib.do.partial \
    build/ARM/mem/ruby/common/lib.do.partial \
    build/ARM/mem/ruby/network/fault_model/lib.do.partial \
    build/ARM/mem/cache/tags/indexing_policies/lib.do.partial \
    build/ARM/mem/qos/lib.do.partial \
    build/ARM/arch/mips/lib.do.partial \
    build/ARM/mem/ruby/network/garnet2.0/lib.do.partial \
    build/ARM/cpu/pred/lib.do.partial \
    build/ARM/mem/ruby/network/lib.do.partial \
    build/ARM/dev/net/lib.do.partial \
    build/ARM/dev/i2c/lib.do.partial \
    build/ARM/mem/cache/prefetch/lib.do.partial \
    build/ARM/mem/lib.do.partial \
    build/ARM/mem/protocol/lib.do.partial \
    build/ARM/dev/pci/lib.do.partial \
    build/ARM/cpu/testers/directedtest/lib.do.partial \
    build/ARM/dev/ps2/lib.do.partial \
    build/ARM/dev/lib.do.partial \
    build/ARM/arch/generic/lib.do.partial \
    build/ARM/cpu/o3/lib.do.partial \
    build/ARM/mem/cache/tags/lib.do.partial \
    build/ARM/arch/arm/tracers/lib.do.partial \
    build/ARM/dev/serial/lib.do.partial \
    build/ARM/cpu/simple/lib.do.partial \
    build/ARM/systemc/core/lib.do.partial \
    build/ARM/dev/storage/lib.do.partial \
    build/ARM/proto/lib.do.partial \
    build/ARM/python/lib.do.partial \
    build/ARM/systemc/dt/lib.do.partial \
    build/ARM/mem/ruby/profiler/lib.do.partial \
    build/ARM/cpu/o3/probe/lib.do.partial \
    build/ARM/arch/arm/lib.do.partial \
    build/ARM/cpu/trace/lib.do.partial \
    build/ARM/dev/virtio/lib.do.partial \
    build/ARM/cpu/minor/lib.do.partial \
    build/ARM/mem/ruby/system/lib.do.partial \
    build/ARM/cpu/testers/garnet_synthetic_traffic/lib.do.partial \
    build/ARM/cpu/testers/rubytest/lib.do.partial \
    build/ARM/systemc/channel/lib.do.partial \
    build/ARM/base/date.do \
    -Lbuild/libelf \
    -Lbuild/iostream3 \
    -Lbuild/nomali \
    -Lbuild/drampower \
    -Lbuild/fputils \
    -Lbuild/libfdt \
    -Lbuild/googletest \
    -lpython2.7 \
    -lpthread \
    -ldl \
    -lutil \
    -lm \
    -lz \
    -lprotobuf \
    -lrt \
    -ltcmalloc \
    -lfdt \
    -lfputils \
    -ldrampower \
    -lnomali \
    -liostream3 \
    -lelf \
    -lpng \
    ;

If I also add `-v` to the GCC command the output is:

    Using built-in specs.
    COLLECT_GCC=g++
    COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper
    OFFLOAD_TARGET_NAMES=nvptx-none
    OFFLOAD_TARGET_DEFAULT=1
    Target: x86_64-linux-gnu
    Configured with: ../src/configure -v --with-pkgversion='Ubuntu 7.3.0-27ubuntu1~18.04' --with-bugurl=file:///usr/share/doc/gcc-7/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-7 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
    Thread model: posix
    gcc version 7.3.0 (Ubuntu 7.3.0-27ubuntu1~18.04) 
    COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/7/:/usr/lib/gcc/x86_64-linux-gnu/7/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/7/:/usr/lib/gcc/x86_64-linux-gnu/
    LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/7/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../../lib/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../:/lib/:/usr/lib/
    COLLECT_GCC_OPTIONS='-v' '-fuse-ld=gold' '-o' 'build/ARM/gem5.debug' '-pthread' '-L/usr/lib/python2.7/config-x86_64-linux-gnu' '-L/usr/lib' '-O0' '-Lbuild/libelf' '-Lbuild/iostream3' '-Lbuild/nomali' '-Lbuild/drampower' '-Lbuild/fputils' '-Lbuild/libfdt' '-Lbuild/googletest' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
    /usr/lib/gcc/x86_64-linux-gnu/7/collect2 -plugin /usr/lib/gcc/x86_64-linux-gnu/7/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper -plugin-opt=-fresolution=/tmp/ccfPjadk.res -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lpthread -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc --sysroot=/ --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie -z now -fuse-ld=gold -z relro -o build/ARM/gem5.debug /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o -L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib -Lbuild/libelf -Lbuild/iostream3 -Lbuild/nomali -Lbuild/drampower -Lbuild/fputils -Lbuild/libfdt -Lbuild/googletest -L/usr/lib/gcc/x86_64-linux-gnu/7 -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/7/../../.. -export-dynamic -O1 -Bsymbolic-functions build/ARM/sim/main.do build/ARM/systemc/dt/fx/lib.do.partial build/ARM/sim/probe/lib.do.partial build/ARM/cpu/testers/traffic_gen/lib.do.partial build/ARM/mem/ruby/filters/lib.do.partial build/ARM/mem/ruby/slicc_interface/lib.do.partial build/ARM/systemc/utils/lib.do.partial build/ARM/cpu/testers/memtest/lib.do.partial build/ARM/systemc/dt/misc/lib.do.partial build/ARM/systemc/dt/int/lib.do.partial build/ARM/mem/cache/lib.do.partial build/ARM/mem/cache/replacement_policies/lib.do.partial build/ARM/mem/ruby/network/simple/lib.do.partial build/ARM/sim/lib.do.partial build/ARM/mem/ruby/structures/lib.do.partial build/ARM/cpu/lib.do.partial build/ARM/learning_gem5/part2/lib.do.partial build/ARM/systemc/dt/bit/lib.do.partial build/ARM/dev/arm/lib.do.partial build/ARM/base/lib.do.partial build/ARM/kern/lib.do.partial build/ARM/base/vnc/lib.do.partial build/ARM/sim/power/lib.do.partial build/ARM/unittest/lib.do.partial build/ARM/mem/probes/lib.do.partial build/ARM/cpu/simple/probes/lib.do.partial build/ARM/mem/ruby/common/lib.do.partial build/ARM/mem/ruby/network/fault_model/lib.do.partial build/ARM/mem/cache/tags/indexing_policies/lib.do.partial build/ARM/mem/qos/lib.do.partial build/ARM/arch/mips/lib.do.partial build/ARM/mem/ruby/network/garnet2.0/lib.do.partial build/ARM/cpu/pred/lib.do.partial build/ARM/mem/ruby/network/lib.do.partial build/ARM/dev/net/lib.do.partial build/ARM/dev/i2c/lib.do.partial build/ARM/mem/cache/prefetch/lib.do.partial build/ARM/mem/lib.do.partial build/ARM/mem/protocol/lib.do.partial build/ARM/dev/pci/lib.do.partial build/ARM/cpu/testers/directedtest/lib.do.partial build/ARM/dev/ps2/lib.do.partial build/ARM/dev/lib.do.partial build/ARM/arch/generic/lib.do.partial build/ARM/cpu/o3/lib.do.partial build/ARM/mem/cache/tags/lib.do.partial build/ARM/arch/arm/tracers/lib.do.partial build/ARM/dev/serial/lib.do.partial build/ARM/cpu/simple/lib.do.partial build/ARM/systemc/core/lib.do.partial build/ARM/dev/storage/lib.do.partial build/ARM/proto/lib.do.partial build/ARM/python/lib.do.partial build/ARM/systemc/dt/lib.do.partial build/ARM/mem/ruby/profiler/lib.do.partial build/ARM/cpu/o3/probe/lib.do.partial build/ARM/arch/arm/lib.do.partial build/ARM/cpu/trace/lib.do.partial build/ARM/dev/virtio/lib.do.partial build/ARM/cpu/minor/lib.do.partial build/ARM/mem/ruby/system/lib.do.partial build/ARM/cpu/testers/garnet_synthetic_traffic/lib.do.partial build/ARM/cpu/testers/rubytest/lib.do.partial build/ARM/systemc/channel/lib.do.partial build/ARM/base/date.do -lpython2.7 -lpthread -ldl -lutil -lz -lprotobuf -lrt -ltcmalloc -lfdt -lfputils -ldrampower -lnomali -liostream3 -lelf -lpng -lstdc++ -lm -lgcc_s -lgcc -lpthread -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o
    /usr/bin/ld.gold: internal error in read_cie, at ../../gold/ehframe.cc:943
    collect2: error: ld returned 1 exit status

The zipped build files are 789M, let me know if you want them and how I can provide them.

Let me know if you need any further details and I'll provide, I'm really eager to profit from the shorter link time of this awesome linker!

Then I also reproduced on master Binutils:

    git checkout 8f531a8d1f9b9b9061368ddd89d1b7e7aafe9e38
    ./configure --enable-gold
    make -j`nproc`

by replacing the `/usr/lib/gcc/x86_64-linux-gnu/7/collect2` with the path to `gold/ld-new`.
Comment 1 Ciro Santilli 2018-11-07 12:41:43 UTC
Ah, the project uses partial linking, I was unknowingly linking the first half with ld and trying the final link with gold, so I'm guessing that's not supported?

If not, please feel free to close this ticket.

Interesting that it fails only with the debug symbols though.

I did the whole build with gold now and it works, and cuts the link time by 4x. You are awesome.