Bug 20006

Summary: Linking gcc and gold with ld.bfd with compressed debug sections fails on final close
Product: binutils Reporter: dilyan.palauzov <dilyan.palauzov>
Component: ldAssignee: H.J. Lu <hjl.tools>
Status: RESOLVED FIXED    
Severity: normal CC: hjl.tools
Priority: P2    
Version: 2.27   
Target Milestone: 2.27   
Host: Target:
Build: Last reconfirmed:

Description dilyan.palauzov@aegee.org 2016-04-26 21:18:38 UTC 
When I compile binutils-gdb.git(3e2e34f86) with "/git/binutils-gdb/configure --enable-plugins --with-system-zlib --enable-gold --with-expat --with-python --disable-tui --with-system-readline && make && make install" and then compile gcc.git(a9ad7efd6e2b255) with "./configure --enable-host-shared --enable-threads=posix --enable-targets=all --enable-nls --with-linker-hash-style=gnu --with-system-zlib --disable-multilib --enable-languages=c,c++,jit,lto && make && make install" everything is fine.

If I pass however to binutils/configure "--enable-compressed-debug-sections=all":

/git/binutils-gdb/configure --enable-plugins --with-system-zlib --enable-gold --with-expat --with-python --disable-tui --with-system-readline --enable-compressed-debug-sections=all && make && make install

and try to recompile gcc with the same options, I get at stage1:

make[3]: Leaving directory '/git/gcc/host-x86_64-pc-linux-gnu/libdecnumber'
make[3]: Entering directory '/git/gcc/host-x86_64-pc-linux-gnu/gcc'
g++ -std=gnu++98 -no-pie   -g -DIN_GCC -fPIC    -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wno-format -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common  -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc -Wl,-z,relro,-O1 -o cc1 c/c-lang.o c-family/stub-objc.o attribs.o c/c-errors.o c/c-decl.o c/c-typeck.o c/c-convert.o c/c-aux-info.o c/c-objc-common.o c/c-parser.o c/c-array-notation.o c/c-fold.o c-family/c-common.o c-family/c-cppbuiltin.o c-family/c-dump.o c-family/c-format.o c-family/c-gimplify.o c-family/c-indentation.o c-family/c-lex.o c-family/c-omp.o c-family/c-opts.o c-family/c-pch.o c-family/c-ppoutput.o c-family/c-pragma.o c-family/c-pretty-print.o c-family/c-semantics.o c-family/c-ada-spec.o c-family/c-cilkplus.o c-family/array-notation-common.o c-family/cilk.o c-family/c-ubsan.o i386-c.o glibc-c.o \
  cc1-checksum.o libbackend.a main.o libcommon-target.a libcommon.a ../libcpp/libcpp.a ../libdecnumber/libdecnumber.a libcommon.a ../libcpp/libcpp.a   ../libbacktrace/.libs/libbacktrace.a ../libiberty/pic/libiberty.a ../libdecnumber/libdecnumber.a  -lisl -lmpc -lmpfr -lgmp -rdynamic -ldl  -lz
cc1: final close failed: Invalid operation
collect2: error: ld returned 1 exit status
../.././gcc/c/Make-lang.in:71: recipe for target 'cc1' failed
make[3]: *** [cc1] Error 1
make[3]: Leaving directory '/git/gcc/host-x86_64-pc-linux-gnu/gcc'
Makefile:4391: recipe for target 'all-stage1-gcc' failed
make[2]: *** [all-stage1-gcc] Error 2
make[2]: Leaving directory '/git/gcc'
Makefile:20572: recipe for target 'stage1-bubble' failed
make[1]: *** [stage1-bubble] Error 2
make[1]: Leaving directory '/git/gcc'
Makefile:913: recipe for target 'all' failed
make: *** [all] Error 2

In fact, I cannot even link gold, when gas and ld.bfd are using compression:

make -k

make[4]: Leaving directory '/home/d/binutils/gold/testsuite'
make[4]: Entering directory '/home/d/binutils/gold'
g++ -W -Wall    -Wstack-usage=262144 -Werror -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -frandom-seed=dwp  -pipe -O3  -static-libstdc++ -static-libgcc -Wl,-z,relro,-O1 -o dwp dwp.o libgold.a ../libiberty/libiberty.a    -ldl -lz -ldl 
dwp: final close failed: Invalid operation
collect2: error: ld returned 1 exit status
Makefile:803: recipe for target 'dwp' failed
make[4]: *** [dwp] Error 1
g++ -W -Wall    -Wstack-usage=262144 -Werror -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -frandom-seed=ld-new  -pipe -O3  -static-libstdc++ -static-libgcc -Wl,-z,relro,-O1 -o ld-new main.o i386.o x86_64.o sparc.o powerpc.o arm.o arm-reloc-property.o tilegx.o mips.o aarch64.o aarch64-reloc-property.o s390.o libgold.a ../libiberty/libiberty.a    -ldl -lz -ldl 
ld-new: final close failed: Invalid operation
collect2: error: ld returned 1 exit status
Makefile:809: recipe for target 'ld-new' failed
make[4]: *** [ld-new] Error 1
g++ -W -Wall    -Wstack-usage=262144 -Werror -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -frandom-seed=incremental-dump  -pipe -O3  -static-libstdc++ -static-libgcc -Wl,-z,relro,-O1 -o incremental-dump incremental-dump.o i386.o x86_64.o sparc.o powerpc.o arm.o arm-reloc-property.o tilegx.o mips.o aarch64.o aarch64-reloc-property.o s390.o libgold.a ../libiberty/libiberty.a   -ldl -lz -ldl 
incremental-dump: final close failed: Invalid operation
collect2: error: ld returned 1 exit status
Makefile:806: recipe for target 'incremental-dump' failed
make[4]: *** [incremental-dump] Error 1
make[4]: Target 'all-am' not remade because of errors.
make[4]: Leaving directory '/home/d/binutils/gold'
Makefile:940: recipe for target 'all-recursive' failed
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory '/home/d/binutils/gold'
Makefile:691: recipe for target 'all' failed
make[2]: *** [all] Error 2
make[2]: Leaving directory '/home/d/binutils/gold'
Makefile:6080: recipe for target 'all-gold' failed
make[1]: *** [all-gold] Error 2
make[2]: Entering directory '/home/d/binutils/gprof'

For all compilations I use gcc 7.0.0 20160426 (a9ad7ef) and
export CFLAGS="-pipe -O3" CXXFLAGS="-pipe -O3" LDFLAGS="-Wl,-z,relro,-O1"
Comment 1 H.J. Lu 2016-04-27 12:22:47 UTC 
It was caused by

commit 4a539596f5d54d3116c5fdebd8be56998757288b
Author: H.J. Lu <hjl.tools@gmail.com>
Date:   Tue Feb 2 08:14:43 2016 -0800

    Store estimated istrances in compressed_size
    
    elf_x86_64_convert_load is very time consuming since it is called on
    each input section and has a loop over input text sections to estimate
    the branch distrance.  We can store the estimated distrances in the
    compressed_size field of the output section, which is only used to
    decompress the compressed input section.
    
    Before the patch, linking clang 3.9 takes 52 seconds.  After the patch,
    it only takes 2.5 seconds.
    
    	PR ld/19542
    	* elf64-x86-64.c (elf_x86_64_convert_load): Store the estimated
    	distrances in the compressed_size field of the output section.
Comment 2 Sourceware Commits 2016-04-27 16:16:09 UTC 
The master branch has been updated by H.J. Lu <hjl@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=91f8bf69a526912f86da81070407bba6a050e27f

commit 91f8bf69a526912f86da81070407bba6a050e27f
Author: H.J. Lu <hjl.tools@gmail.com>
Date:   Wed Apr 27 09:13:10 2016 -0700

    Skip debug sections when estimating distances
    
    Skip debug sections when estimating distances between output sections
    since compressed_size is used to compress debug sections and debug
    sections aren't excluded from distances between output sections.
    
    bfd/
    
    	PR ld/20006
    	* elf64-x86-64.c (elf_x86_64_convert_load): Skip debug sections
    	when estimating distances between output sections.
    
    ld/
    
    	PR ld/20006
    	* testsuite/ld-elfvsb/elfvsb.exp (COMPRESS_LDFLAG): New.
    	(visibility_run): Pass COMPRESS_LDFLAG to visibility_test on
    	ELF targets.
Comment 3 Sourceware Commits 2016-04-30 12:57:52 UTC 
The binutils-2_26-branch branch has been updated by H.J. Lu <hjl@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=805af9f6edbc8e1b1fa8fb6b30902decc5433bd0

commit 805af9f6edbc8e1b1fa8fb6b30902decc5433bd0
Author: H.J. Lu <hjl.tools@gmail.com>
Date:   Wed Apr 27 09:13:10 2016 -0700

    Skip debug sections when estimating distances
    
    Skip debug sections when estimating distances between output sections
    since compressed_size is used to compress debug sections and debug
    sections aren't excluded from distances between output sections.
    
    Backport from master
    
    bfd/
    
    	PR ld/20006
    	* elf64-x86-64.c (elf_x86_64_convert_load): Skip debug sections
    	when estimating distances between output sections.
    
    ld/
    
    	PR ld/20006
    	* testsuite/ld-elfvsb/elfvsb.exp (COMPRESS_LDFLAG): New.
    	(visibility_run): Pass COMPRESS_LDFLAG to visibility_test on
    	ELF targets.
Comment 4 H.J. Lu 2016-04-30 12:58:37 UTC 
Fixed for 2.27 and 2.26 branch.