Hello, I suspect the changes below introduced a regression in the linker for mips architecture, now we can't build ghc anymore in Debian +2017-02-13 Maciej W. Rozycki <macro@imgtec.com> + + * elfxx-mips.c (mips_elf_hash_sort_data): Add + `max_local_dynindx'. + (mips_elf_sort_hash_table): Handle it. + (mips_elf_sort_hash_table_f) <GGA_NONE>: For forced local + symbols bump up `max_local_dynindx' rather than + `max_non_got_dynindx'. + +2017-02-13 Maciej W. Rozycki <macro@imgtec.com> + + * elfxx-mips.c (mips_elf_hash_sort_data): Convert the + `min_got_dynindx', `max_unref_got_dynindx' and + `max_non_got_dynindx' members to the `bfd_size_type' data type. + (mips_elf_sort_hash_table): Adjust accordingly. this happens on mips, mipsel, and mips64el You can see build logs here https://buildd.debian.org/status/fetch.php?pkg=ghc&arch=mips&ver=8.0.1-17%2Bb1&stamp=1488954233&raw=0 https://buildd.debian.org/status/package.php?p=ghc&suite=unstable I hope I can avoid to have to recompile the whole ghc package in a porterbox, but in case, please ask and I'll provide whatever can help in debug this issue thanks
Reverting e7ec0c47c5500b572b847cddd5b0868ef3784473 fixes "... 2.28 assertion fail ../../bfd/elfxx-mips.c:3860" (note that the assertion failure is on the prior line) for me. I encountered this while building a cross compiler for mips on an x86_64 host; building binutils 2.28 without reverting that commit caused the failure when trying to run the configure tests for libgomp in the full cross gcc. I did not try reverting the earlier commit.
This appears to be strongly related to PR/20828. I managed to reduce it down to this: $ cat alpha-lib.c void alpha(void) {} $ cat main.c void alpha(void); void beta(void) { alpha(); } $ gcc -shared alpha-lib.c -o alpha-lib.so $ gcc -c main.c $ ld -o broken --gc-sections main.o -u alpha alpha-lib.so Using an old binutils (2.27.90.20170124) without the fix for PR/20828 produces an invalid dyn-sym table: Symbol table '.dynsym' contains 10 entries: Num: Value Size Type Bind Vis Ndx Name 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND 1: 0000000000000001 0 SECTION GLOBAL DEFAULT ABS _DYNAMIC_LINKING 2: 0000000120010518 0 NOTYPE GLOBAL DEFAULT 8 _edata 3: 00000001200104f0 0 NOTYPE GLOBAL DEFAULT 7 _fdata 4: 00000001200104f0 0 OBJECT GLOBAL DEFAULT 7 __RLD_MAP 5: 0000000120010518 0 NOTYPE GLOBAL DEFAULT 8 _end 6: 0000000120010518 0 NOTYPE GLOBAL DEFAULT 8 __bss_start 7: 00000001200004db 0 NOTYPE GLOBAL DEFAULT 6 _ftext 8: 0000000120010518 0 NOTYPE GLOBAL DEFAULT 8 _fbss 9: 0000000000000000 0 FUNC LOCAL DEFAULT UND alpha Using 2.28 crashes with the aforementioned assertion fail: ld: BFD (GNU Binutils for Debian) 2.28 assertion fail ../../bfd/elfxx-mips.c:3861
James, Your reduced test case seems the best hope right now, however I cannot reproduce the problem with the 2.28 branch using the recipe provided. The cause might be a difference in binaries produced originating from a different compiler version for example, so can you please supply the exact objects (i.e. main.o and alpha-lib.so) you have that trigger the assertion? Maciej
Created attachment 9922 [details] main.o built with GCC 6
Created attachment 9923 [details] alpha-lib.so built with GCC 6
I've attached the built objects. They were built with Debian's mips cross compiler: $ mipsel-linux-gnu-gcc -v Using built-in specs. COLLECT_GCC=mipsel-linux-gnu-gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc-cross/mipsel-linux-gnu/6/lto-wrapper Target: mipsel-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Debian 6.3.0-8' --with-bugurl=file:///usr/share/doc/gcc-6/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-6 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libitm --disable-libsanitizer --disable-libquadmath --enable-plugin --enable-default-pie --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-6-mipsel-cross/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-6-mipsel-cross --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-6-mipsel-cross --with-arch-directory=mips --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libgcj --enable-multiarch --enable-multilib --with-arch-32=mips32r2 --with-fp-32=xx --with-madd4=no --with-lxc1-sxc1=no --enable-targets=all --with-arch-64=mips64r2 --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=mipsel-linux-gnu --program-prefix=mipsel-linux-gnu- --includedir=/usr/mipsel-linux-gnu/include Thread model: posix gcc version 6.3.0 20170221 (Debian 6.3.0-8)
I think I see the problem, you need to pass -fPIE when building main.o to trigger the bug.
Thanks for the update, I can reproduce it now -- with -fPIE, or any other options causing a SVR4 rather than PLT binary to be produced (I can see now from your `readelf' dump rather than options used, that you originally chose an n64 binary to be made and those are normally SVR4).
Created attachment 9925 [details] WIP bug fix This is indeed another incorrectly handled scenario that would have qualified under PR ld/20828. Here we have a symbol which has been forced local, however it has retained its GGA_NORMAL GOT assignment. Consequently it is considered external by the MIPS backend and assigned to the global part of both GOT and the dynamic symbol table. However the generic ELF linker has already included it in the local symbol count and as a result a mismatch happens in the MIPS backend between the number of symbols expected and actually assigned to the non-GOT-mapped and GOT-mapped parts of the dynamic symbol table, which is caught by the: BFD_ASSERT (hsd.max_non_got_dynindx <= hsd.min_got_dynindx) assertion, saving the linker's skin as with this assertion removed a broken binary (with the offending symbol missing due to a wrong dynamic symbol index assignment) would be produced: Symbol table '.dynsym' contains 10 entries: Num: Value Size Type Bind Vis Ndx Name 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND 1: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND 2: 00000001200104f0 0 NOTYPE GLOBAL DEFAULT 7 _fdata 3: 0000000000000001 0 SECTION GLOBAL DEFAULT ABS _DYNAMIC_LINKING 4: 00000001200004db 0 NOTYPE GLOBAL DEFAULT 6 _ftext 5: 00000001200104f0 0 OBJECT GLOBAL DEFAULT 7 __RLD_MAP 6: 0000000120010518 0 NOTYPE GLOBAL DEFAULT 8 __bss_start 7: 0000000120010518 0 NOTYPE GLOBAL DEFAULT 8 _edata 8: 0000000120010518 0 NOTYPE GLOBAL DEFAULT 8 _end 9: 0000000120010518 0 NOTYPE GLOBAL DEFAULT 8 _fbss -- notice the bad entry at 1, where `alpha' should be if considered local. Here is another similarity though to what has already been covered in PR ld/20828 in that I think with `alpha' chosen with `-u' (or, for that matter `--require-defined') or the equivalent EXTERN linker script command the symbol shouldn't have been garbage-collected in the first place, as this defeats the purpose of these constructs. So I believe that by these contstructs' definition symbols thus created have to be retained and therefore remain external. That has led me to the patch proposed, which obviously fixes James's reduced test case. Unfortunately I am unable to reproduce the libgomp's configuration failure and trying to build ghc is out of question too, so I do not know if they are cases of `-u'/`--require-defined'/EXTERN or maybe there is yet another way for forced local GGA_NORMAL symbols to make it through linking. So, Gianfranco and Alastair, would you please try the patch proposed and see if it fixes the issue for you? If so, then I'll proceed with creating suitable linker test suite cases and following the usual procedure to apply the fix to binutils sources, or at least trunk where it also triggers. For 2.28, like with PR ld/20828, I may instead choose to limit the solution to the MIPS backend, and only use a workaround where forced local symbols are made GGA_NONE, which I'm leaning towards doing anyway.
The fix is confirmed, after two days ghc finally is finishing the build! (the part where the link was failing is succeeding now, and the build is ending in a few hours). So, confirming that now it works.
thanks you all!
The WIP bug fix patch does not fix the problem I had. The assertion failure was on the prior line, though, so it could be a different issue. What can I do to help?
Alastair, was the issue you got introduced by the same commit? You might want to understand when the regression started, and open a new issue in case. Fixing two different bugs in a single bug number is something not really trivial to follow
Alastair, preferably I'll need all the objects (static and dynamic) and linker scripts used in the link, and the `ld' invocation line, to be able to reproduce the problem. You can ask GCC to show the invocation line with the `-v' option; the line will be buried in the output produced as `collect2' invocation. If making the objects available is not possible, then I'll have to think how to reduce it or I might ask you to try diagnostic patches, though given how the circumstances of these failures are subtle this could make tracking down the cause of the issue really tough. This particular assertion failure you have observed means that a local dynamic symbol was somehow missed by the generic ELF linker in counting and the MIPS backend chokes as it breaks its symbol index management. Output from `readelf --dyn-syms' with the binary produced with commit e7ec0c47c550 reverted might give a clue though. Also can you check if this happens on trunk too? And yes, this will have to be a separate PR.
James Cowgill CC'd me for PR ld/21334, which appears to be the issue I had encountered - I'll follow up there.
The master branch has been updated by Maciej W. Rozycki <macro@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=80070c0d3491347f11283c5791b9dd040fedbd4f commit 80070c0d3491347f11283c5791b9dd040fedbd4f Author: Maciej W. Rozycki <macro@imgtec.com> Date: Mon Mar 27 12:39:07 2017 +0100 PR ld/21233: Avoid sweeping forced-undefined symbols in section GC Complement commit 902e9fc76a0e ("PR ld/20828: Move symbol version processing ahead of GC symbol sweep"), commit b531344c34b0 ("PR ld/20828: Reorder the symbol sweep stage of section GC") and commit 81ff47b3a546 ("PR ld/20828: Fix linker script symbols wrongly forced local with section GC"), and prevent symbols forcibly entered in the output file with the use of the `--undefined=' or `--require-defined=' linker command line options or the EXTERN linker script command from being swept in section garbage collection and consequently recorded in the dynamic symbol table as local entries. This happens in certain circumstances, where a symbol reference also exists in one of the static input files, however only in a section which is garbage-collected and does not make it to the output file, and the symbol is defined in a dynamic object present in the link. For example with the `i386-linux' target and the `pr21233.s' and `pr21233-l.s' sources, and the `pr21233.ld' linker script included with this change we get: $ as -o pr21233-l.o pr21233-l.s $ ld -shared -T pr21233.ld -o libpr21233.so pr21233-l.o $ as -o pr21233.o pr21233.s $ ld --gc-sections -e foo --require-defined=bar -T pr21233.ld -o pr21233 pr21233.o libpr21233.so $ readelf --dyn-syms pr21233 Symbol table '.dynsym' contains 2 entries: Num: Value Size Type Bind Vis Ndx Name 0: 00000000 0 NOTYPE LOCAL DEFAULT UND 1: 00000000 0 OBJECT LOCAL DEFAULT UND bar $ which makes the run-time `bar' dependency of the `pr21233' executable different from its corresponding link-time dependency, i.e. the presence of `libpr21233.so' and its `bar' symbol is required at the link time, however at the run time a copy of `libpr21233.so' without `bar' will do. Similarly with `--undefined=' and EXTERN which do not actually require the reference to the symbol requested to be satisfied with a definition at the link time, however once the definition has been pulled at the link time, so it should at the dynamic load time. Additionally with the `mips-linux' target we get: $ ld --gc-sections -e foo --require-defined=bar -T pr21233.ld -o pr21233 pr21233.o libpr21233.so ld: BFD (GNU Binutils) 2.28.51.20170324 assertion fail .../bfd/elfxx-mips.c:3861 $ as the target is not prepared to handle such a local dynamic symbol. With this change in effect we get: $ readelf --dyn-syms pr21233 Symbol table '.dynsym' contains 2 entries: Num: Value Size Type Bind Vis Ndx Name 0: 00000000 0 NOTYPE LOCAL DEFAULT UND 1: 00000000 0 OBJECT GLOBAL DEFAULT UND bar $ instead, for both targets. ld/ PR ld/21233 * ldlang.c (insert_undefined): Set `mark' for ELF symbols. * testsuite/ld-elf/pr21233.sd: New test. * testsuite/ld-elf/pr21233-l.sd: New test. * testsuite/ld-elf/pr21233.ld: New test linker script. * testsuite/ld-elf/pr21233-e.ld: New test linker script. * testsuite/ld-elf/pr21233.s: New test source. * testsuite/ld-elf/pr21233-l.s: New test source. * testsuite/ld-elf/shared.exp: Run the new tests.
The master branch has been updated by Hans-Peter Nilsson <hp@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=4d5efb852be6554305dfaf17c1e4f07bb9514273 commit 4d5efb852be6554305dfaf17c1e4f07bb9514273 Author: Hans-Peter Nilsson <hp@bitrange.com> Date: Wed Apr 5 03:00:40 2017 +0200 PR ld/21233: xfail failing test-parts for cris*-*-*
The binutils-2_28-branch branch has been updated by Maciej W. Rozycki <macro@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=beefa9cd657b7187659ba455b704b4be3c28ab17 commit beefa9cd657b7187659ba455b704b4be3c28ab17 Author: Maciej W. Rozycki <macro@imgtec.com> Date: Mon Mar 27 12:39:07 2017 +0100 PR ld/21233: Avoid sweeping forced-undefined symbols in section GC Complement commit 902e9fc76a0e ("PR ld/20828: Move symbol version processing ahead of GC symbol sweep"), commit b531344c34b0 ("PR ld/20828: Reorder the symbol sweep stage of section GC") and commit 81ff47b3a546 ("PR ld/20828: Fix linker script symbols wrongly forced local with section GC"), and prevent symbols forcibly entered in the output file with the use of the `--undefined=' or `--require-defined=' linker command line options or the EXTERN linker script command from being swept in section garbage collection and consequently recorded in the dynamic symbol table as local entries. This happens in certain circumstances, where a symbol reference also exists in one of the static input files, however only in a section which is garbage-collected and does not make it to the output file, and the symbol is defined in a dynamic object present in the link. For example with the `i386-linux' target and the `pr21233.s' and `pr21233-l.s' sources, and the `pr21233.ld' linker script included with this change we get: $ as -o pr21233-l.o pr21233-l.s $ ld -shared -T pr21233.ld -o libpr21233.so pr21233-l.o $ as -o pr21233.o pr21233.s $ ld --gc-sections -e foo --require-defined=bar -T pr21233.ld -o pr21233 pr21233.o libpr21233.so $ readelf --dyn-syms pr21233 Symbol table '.dynsym' contains 2 entries: Num: Value Size Type Bind Vis Ndx Name 0: 00000000 0 NOTYPE LOCAL DEFAULT UND 1: 00000000 0 OBJECT LOCAL DEFAULT UND bar $ which makes the run-time `bar' dependency of the `pr21233' executable different from its corresponding link-time dependency, i.e. the presence of `libpr21233.so' and its `bar' symbol is required at the link time, however at the run time a copy of `libpr21233.so' without `bar' will do. Similarly with `--undefined=' and EXTERN which do not actually require the reference to the symbol requested to be satisfied with a definition at the link time, however once the definition has been pulled at the link time, so it should at the dynamic load time. Additionally with the `mips-linux' target we get: $ ld --gc-sections -e foo --require-defined=bar -T pr21233.ld -o pr21233 pr21233.o libpr21233.so ld: BFD (GNU Binutils) 2.28.51.20170324 assertion fail .../bfd/elfxx-mips.c:3861 $ as the target is not prepared to handle such a local dynamic symbol. With this change in effect we get: $ readelf --dyn-syms pr21233 Symbol table '.dynsym' contains 2 entries: Num: Value Size Type Bind Vis Ndx Name 0: 00000000 0 NOTYPE LOCAL DEFAULT UND 1: 00000000 0 OBJECT GLOBAL DEFAULT UND bar $ instead, for both targets. ld/ PR ld/21233 * ldlang.c (insert_undefined): Set `mark' for ELF symbols. (backported from commit 80070c0d3491347f11283c5791b9dd040fedbd4f)
Fixed on master and backported to 2.28, as per the automatic commit messages.
The master branch has been updated by Hans-Peter Nilsson <hp@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=6ca30fa5613cfa935834f27114d6db4f3854a13e commit 6ca30fa5613cfa935834f27114d6db4f3854a13e Author: Hans-Peter Nilsson <hp@bitrange.com> Date: Wed Apr 5 23:12:16 2017 +0200 PR ld/21233: change xfails to kfails, fix the indentation issue
The master branch has been updated by Hans-Peter Nilsson <hp@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=4e15b78be4bf008fb4f97bffbc0865dd0e4833ce commit 4e15b78be4bf008fb4f97bffbc0865dd0e4833ce Author: Hans-Peter Nilsson <hp@bitrange.com> Date: Sat Oct 21 19:46:22 2017 +0200 PR ld/21233 * testsuite/ld-elf/shared.exp: Remove kfails. The test-cases started passing with 5c3261b0e834647c, "ELF: Call check_relocs after opening all inputs". The lists could now be re-concatenated (see other run_ld_link_tests calls in shared.exp), but are for now left separate to simplify future kfail/xfailing.
The master branch has been updated by Alan Modra <amodra@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=18e404c4e2eb15443cc6dda93cbd38bdfeb17667 commit 18e404c4e2eb15443cc6dda93cbd38bdfeb17667 Author: Alan Modra <amodra@gmail.com> Date: Wed Apr 11 11:10:18 2018 +0930 Silence nds32 pic warnings Fixes: FAIL: Build pr22471a.so FAIL: Build pr22471b.so FAIL: Build pr22649-1.so FAIL: Build pr22649-2c.so FAIL: Build pr22649-2d.so FAIL: PR ld/20828 dynamic symbols with section GC (auxiliary shared library) FAIL: PR ld/20828 dynamic symbols with section GC (plain) FAIL: PR ld/20828 dynamic symbols with section GC (version script) FAIL: PR ld/20828 dynamic symbols with section GC (versioned shared library) FAIL: PR ld/20828 dynamic symbols with section GC (versioned) FAIL: PR ld/21233 dynamic symbols with section GC (auxiliary shared library) FAIL: Build pr22150.so FAIL: Build shared library for pr14170 FAIL: PR ld/21703 shared FAIL: Build shared library for broken linker script test FAIL: Build pr17068.so FAIL: -Bsymbolic-functions FAIL: Build pr20995.so FAIL: Build pr20995-2.so FAIL: Build pr22374 shared library * testsuite/ld-elf/shared.exp (AFLAGS_PIC): Add -mpic for nds32.
The master branch has been updated by Alan Modra <amodra@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=980a2e42f7439015defdcedad89a13a72749bdb0 commit 980a2e42f7439015defdcedad89a13a72749bdb0 Author: Alan Modra <amodra@gmail.com> Date: Thu Mar 21 08:39:18 2019 +1030 lm32-linux ld testsuite fails A number of the fails are due to ld supporting the creation of shared libraries but not allowing linking against them without using an option like -Bdynamic. FAIL: Symbol export class test (final shared object) FAIL: PROVIDE_HIDDEN test 4 FAIL: PROVIDE_HIDDEN test 6 FAIL: PROVIDE_HIDDEN test 10 FAIL: PROVIDE_HIDDEN test 12 FAIL: Build pr22471b.so FAIL: Build pr22649-2b.so FAIL: Build pr22649-2d.so FAIL: PR ld/20828 dynamic symbols with section GC (plain) FAIL: PR ld/20828 dynamic symbols with section GC (version script) FAIL: PR ld/20828 dynamic symbols with section GC (versioned) FAIL: PR ld/21233 dynamic symbols with section GC (--undefined) FAIL: PR ld/21233 dynamic symbols with section GC (--require-defined) FAIL: PR ld/21233 dynamic symbols with section GC (EXTERN) FAIL: Build pr22150 FAIL: PR ld/14170 FAIL: Link using broken linker script FAIL: pr17068 link --as-needed lib in group FAIL: ld-gc/pr20022 * emulparams/elf32lm32fd.sh (DYNAMIC_LINK): Undef.
The master branch has been updated by Alan Modra <amodra@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=260bcd09bfb98ebc5d8f0eb564edca21872e9f7f commit 260bcd09bfb98ebc5d8f0eb564edca21872e9f7f Author: Alan Modra <amodra@gmail.com> Date: Tue Dec 17 17:46:04 2019 +1030 Accept mips-sgi-irix output in a few ld tests mips-sgi-irix gas emits STT_OBJECT symbols where other assemblers would use STT_NOTYPE. See mips_frob_symbol in gas/config/tc-mips.c. Also, the section of some dynamic symbols is set to SHN_MIPS_TEXT or SHN_MIPS_DATA. See _bfd_mips_elf_finish_dynamic_symbol in bfd/elfxx-mips.c. These differences are visible in readelf output and cause some tests to fail for no other good reason. The patch fixes the following fails and removes an xfail. FAIL: ld-elf/pr23591 FAIL: PROVIDE_HIDDEN test (auxiliary shared object) FAIL: PR ld/21233 dynamic symbols with section GC (auxiliary shared library) * testsuite/ld-elf/pr21233-l.sd: Accept OBJECT for type and PRC for section of symbols. * testsuite/ld-elf/pr23591.d: Likewise. * testsuite/ld-elf/provide-hidden-s.nd: Likewise. * testsuite/ld-mips-elf/start.s: Make symbols function type. * testsuite/ld-mips-elf/hash2.d: Adjust. Don't xfail irix.