For shared objects, all information needed for linking is in the dynamic array, and section headers are optional. However, gold looks up the information it needs in the (optional) section headers instead of the (mandatory) dynamic array, and fails if the section headers are not present. As a result, gold is unable to link to shared objects with no section headers, even though such shared objects are spec-compliant and the dynamic linker can load them without issue. This is unfortunate, as working around it requires redundantly respecifying most of the dynamic array in section headers. This seems like it should not be too hard to fix, from what I've seen of gold's shared object handling, but I'm not particularly familiar with the codebase so I don't know if there are things I'm missing. The BFD-based linker also exhibits this issue.
Please provide a testcase. A small binary file is OK.
Created attachment 12331 [details] testcase with no section headers This test case has no section headers; gold immediately errors due to attempting to read the nonexistent header for `.shstrtab`.
Created attachment 12332 [details] testcase with only shstrtab This testcase has a minimal section header table containing only `.shstrtab`. This works around the previous error, but linking against it fails anyway as gold only tries to find `.dynsym` through the section header table. Both test cases expose a single function, `run()`, that does nothing but return.
Linker consumes what linker generates. Since currently linker always generates section header, linker uses it unconditionally. We need a linker switch not to generate section header. Then other consumers in binutils, like ld, nm, objdump and readelf can be updated to support it.
Created attachment 12336 [details] patch to add flag to not emit section header tables I've written a small patch to add such a flag, does this look good to you?
(In reply to Kaylee from comment #5) > Created attachment 12336 [details] > patch to add flag to not emit section header tables > > I've written a small patch to add such a flag, does this look good to you? You need at least one run-time test. Can you also make a patch for ld? Ld patch will be reviewed much quicker.
Created attachment 12340 [details] patch to add flag to not emit section header tables Re-uploading with an email I'm less worried about getting scraped and spammed at.
Created attachment 12341 [details] patch to add a test for the flag This adds some testing for the flag.
Created attachment 12342 [details] patch to add BFD support for not emitting the section header table
Created attachment 12343 [details] patch to add flag to not emit the section header table to the BFD-based ld I'll add a testcase shortly.
(In reply to Kaylee from comment #10) > Created attachment 12343 [details] > patch to add flag to not emit the section header table to the BFD-based ld > > I'll add a testcase shortly. Thanks. I put your patches on users/hjl/pr25617/master branch at https://gitlab.com/x86-binutils/binutils-gdb I renamed the command-line options to -z sectionheader Generate section header (default) -z nosectionheader Do not generate section header Please add some ld testcases to that branch.
[hjl@gnu-cfl-1 pr25617]$ cat bar.c void bar (void) { } [hjl@gnu-cfl-1 pr25617]$ make libbar.so gcc -B./ -fpic -c -o bar.o bar.c ./ld -shared -z nosectionheader -o libbar.so bar.o [hjl@gnu-cfl-1 pr25617]$ readelf -d libbar.so Dynamic section at offset 0x2f40 contains 7 entries: Tag Type Name/Value 0x0000000000000004 (HASH) 0x238 0x000000006ffffef5 (GNU_HASH) 0x250 0x0000000000000005 (STRTAB) 0x2a8 0x0000000000000006 (SYMTAB) 0x278 0x000000000000000a (STRSZ) 5 (bytes) 0x000000000000000b (SYMENT) 24 (bytes) 0x0000000000000000 (NULL) 0x0 [hjl@gnu-cfl-1 pr25617]$ libbar.so is OK for ld.so. But there is no sufficient info for ld to tell how many entries in SYMTAB.
(In reply to H.J. Lu from comment #12) > > libbar.so is OK for ld.so. But there is no sufficient info for ld to tell > how many entries in SYMTAB. ld.so wants to know if a shared library has a definition of symbol `foo'. But ld wants to know how many symbols are in a shared library. PT_DYNAMIC segment provides answers for ld.so, but not for ld.
The information on the number of symbols in the symbol table is available in DT_HASH->nchain, as the chain table parallels the symbol table.
We can use DT_HASH/DT_GNU_HASH
Created attachment 12348 [details] patch to add test for -z nosectionheader This patch adds the test for it, based against your branch.
(In reply to Kaylee from comment #16) > Created attachment 12348 [details] > patch to add test for -z nosectionheader > > This patch adds the test for it, based against your branch. Please give my branch a try.
I get uninitialised variable usage errors when I try to build it, in this section of elf_link_is_defined_archive_symbol(): > if (elf_bad_symtab (abfd)) > { > extsymcount = symcount; > extsymoff = 0; > } > else > { > extsymcount = symcount - hdr->sh_info; > extsymoff = hdr->sh_info; > } Looks like symcount and hdr aren't initialized if extsymcount was nonzero in the previous conditional?
(In reply to Kaylee from comment #18) > I get uninitialised variable usage errors when I try to build it, in this > section of elf_link_is_defined_archive_symbol(): > > > if (elf_bad_symtab (abfd)) > > { > > extsymcount = symcount; > > extsymoff = 0; > > } > > else > > { > > extsymcount = symcount - hdr->sh_info; > > extsymoff = hdr->sh_info; > > } > > Looks like symcount and hdr aren't initialized if extsymcount was nonzero in > the previous conditional? Please try again.
Works for me now!
(In reply to Kaylee from comment #20) > Works for me now! I added --remove_section_header to objcopy/strip. Please take a look.
(In reply to H.J. Lu from comment #21) > (In reply to Kaylee from comment #20) > > Works for me now! > > I added --remove_section_header to objcopy/strip. Please take a look. It is --remove-section-header.
I got a failure in the ld testsuite in x86-64.exp. Unfortunately, about an hour later, my computer crashed, and I have since been unable to reproduce it. I think the text was something like "RELATIVE RELOCATION FAULT"?. It's possible that it was just some transient failure in my system that also lead to the crash. Using --remove-section-header on one of my test cases, objcopy issues a warning "Could not find any mergeable note sections". Probably it shouldn't warn about that unless --merge-notes was explicitly specified. Using objcopy (even without any flags) results in a shared object that segfaults at runtime. Looking at readelf output, it seems like objcopy is modifying the PT_LOAD entries. In particular, it seems to be moving the program header to the start of the file, and then modifying the PT_LOAD entry that covers the program header to cover it's new location, without taking into account that this makes two segments overlap, which is illegal. Though even if it modified it correctly, as a shared object its loaded segments may contain arbitrary cross-references from e.g. PC-relative relocations which were resolved at link time, and so can't be safely modified at all. strip has the same issues.
Apologies, with the new objcopy version it isn't generating overlapping PT_LOAD entries, though it is still modifying them and moving the program header.
(In reply to Kaylee from comment #23) > I got a failure in the ld testsuite in x86-64.exp. Unfortunately, about an > hour later, my computer crashed, and I have since been unable to reproduce > it. I think the text was something like "RELATIVE RELOCATION FAULT"?. It's > possible that it was just some transient failure in my system that also lead > to the crash. > > Using --remove-section-header on one of my test cases, objcopy issues a > warning "Could not find any mergeable note sections". Probably it shouldn't > warn about that unless --merge-notes was explicitly specified. Do you have a testcase?
(In reply to Kaylee from comment #24) > Apologies, with the new objcopy version it isn't generating overlapping > PT_LOAD entries, though it is still modifying them and moving the program > header. A testcase?
Created attachment 12357 [details] testcase for objcopy breakage This .so works fine, but if put through objcopy (even without flags), objcopy will attempt to copy the program header, and extend the relevant PT_LOAD entry to cover the new location. Unfortunately, it does not take into account that the endpoints of a loaded region are rounded up to page sizes, and as a result it will cause the first page of the second segment to overlap the last page of the first segment. I don't know if it's trying to create a second header because the .so -> BFD -> .so conversion is lossy, or if it's actively trying to modify it, but probably it should treat the program header as fixed, since application code may depend both on the specific values stored in it and also that the values are correct.
Created attachment 12358 [details] test case for note warning This is a testcase for the warning about no note section. I think any shared library compiled by clang or gcc should trigger this warning. I would guess that it's just that --remove-section-header implies --merge-notes, and --merge-notes warns if there aren't any notes to merge because it assumes the user explicitly requested it?
(In reply to Kaylee from comment #28) > Created attachment 12358 [details] > test case for note warning > > This is a testcase for the warning about no note section. I think any shared > library compiled by clang or gcc should trigger this warning. I would guess > that it's just that --remove-section-header implies --merge-notes, and > --merge-notes warns if there aren't any notes to merge because it assumes > the user explicitly requested it? Please try again.
(In reply to Kaylee from comment #27) > Created attachment 12357 [details] > testcase for objcopy breakage > > This .so works fine, but if put through objcopy (even without flags), > objcopy will attempt to copy the program header, and extend the relevant > PT_LOAD entry to cover the new location. Unfortunately, it does not take > into account that the endpoints of a loaded region are rounded up to page > sizes, and as a result it will cause the first page of the second segment to > overlap the last page of the first segment. > > I don't know if it's trying to create a second header because the .so -> BFD > -> .so conversion is lossy, or if it's actively trying to modify it, but > probably it should treat the program header as fixed, since application code > may depend both on the specific values stored in it and also that the values > are correct. It has nothing to do with --remove-section-header. You get the same result without --remove-section-header using objcopy on master branch.
Yep, I'm not getting the warning with the new version. Also the test case that I mentioned failed appears to be ld/testsuite/ld-x86-64/pr17618.d, which I think is probably unrelated.
(In reply to Kaylee from comment #31) > Yep, I'm not getting the warning with the new version. > > Also the test case that I mentioned failed appears to be > ld/testsuite/ld-x86-64/pr17618.d, which I think is probably unrelated. I didn't get any failures on x86.
(In reply to Kaylee from comment #31) > Yep, I'm not getting the warning with the new version. > > Also the test case that I mentioned failed appears to be > ld/testsuite/ld-x86-64/pr17618.d, which I think is probably unrelated. I fixed a couple bugs on users/hjl/pr25617/master branch. I think it is ready for master. Please give it some more thorough tests.
I'm getting several failures: > FAIL: Build pr22393-2 (-z nosectionheader, none) > FAIL: Build pr22393-2 (PIE, -z nosectionheader, none) > FAIL: Build pr22393-2 (-z nosectionheader, objcopy) > FAIL: Build pr22393-2 (PIE, -z nosectionheader, objcopy) > FAIL: Build pr22393-2 (-z nosectionheader, strip) > FAIL: Build pr22393-2 (PIE, -z nosectionheader, strip) Looking at the logs, they all have this: > extra lines in dump.out starting with "^0000000000001000 T _init$" Also I've managed to get the log for the sporadic x86-64.exp failure I mentioned, and it's just failing to allocate the 2GB it needs because I don't happen to have that amount free, so that's nothing to worry about.
There are a couple of typos in the comments: "symbol stable", "string stable", and "memory chekers".
(In reply to Kaylee from comment #34) > I'm getting several failures: > > > FAIL: Build pr22393-2 (-z nosectionheader, none) > > FAIL: Build pr22393-2 (PIE, -z nosectionheader, none) > > FAIL: Build pr22393-2 (-z nosectionheader, objcopy) > > FAIL: Build pr22393-2 (PIE, -z nosectionheader, objcopy) > > FAIL: Build pr22393-2 (-z nosectionheader, strip) > > FAIL: Build pr22393-2 (PIE, -z nosectionheader, strip) > > Looking at the logs, they all have this: > > > extra lines in dump.out starting with "^0000000000001000 T _init$" How can I reproduce it? Do they fail on master branch? > Also I've managed to get the log for the sporadic x86-64.exp failure I > mentioned, and it's just failing to allocate the 2GB it needs because I > don't happen to have that amount free, so that's nothing to worry about. Good to know.
(In reply to Kaylee from comment #35) > There are a couple of typos in the comments: "symbol stable", "string > stable", and "memory chekers". Fixed on users/hjl/pr25617/master branch. Thanks.
> How can I reproduce it? Do they fail on master branch? Yeah, they fail on master for me. Additionally, I've just found out that ld segfaults when trying to link against a shared object with GNU_PROPERTY_NO_COPY_ON_PROTECTED and no section headers; am looking into it.
Created attachment 12359 [details] testcase for GNU_PROPERTY_NO_COPY_ON_PROTECTED segfault The attached file exports an `int foo`, linking against it will segfault ld. It seems like it is segfaulting because it's calling SYMBOL_NO_COPYRELOC(info, eh), which is doing checks on members of `(EH)->elf.root.u.def.section->owner`, but this is set to NULL.
(In reply to Kaylee from comment #39) > Created attachment 12359 [details] > testcase for GNU_PROPERTY_NO_COPY_ON_PROTECTED segfault > > The attached file exports an `int foo`, linking against it will segfault ld. > > It seems like it is segfaulting because it's calling > SYMBOL_NO_COPYRELOC(info, eh), which is doing checks on members of > `(EH)->elf.root.u.def.section->owner`, but this is set to NULL. How do I use it to reproduce linker segfault?
(In reply to H.J. Lu from comment #40) > (In reply to Kaylee from comment #39) > > Created attachment 12359 [details] > > testcase for GNU_PROPERTY_NO_COPY_ON_PROTECTED segfault > > > > The attached file exports an `int foo`, linking against it will segfault ld. > > > > It seems like it is segfaulting because it's calling > > SYMBOL_NO_COPYRELOC(info, eh), which is doing checks on members of > > `(EH)->elf.root.u.def.section->owner`, but this is set to NULL. > > How do I use it to reproduce linker segfault? Never mind.
(In reply to Kaylee from comment #39) > Created attachment 12359 [details] > testcase for GNU_PROPERTY_NO_COPY_ON_PROTECTED segfault > > The attached file exports an `int foo`, linking against it will segfault ld. > > It seems like it is segfaulting because it's calling > SYMBOL_NO_COPYRELOC(info, eh), which is doing checks on members of > `(EH)->elf.root.u.def.section->owner`, but this is set to NULL. Fixed on users/hjl/pr25617/master branch. I will add a testcase later.
Based on the feedbacks, I updated users/hjl/pr25617/master branch to issue a linker error when dynamic symbol table from PT_DYNAMIC segment is used.
The master branch has been updated by H.J. Lu <hjl@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=96cc7918c18cdb7bd0331d836fa7f05341732580 commit 96cc7918c18cdb7bd0331d836fa7f05341732580 Author: Kaylee Blake <klkblake@gmail.com> Date: Wed Mar 4 19:18:07 2020 +1030 ELF: Strip section header in ELF objects Section header isn't mandatory on ELF executable nor shared library. This patch adds a new linker option, -z nosectionheader, to omit ELF section header, a new objcopy and strip option, --strip-section-headers, to remove ELF section headers. bfd/ 2023-06-06 H.J. Lu <hongjiu.lu@intel.com> Kaylee Blake <klkblake@gmail.com> PR ld/25617 * bfd.c (BFD_NO_SECTION_HEADER): New. (BFD_FLAGS_SAVED): Add BFD_NO_SECTION_HEADER. (BFD_FLAGS_FOR_BFD_USE_MASK): Likewise. * elfcode.h (elf_swap_ehdr_out): Omit section header with BFD_NO_SECTION_HEADER. (elf_write_shdrs_and_ehdr): Likewise. * elfxx-target.h (TARGET_BIG_SYM): Add BFD_NO_SECTION_HEADER to object_flags. (TARGET_LITTLE_SYM): Likewise. * bfd-in2.h: Regenerated. binutils/ 2023-06-06 H.J. Lu <hongjiu.lu@intel.com> PR ld/25617 * NEWS: Mention --strip-section-headers for objcopy and strip. * objcopy.c (strip_section_headers): New. (command_line_switch): Add OPTION_STRIP_SECTION_HEADERS. (strip_options): Add --strip-section-headers. (copy_options): Likewise. (copy_usage): Add --strip-section-headers. (strip_usage): Likewise. (copy_object): Handle --strip-section-headers for ELF files. (strip_main): Handle OPTION_STRIP_SECTION_HEADERS. (copy_main): Likewise. * doc/binutils.texi: Document --strip-section-headers for objcopy and strip. ld/ 2023-06-06 H.J. Lu <hongjiu.lu@intel.com> Kaylee Blake <klkblake@gmail.com> PR ld/25617 * NEWS: Mention -z nosectionheader. * emultempl/elf.em: Support -z sectionheader and -z nosectionheader. * ld.h (ld_config_type): Add no_section_header. * ld.texi: Document -z sectionheader and -z nosectionheader. * ldlang.c (ldlang_open_output): Handle config.no_section_header. * lexsup.c (parse_args): Enable --strip-all with -z nosectionheader. Disallow -r with -z nosectionheader. (elf_static_list_options): Add -z sectionheader and -z nosectionheader.
The master branch has been updated by H.J. Lu <hjl@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=02c1ba6c94d4bb3f53dfeeb4401c8434c7834a32 commit 02c1ba6c94d4bb3f53dfeeb4401c8434c7834a32 Author: H.J. Lu <hjl.tools@gmail.com> Date: Wed May 31 12:36:49 2023 -0700 ELF: Discard non-alloc sections without section header Discard non-alloc sections when section headers are stripped. bfd/ PR ld/25617 * elf.c (_bfd_elf_assign_file_positions_for_non_load): Skip non-load sections without section header. (_bfd_elf_write_object_contents): Don't set the sh_name field without section header. Write out the .shstrtab section only if its sh_offset field isn't -1. binutils/ PR ld/25617 * objcopy.c (is_strip_section_1): Remove non-alloc sections for --strip-section-headers. ld/ PR ld/25617 * ldlang.c (lang_discard_section_p): Discard non-alloc sections if we are stripping section headers.
The master branch has been updated by H.J. Lu <hjl@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=46675b6b8160e97485583522852ad86507bd9072 commit 46675b6b8160e97485583522852ad86507bd9072 Author: H.J. Lu <hjl.tools@gmail.com> Date: Wed Mar 4 20:32:35 2020 -0800 bfd: Improve nm and objdump without section header When there is no section header in an executable or shared library, we reconstruct dynamic symbol table from the PT_DYNAMIC segment, which contains DT_HASH/DT_GNU_HASH/DT_MIPS_XHASH, DT_STRTAB, DT_SYMTAB, DT_STRSZ, and DT_SYMENT entries, to improve nm and objdump. For DT_HASH, the number of dynamic symbol table entries equals the number of chains. For DT_GNU_HASH/DT_MIPS_XHASH, only defined symbols with non-STB_LOCAL indings are in hash table. Since DT_GNU_HASH/DT_MIPS_XHASH place all symbols with STB_LOCAL binding before symbols with other bindings and all undefined symbols defined ones in dynamic symbol table, the highest symbol index in DT_GNU_HASH/DT_MIPS_XHASH is the highest dynamic symbol table index. We can also get symbol version from DT_VERSYM, DT_VERDEF and DT_VERNEED entries. dt_symtab, dt_versym, dt_verdef, dt_verneed, dt_symtab_count, dt_verdef_count, dt_verneed_count and dt_strtab are added to elf_obj_tdata to store dynamic symbol table information. PR ld/25617 * elf-bfd.h (elf_obj_tdata): Add dt_symtab, dt_verdef, dt_verneed, dt_symtab_count, dt_verdef_count, dt_verneed_count and dt_strtab. (elf_use_dt_symtab_p): New. (_bfd_elf_get_dynamic_symbols): Likewise. (_bfd_elf_get_section_from_dynamic_symbol): Likewise. * elf.c (bfd_elf_get_elf_syms): Use dynamic symbol table if neeeded. (_bfd_elf_get_dynamic_symtab_upper_bound): Likewise. (_bfd_elf_slurp_version_tables): Likewise. (offset_from_vma): New function. (get_hash_table_data): Likewise. (_bfd_elf_get_dynamic_symbols): Likewise. (_bfd_elf_get_section_from_dynamic_symbol): Likewise. (_bfd_elf_get_symbol_version_name): Likewise. * elfcode.h (elf_object_p): Call _bfd_elf_get_dynamic_symbols to reconstruct dynamic symbol table from PT_DYNAMIC segment if there is no section header. (elf_slurp_symbol_table): Use dynamic symbol table if neeeded. Don't free isymbuf when dynamic symbol table is used. * elflink.c (elf_link_is_defined_archive_symbol): Return wrong format error when dynamic symbol table is used. (elf_link_add_object_symbols): Likewise.
The master branch has been updated by H.J. Lu <hjl@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=0a06ea7a767579aade45ff779170f2fa81dfcb11 commit 0a06ea7a767579aade45ff779170f2fa81dfcb11 Author: Kaylee Blake <klkblake@gmail.com> Date: Thu Mar 5 13:42:31 2020 +1030 ld: Add simple tests for -z nosectionheader 2020-06-06 Kaylee Blake <klkblake@gmail.com> H.J. Lu <hongjiu.lu@intel.com> PR ld/25617 * testsuite/ld-elf/nosectionheader-1.d: New file. * testsuite/ld-elf/nosectionheader-2.d: Likewise.
The master branch has been updated by H.J. Lu <hjl@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=1ad1b8865c19598326ef9bbb125ab60e4e462b55 commit 1ad1b8865c19598326ef9bbb125ab60e4e462b55 Author: H.J. Lu <hjl.tools@gmail.com> Date: Sat Mar 7 05:27:12 2020 -0800 binutils: Add a --strip-section-headers test PR ld/25617 * testsuite/binutils-all/objcopy.exp: Run strip-section-headers-1. * testsuite/binutils-all/strip-section-headers-1.d: New file.
The master branch has been updated by H.J. Lu <hjl@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=3da917d534954ae727a2174039a3786b474dd712 commit 3da917d534954ae727a2174039a3786b474dd712 Author: H.J. Lu <hjl.tools@gmail.com> Date: Sat Mar 7 08:24:35 2020 -0800 ld: Add tests for -z nosectionheader and --strip-section-headers Add tests to verify that the linker option, -z nosectionheader and objcopy and strip option, --strip-section-headers, work correctly as well as linker issues an error when dynamic symbol table from PT_DYNAMIC segment is used. PR ld/25617 * testsuite/ld-elf/hash-2.d: New file. * testsuite/ld-elf/no-section-header.exp: Likewise. * testsuite/ld-elf/pr25617-1-no-sec-hdr.nd: Likewise. * testsuite/ld-elf/pr25617-1-no-sec-hdr.rd: Likewise. * testsuite/ld-elf/pr25617-1-static-no-sec-hdr.rd: Likewise. * testsuite/ld-elf/pr25617-1a-no-sec-hdr.nd: Likewise. * testsuite/ld-elf/pr25617-1a-no-sec-hdr.rd: Likewise. * testsuite/ld-elf/pr25617-1a-sec-hdr.rd: Likewise. * testsuite/ld-elf/pr25617-1a.c: Likewise. * testsuite/ld-elf/pr25617-1b.c: Likewise. * testsuite/ld-elf/start-noheader.rd: Likewise. * testsuite/ld-elf/start-shared-noheader-gnu.rd: Likewise. * testsuite/ld-elf/start-shared-noheader-sysv.rd: Likewise. * testsuite/ld-elf/start-shared-noheader.nd: Likewise.
The master branch has been updated by H.J. Lu <hjl@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=b7b6f36275d5ff6a9e2bf679a5e3d354e531648a commit b7b6f36275d5ff6a9e2bf679a5e3d354e531648a Author: H.J. Lu <hjl.tools@gmail.com> Date: Mon Mar 9 14:37:26 2020 -0700 ld: Add -z nosectionheader test to bootstrap.exp PR ld/25617 * testsuite/ld-bootstrap/bootstrap.exp: Add -z nosectionheader test.
Done