[ This is essentially a refiling of PR21191 - "objcopy --only-keep-debug creates non-monotonically increasing section offsets", which was closed as WONTFIX, for dwz. ] For the example from PR21191, with trunk dwz and objcopy we have: ... $ cat t.c int main() { } $ gcc t.c -g $ objcopy --only-keep-debug a.out a.debug $ dwz a.debug dwz: Allocatable section in a.debug after non-allocatable ones ... The dwz tool complains that allocatable section .init_array at offset 0xe40: ... [17] .init_array NOBITS 0000000000600e40 00000e40 0000000000000008 0000000000000008 WA 0 0 8 ... is at a bigger offset than non-allocatable section .debug_info at offset 0x410: ... [26] .debug_info PROGBITS 0000000000000000 00000410 00000000000002f3 0000000000000000 0 0 1 ... In the original a.out this situation does not occur: ... [17] .init_array INIT_ARRAY 0000000000600e40 00000e40 0000000000000008 0000000000000008 WA 0 0 8 [26] .debug_info PROGBITS 0000000000000000 000011a0 00000000000002f3 0000000000000000 0 0 1 ... The rationale with which PR21191 was closed as WONTFIX is: "File offset for a SHT_NOBITS section is irrelevant". Following that rationale, dwz should not complain here.
Created attachment 11630 [details] readelf -S a.out
Created attachment 11631 [details] readelf -S a.debug
Created attachment 11632 [details] a.out
Created attachment 11633 [details] a.debug
Created attachment 11636 [details] Tentative patch
(In reply to Tom de Vries from comment #5) > Created attachment 11636 [details] > Tentative patch Also fixed by dwz-0.12-ignore-nobits.patch at https://sourceware.org/bugzilla/show_bug.cgi?id=24249#c3 .
Submitted patch: https://sourceware.org/ml/dwz/2019-q1/msg00110.html
Fix committed: https://sourceware.org/git/?p=dwz.git;a=commit;h=66616a47d2d6094a48ada2a51dc05aad8ee58649 I'm not yet closing the PR, as I'm investigating the failure of the added test-cases for SLE_12_SP4 s390x aarch64 ppc64le x86_64 @ https://build.opensuse.org/package/show/home:tomdevries:branches:devel:tools:compiler/dwz , due to eu-unstrip failing to find matching sections.
(In reply to Tom de Vries from comment #8) > Fix committed: > https://sourceware.org/git/?p=dwz.git;a=commit; > h=66616a47d2d6094a48ada2a51dc05aad8ee58649 > > I'm not yet closing the PR, as I'm investigating the failure of the added > test-cases for SLE_12_SP4 s390x aarch64 ppc64le x86_64 @ > https://build.opensuse.org/package/show/home:tomdevries:branches:devel:tools: > compiler/dwz , due to eu-unstrip failing to find matching sections. Fixed by https://sourceware.org/git/?p=dwz.git;a=commit;h=8e56b32484d5f5d525255fa4cbc656aeb613ecd3
FYI even dwz-0.13 which already contains this patch still shows: dwz: Allocatable section in ./usr/bin/boottime-0-0.3.20200615git15668db.fc33.x86_64.debug after non-allocatable ones in many packages, such as: https://kojipkgs.fedoraproject.org//packages/sysutil/0/0.3.20200615git15668db.fc33/data/logs/x86_64/build.log https://koji.fedoraproject.org/koji/rpmlist?buildrootID=21879423&type=component = dwz-0.13-2.fc32.x86_64.rpm
Note that there are multiple warnings in that log: warning: Unsupported auto-load script at offset 0 in section .debug_gdb_scripts of file /builddir/build/BUILDROOT/sysutil-0-0.3.20200615git15668db.fc33.x86_64/usr/bin/boottime. Use `info auto-load python-scripts [REGEXP]' to list them. nm: /builddir/build/BUILDROOT/sysutil-0-0.3.20200615git15668db.fc33.x86_64/usr/bin/boottime: .gnu.version_r invalid entry nm: /builddir/build/BUILDROOT/sysutil-0-0.3.20200615git15668db.fc33.x86_64/usr/bin/boottime: no symbols dwz: Allocatable section in ./usr/bin/boottime-0-0.3.20200615git15668db.fc33.x86_64.debug after non-allocatable ones It is a golang binary and seems to use both the golang linker and the system linker. I suspect that it really is slightly corrupt because of that. Note that the section offsets jump around a bit: Section Headers: [Nr] Name Type Addr Off Size ES Flags Lk Inf Al [ 0] NULL 0000000000000000 00000000 00000000 0 0 0 0 [ 1] .text PROGBITS 0000000000401000 00001000 000ab22e 0 AX 0 0 32 [ 2] .plt PROGBITS 00000000004ac230 000ac230 00000010 16 AX 0 0 16 [ 3] .rodata PROGBITS 00000000004ad000 000ad000 00020123 0 A 0 0 32 [ 4] .gnu.version_r GNU_verneed 00000000004cd123 000cd123 00000000 0 A 8 0 1 [ 5] .dynsym DYNSYM 0000000000526970 00126970 00000018 24 A 8 1 8 [ 6] .rela RELA 00000000004cd128 000cd128 00059820 24 A 5 0 8 [ 7] .rela.plt RELA 0000000000526948 00126948 00000000 24 AI 5 17 8 [ 8] .dynstr STRTAB 0000000000526948 00126948 00000001 0 A 0 0 1 [ 9] .gnu.version GNU_versym 000000000052694a 0012694a 00000002 2 A 5 0 2 [10] .hash HASH 0000000000526950 00126950 00000014 4 A 5 0 8 [11] .data.rel.ro PROGBITS 0000000000527000 00127000 00027f40 0 WA 0 0 32 [12] .data.rel.ro.typelink PROGBITS 000000000054ef40 0014ef40 000007a4 0 WA 0 0 32 [13] .data.rel.ro.itablink PROGBITS 000000000054f6e8 0014f6e8 00000070 0 WA 0 0 8 [14] .data.rel.ro.gosymtab PROGBITS 000000000054f758 0014f758 00000000 0 WA 0 0 1 [15] .data.rel.ro.gopclntab PROGBITS 000000000054f760 0014f760 00068900 0 WA 0 0 32 [16] .go.buildinfo PROGBITS 00000000005b9000 001b9000 00000020 0 WA 0 0 16 [17] .got.plt PROGBITS 00000000005b9020 001b9020 00000018 8 WA 0 0 8 [18] .dynamic DYNAMIC 00000000005b9040 001b9040 000000b0 16 WA 8 0 8 [19] .got PROGBITS 00000000005b90f0 001b90f0 00000000 8 WA 0 0 8 [20] .noptrdata PROGBITS 00000000005b9100 001b9100 0000e740 0 WA 0 0 32 [21] .data PROGBITS 00000000005c7840 001c7840 000076d0 0 WA 0 0 32 [22] .bss NOBITS 00000000005cef20 001cef10 00030050 0 WA 0 0 32 [23] .noptrbss NOBITS 00000000005fef80 001cef10 00002888 0 WA 0 0 32 [24] .tbss NOBITS 0000000000000000 001cef10 00000008 0 WAT 0 0 8 [25] .interp PROGBITS 0000000000400fe4 00000fe4 0000001c 0 A 0 0 1 [26] .note.gnu.build-id NOTE 0000000000400fc0 00000fc0 00000024 0 A 0 0 4 [27] .note.go.buildid NOTE 0000000000400f5c 00000f5c 00000064 0 A 0 0 4 [28] .gnu_debuglink PROGBITS 0000000000000000 001cef10 00000038 0 0 0 4 [29] .gnu_debugdata PROGBITS 0000000000000000 001cef48 00004624 0 0 0 1 [30] .shstrtab STRTAB 0000000000000000 001d356c 00000156 0 0 0 1 In particular .intrp (25), .note.gnu.build-id (26) and .note.go.buildid (27) have offsets that place them at the start of the file instead of at the end.
(In reply to Mark Wielaard from comment #11) > Section Headers: > [Nr] Name Type Addr Off Size ES > Flags Lk Inf Al > [25] .interp PROGBITS 0000000000400fe4 00000fe4 0000001c 0 > A 0 0 1 > [26] .note.gnu.build-id NOTE 0000000000400fc0 00000fc0 00000024 0 > A 0 0 4 > [27] .note.go.buildid NOTE 0000000000400f5c 00000f5c 00000064 0 > A 0 0 4 > > In particular .intrp (25), .note.gnu.build-id (26) and .note.go.buildid (27) > have offsets that place them at the start of the file instead of at the end. .note.gnu.build-id (or rather NT_GNU_BUILD_ID) must be in the first page (for Linux kernel core files) so rather the possible bug is numbering of the section, not their offset in file. But then I do not see in ELF specification the offset or address would be required in nondecreasing order so I find it only as a bug in DWZ.
(In reply to Jan Kratochvil from comment #10) > FYI even dwz-0.13 which already contains this patch still shows: > > dwz: Allocatable section in > ./usr/bin/boottime-0-0.3.20200615git15668db.fc33.x86_64.debug after > non-allocatable ones > > in many packages, such as: > > https://kojipkgs.fedoraproject.org//packages/sysutil/0/0.3. > 20200615git15668db.fc33/data/logs/x86_64/build.log > https://koji.fedoraproject.org/koji/ > rpmlist?buildrootID=21879423&type=component > = dwz-0.13-2.fc32.x86_64.rpm Can you file a PR with a reproducer?
Created attachment 12847 [details] boottime-debug (In reply to Tom de Vries from comment #13) > (In reply to Jan Kratochvil from comment #10) > > FYI even dwz-0.13 which already contains this patch still shows: > > > > dwz: Allocatable section in > > ./usr/bin/boottime-0-0.3.20200615git15668db.fc33.x86_64.debug after > > non-allocatable ones > > > > in many packages, such as: > > > > https://kojipkgs.fedoraproject.org//packages/sysutil/0/0.3. > > 20200615git15668db.fc33/data/logs/x86_64/build.log > > https://koji.fedoraproject.org/koji/ > > rpmlist?buildrootID=21879423&type=component > > = dwz-0.13-2.fc32.x86_64.rpm > > Can you file a PR with a reproducer? Got one, extracted from https://koji.fedoraproject.org/koji/buildinfo?buildID=1570316 sysutil-debuginfo-0-0.3.20200615git15668db.fc33.x86_64.rpm
(In reply to Mark Wielaard from comment #14) > Created attachment 12847 [details] > boottime-debug > > Got one, extracted from > https://koji.fedoraproject.org/koji/buildinfo?buildID=1570316 > sysutil-debuginfo-0-0.3.20200615git15668db.fc33.x86_64.rpm And dwz is right, it really has an allocated section in between non-allocated ones: [33] .debug_ranges PROGBITS 000000000067e88f 0017edfd 00034d10 0 0 0 1 [34] .interp NOBITS 0000000000400fe4 001b3b0d 0000001c 0 A 0 0 1 [35] .note.gnu.build-id NOTE 0000000000400fc0 001b3b10 00000024 0 A 0 0 4 [36] .note.go.buildid NOTE 0000000000400f5c 001b3b34 00000064 0 A 0 0 4 [37] .gdb_index PROGBITS 0000000000000000 001b3b98 0004321f 0 0 0 1 We can (and I believe dwz does) ignore the .interp section, since that is NOBITS. But those two build-id/buildid NOTEs are allocated in the middle of the file and so cannot be moved. This looks like a broken ELF file to begin with. Those sections really should have been at the start of the file. I wonder if this is a bug in the golang toolchain. If so, it would be better to fix that than to try work around it.
(In reply to Mark Wielaard from comment #15) > But those two build-id/buildid NOTEs are allocated in the middle of > the file and so cannot be moved. This looks like a broken ELF file to begin > with. Those sections really should have been at the start of the file. I > wonder if this is a bug in the golang toolchain. If so, it would be better > to fix that than to try work around it. Have you read my Comment 12?
ELF doesn't have many requirements and the consumers just choose what to ignore. The kernel loader, glibc etc. will reject many "valid" kinds of ELF too, they just expect sane linkers to generate something that makes sense. And the same is with dwz (and used to be e.g. with prelink), if the linker/assembler produces garbage, then it is better to fix those. When binutils at some point produced also section orderings that dwz/prelink didn't like, we've fixed binutils not to do that.
(In reply to Jan Kratochvil from comment #16) > Have you read my Comment 12? Yes I did of course. In that comment you describe seeing the original issue which should have already been fixed. So in comment 13 Tom asked you for a reproducer so we could investigate why the fix doesn't work for your case. You didn't provide such a reproducer. So we tried to get a reproducer ourselves and that shows the issue is actually precisely what dwz warns about and what looks like a bug in the golang toolchain.
(In reply to Mark Wielaard from comment #18) > You didn't provide such a reproducer. I did in Comment 10. Moreover even if I did not (which I did) the Fedora build.logs are freely available to anyone. > what looks like a bug in the golang toolchain. golang toolchain will probably disagree.
(In reply to Jan Kratochvil from comment #19) > (In reply to Mark Wielaard from comment #18) > > You didn't provide such a reproducer. > > I did in Comment 10. Moreover even if I did not (which I did) the Fedora > build.logs are freely available to anyone. You did not produce a reproducer. I can see the build.log but I cannot reproduce your issue. What I can see from the build.log is a situation that I believe has been fixed by Tom. But it is impossible to be sure because you didn't provide an exact reproducer or binary. The binary I could reproduce is slightly different from the case you report and seems to be a real bug in either the golang toolchain or the spec build file. dwz correctly gives an error in that case. If you do provide a reproducer then I am happy to take a look at why Tom's fix isn't working for you. But you cannot expect us to handwave at some build.log and make use figure out what you are exactly doing without having access to either the exact build environment or the actual binary you are seeing the issue with.
This issue was reopened, but for something unrelated to the original bug fixed. If the new issue can still be reproduced, and it isn't a toolchain issue, then please open a new bug.