Bug 24766 - [Regression] dwz exits with 1 when running into ".debug_info section not present" error
Summary: [Regression] dwz exits with 1 when running into ".debug_info section not pres...
Status: RESOLVED FIXED
Alias: None
Product: dwz
Classification: Unclassified
Component: default (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-07-04 11:37 UTC by Matthias Klose
Modified: 2019-07-11 10:55 UTC (History)
4 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matthias Klose 2019-07-04 11:37:32 UTC
This worked with 0.12 release:

$ ./dwz.sh 
++ dwz -m.dwz/verilator.debug -M.dwz/verilator.debug -- verilator_bin verilator_bin_dbg verilator_coverage_bin_dbg
dwz: verilator_bin: .debug_info section not present
dwz: verilator_bin_dbg: DWARF compression not beneficial - old size 44629826 new size 44629826
dwz: verilator_coverage_bin_dbg: .gnu_debugaltlink section already present
dwz: Too few files for multifile optimization

test case:
https://people.debian.org/~doko/tmp/tst-verilator.tar.xz
Comment 1 Gianfranco 2019-07-04 13:42:49 UTC
This might have been introduced by
https://sourceware.org/git/?p=dwz.git;a=commitdiff;h=08becc8b33453b6d013a65e7eeae57fc1881e801
Comment 2 Tom de Vries 2019-07-04 14:10:20 UTC
(In reply to Matthias Klose from comment #0)
> This worked with 0.12 release:
> 
> $ ./dwz.sh 
> ++ dwz -m.dwz/verilator.debug -M.dwz/verilator.debug -- verilator_bin
> verilator_bin_dbg verilator_coverage_bin_dbg
> dwz: verilator_bin: .debug_info section not present
> dwz: verilator_bin_dbg: DWARF compression not beneficial - old size 44629826
> new size 44629826
> dwz: verilator_coverage_bin_dbg: .gnu_debugaltlink section already present
> dwz: Too few files for multifile optimization
> 
> test case:
> https://people.debian.org/~doko/tmp/tst-verilator.tar.xz

I think these files are the ones after dwz has run. I'll need the ones before dwz has run in order to reproduce. [ This is a known PITA of dwz. ]
Comment 3 Matthias Klose 2019-07-04 14:48:53 UTC
updated: https://people.debian.org/~doko/tmp/tst-v.tar.xz
Comment 4 Tom de Vries 2019-07-04 15:00:37 UTC
(In reply to Matthias Klose from comment #3)
> updated: https://people.debian.org/~doko/tmp/tst-v.tar.xz

So, the error is:
...
$ ./run.sh
+ cp verilator_bin 1
+ cp verilator_bin_dbg 2
+ cp verilator_coverage_bin_dbg 3
+ dwz -m 4 1 2 3
dwz: 1: .debug_info section not present
dwz: 2: DWARF compression not beneficial - old size 44630638 new size 44641594
+ echo 'exit status: 1'
exit status: 1
...

This is fallout of 815ac61 "Honour errors when processing more than one file".

If we run dwz on an executable without .debug_info section, we get an exit status of 1:
...
$ dwz verilator_bin; echo $?
dwz: verilator_bin: .debug_info section not present
1
...

However, before the 815ac61 commit, if we did the same in multifile mode, we got an exit status of 0:
...
$ cp verilator_bin 1
$ cp 1 2
$ dwz -m 3 1 2; echo $?
dwz: 1: .debug_info section not present
dwz: 2: .debug_info section not present
dwz: Too few files for multifile optimization
0
...

After the 815ac61 commit, we get an exit status of 1, just as in regular mode:
...
$ dwz -m 3 1 2; echo $?
dwz: 1: .debug_info section not present
dwz: 2: .debug_info section not present
dwz: Too few files for multifile optimization
1
...

Jakub, WDYT? We could argue that not having .debug_info (due to an executable not being compiled with -g) is not worth exiting with 1 for.
Comment 5 Jakub Jelinek 2019-07-04 15:09:19 UTC
Unsure what people want.  E.g. the rpm find-debuginfo.sh I think just ignores the return status of dwz, the program should either perform the compression if beneficial and possible, or give up, but neither case is some kind of error.
The only "errors" are dwz assertion failures and even those should hopefully not break stuff, just worse case keep it uncompressed.
Comment 6 Matthias Klose 2019-07-04 15:28:12 UTC
should there be distinct error codes for errors to ignore, and errors which shouldn't be ignored?  I assume if you just ignore everything, then you won't get more issues reported, because nobody sees these missed compressions anymore.
Comment 7 Tom de Vries 2019-07-10 08:32:13 UTC
(In reply to Jakub Jelinek from comment #5)
> Unsure what people want.  E.g. the rpm find-debuginfo.sh I think just
> ignores the return status of dwz, the program should either perform the
> compression if beneficial and possible, or give up, but neither case is some
> kind of error.

Confirmed, that's the case.

> The only "errors" are dwz assertion failures and even those should hopefully
> not break stuff, just worse case keep it uncompressed.

That's the case as well. [ I've had a report from somebody saying that changes in OBS meant that a dwz assert broke a package build, but I'm not yet sure what changes those are, I'm currently chasing that up. ]
Comment 8 Tom de Vries 2019-07-10 09:13:31 UTC
(In reply to Matthias Klose from comment #6)
> should there be distinct error codes for errors to ignore, and errors which
> shouldn't be ignored?  I assume if you just ignore everything, then you
> won't get more issues reported, because nobody sees these missed
> compressions anymore.

The strategy of ignoring exit status is valid, but it also gives us freedom to make changes to the exit code without disturbing that strategy.

The question is though: can we define a rationale for dwz exit codes that allows us to use the exit codes in a meaningful way? What is the difference between "error to ignore", and "error that shouldn't be ignored"?

In other words, if we keep fixing bug reports that say "this dwz error breaks my package build, so it's an error that should be ignored", won't we end up with effectively ignoring all errors?

[ FWIW, as for noticing missing compressions, find-debuginfo.sh has a Aug 2018 patch that added printing of a line like "original debug info size: 120600, size after compression: 77604", so perhaps that can be addressed more adequately outside dwz.  ]
Comment 9 Matthias Klose 2019-07-10 13:11:14 UTC
that's the Debian issue connected with this issue:

  https://bugs.debian.org/931792

for the error codes: compressed debug info maybe should have an error code which shows that it can be ignored?  same for code not built with -g?
Comment 10 Tom de Vries 2019-07-11 10:13:30 UTC
(In reply to Matthias Klose from comment #9)
> that's the Debian issue connected with this issue:
> 
>   https://bugs.debian.org/931792
> 
> for the error codes: compressed debug info maybe should have an error code
> which shows that it can be ignored?  same for code not built with -g?

So, AFAIU, you mean something along the lines of:
...
$ dwz 1; echo $?
0
$ dwz 1; echo $?
dwz: 1: DWARF compression not beneficial
2
$ dwz 2; echo $?
dwz: 2: .debug_info section not present
3
$ dwz 3; echo $?
dwz: 3: invalid dwarf: no DIE at reference 0x123
4
...

That is, we let the exit status be informative about what type of situations we find, and let the caller decide how to handle these.

We could do something like that, but that looks like an enhancement for a next release to me, not a regression fix. I'll file this as spin-off PR.

For now, let's fix this regression more pragmatically.
Comment 11 Tom de Vries 2019-07-11 10:15:42 UTC
(In reply to Tom de Vries from comment #10)
> For now, let's fix this regression more pragmatically.

https://sourceware.org/git/?p=dwz.git;a=commit;h=51a7fba9895c0256888fe8996f37c598ca56749a
Comment 12 Jakub Jelinek 2019-07-11 10:19:05 UTC
And then the question is what happens if there are multiple issues (or harmless warnings) reported, what the final error code should be.  Say -m, one file has no .debug_info, another one where no compression is beneficial, another one has corrupt .debug_info, etc.
Comment 13 Tom de Vries 2019-07-11 10:55:10 UTC
(In reply to Jakub Jelinek from comment #12)
> And then the question is what happens if there are multiple issues (or
> harmless warnings) reported, what the final error code should be.  Say -m,
> one file has no .debug_info, another one where no compression is beneficial,
> another one has corrupt .debug_info, etc.

I suppose you could classify the exit status in increasing level of severity, and return the maximum severity.

Something like:
- 0: valid input, compressed
- 1: valid input, not compressed
- 2: invalid input, compressed
- 3: invalid input, not compressed
[ Though I'm not sure about the relative order of 1 and 2 ]