Users use ldd to process directories full of files. In some cases ldd is used to process what appear to be ELF files. These ELF files are actually debuginfo objects that are only intended to be used with debuggers. The implementation of debuginfo objects is missing clear identification information because it does not allow for the easy disambiguation of these ELF files as debuginfo. When ldd is run on these objects files the corrupt and invalid ELF data leads to crashes in ldd as it considers the objects to be valid ELF files (they contain valid ELF headers). There are a few solutions to this problem: (a) Duplicate in ldd what eu-elfclassify does. In an attempt to create a robust identification tooling for object files Mark Wielaard and Florian Weimer created eu-elfclassify. This solution is conceptually wrong for ldd which should need to look at section headers, but would need to in order to duplicate what eu-elfclassify does. (b) Add ET_* markup for debuginfo files to clearly mark them as debuginfo files and avoid the problem by teaching the downstream tools, ldd first, that such files are not be loaded or handled like normal ELF files. From an ELF perspective they are invalid and incomplete and must be parsed with different kinds of rules than normal ELF files. My preferred solution is (b), but doing that may take a long time to deploy. The short-term alternative is (a) followed eventually by (b) to remove the section header processing at a later date.
Other issues to consider at the same time: (1) Kernel objects. - In Solaris these have special markup to avoid reading them as-if they were plain ET_REL objects. (2) dwz multi files are also ET_REL. One of the proposals for DWARFv5 is Split DWARF Objects (DWO) which is already implemented in some parts of the GNU toolchain. https://gcc.gnu.org/wiki/DebugFission http://dwarfstd.org/ShowIssue.php?issue=130313.4 This would leave some parts of the DWARF info in the .dwo (ET_REL) file, so the linker doesn't have to process it. Later these .dwo would be packaged together in a .dwp file for use with debuggers. --- In the case of (1) it's a distinct issue that needs a new markup. Solaris uses DF_1_KMOD to mark kernel modules. This means that with https://docs.oracle.com/cd/E88353_01/html/E37839/ld-1.html you can use -z type to say: exec, kmod, reloc, shared, pie, and static. That means you don't have to guess.
The binutils work is tracked here: https://sourceware.org/bugzilla/show_bug.cgi?id=22136
(In reply to Carlos O'Donell from comment #1) > Other issues to consider at the same time: > > (1) Kernel objects. > - In Solaris these have special markup to avoid reading them as-if they were > plain ET_REL objects. > > (2) dwz multi files are also ET_REL. > > One of the proposals for DWARFv5 is Split DWARF Objects (DWO) which is > already implemented in some parts of the GNU toolchain. > > https://gcc.gnu.org/wiki/DebugFission > http://dwarfstd.org/ShowIssue.php?issue=130313.4 > > This would leave some parts of the DWARF info in the .dwo (ET_REL) file, so > the linker doesn't have to process it. Later these .dwo would be packaged > together in a .dwp file for use with debuggers. > > --- > > In the case of (1) it's a distinct issue that needs a new markup. Solaris > uses DF_1_KMOD to mark kernel modules. This means that with > https://docs.oracle.com/cd/E88353_01/html/E37839/ld-1.html you can use -z > type to say: exec, kmod, reloc, shared, pie, and static. That means you > don't have to guess. I prefer DF_1_XXX over ET_XXX.
(In reply to H.J. Lu from comment #3) > (In reply to Carlos O'Donell from comment #1) > > In the case of (1) it's a distinct issue that needs a new markup. Solaris > > uses DF_1_KMOD to mark kernel modules. This means that with > > https://docs.oracle.com/cd/E88353_01/html/E37839/ld-1.html you can use -z > > type to say: exec, kmod, reloc, shared, pie, and static. That means you > > don't have to guess. > > I prefer DF_1_XXX over ET_XXX. It doesn't make sense to me. The use of DF_1_* requires PT_DYANMIC/.dynamic and DT_FLAGS_1. The PT_DYNAMIC/.dynamic is not present in debuginfo objects, it is stripped and NOBITS and should come from the original executable. This makes it less than useful to use as a way to self-identify the files. Likewise kernel modules have no PT_DYNAMIC/.dyanmic because they are ET_REL. So there is additional work to pursue a solution based on DF_1_* markup. The DF_1_* markup is also semantically about marking up dynamic objects, and debug info is not a dynamic object, it would make more sense to add new fundamental ELF types since that is what they are. Do you have any specific objection to a new ET_* markup?
Who are the consumers of this kind of special debug files?
(In reply to H.J. Lu from comment #5) > Who are the consumers of this kind of special debug files? The general problem is as follows. Downstream distributions need to build, split, and package debug information. During the process of building the debug information, for example in Fedora, you use a special scripts which include find-debuginfo.sh. The point is that because source files change frequently and you don't want to have to keep altering a static list of debug information files so the scripts automatically generate these files. In order to automate these processes there is a mix of file list scanning and analysis of built files being done. In some cases the lists of files to be scanned are inaccurate so some users use ldd to distinguish between certain "types" of files. We continue to get examples of cases where ldd can crash. My goal is to try make ldd more robust against crashing: (a) Tell users to stop using ldd and recommend eu-elfclassify. This action is already taken and eu-elfclassify is getting used in places where we need robust classification. (b) Make ldd more robust. This action is not yet taken and is the reason I filed this bug. If the dynamic loader could easily exclude processing certain files then existing user scripts using ldd to examine files would continue to work and be more robust. (c) Eventually replace ldd with eu-ldd that can do more complex analysis and assure users that they will never run code, and support container inspection. In the short term we've done (a), I think (b) has value, and I think (c) is the right long-term solution. To answer your question the user of the markup is going to be the dynamic loader ldd implementation (LD_DEBUG_TRACED_OBJECTS), eu-elfclassify could use the same markup to avoid guessing, future eu-ldd would use it also.
(In reply to H.J. Lu from comment #5) > Who are the consumers of this kind of special debug files? Split debug files are used by debuggers. The problem is that tooling to build packages has to manipulate lists of files and distinguish between different types of files in a robust manner to build installed file lists.
(In reply to Carlos O'Donell from comment #7) > (In reply to H.J. Lu from comment #5) > > Who are the consumers of this kind of special debug files? > > Split debug files are used by debuggers. > > The problem is that tooling to build packages has to manipulate lists of > files and distinguish between different types of files in a robust manner to > build installed file lists. Since debuggers can exam special debug files, will a single ET_GNU_DEBUG be sufficient?
(In reply to H.J. Lu from comment #8) > (In reply to Carlos O'Donell from comment #7) > > (In reply to H.J. Lu from comment #5) > > > Who are the consumers of this kind of special debug files? > > > > Split debug files are used by debuggers. > > > > The problem is that tooling to build packages has to manipulate lists of > > files and distinguish between different types of files in a robust manner to > > build installed file lists. > > Since debuggers can exam special debug files, will a single ET_GNU_DEBUG > be sufficient? Yes, I think that would suffice. When the ELF file requires completely different semantics for processing then my opinion is that the ET_* value should change. Likewise I think we need ET_GNU_KMOD, but given that we haven't seen any problem there, and *.ko's are limited to specific packages, the issue isn't as general as *all* packages having to handle split debug information.
(In reply to Carlos O'Donell from comment #9) > (In reply to H.J. Lu from comment #8) > > (In reply to Carlos O'Donell from comment #7) > > > (In reply to H.J. Lu from comment #5) > > > > Who are the consumers of this kind of special debug files? > > > > > > Split debug files are used by debuggers. > > > > > > The problem is that tooling to build packages has to manipulate lists of > > > files and distinguish between different types of files in a robust manner to > > > build installed file lists. > > > > Since debuggers can exam special debug files, will a single ET_GNU_DEBUG > > be sufficient? > > Yes, I think that would suffice. > > When the ELF file requires completely different semantics for processing > then my opinion is that the ET_* value should change. Let's make a proposal for ET_DEBUG to gABI. If it is rejected, we send ET_GNU_DEBUG to gnu-gABI. > Likewise I think we need ET_GNU_KMOD, but given that we haven't seen any > problem there, and *.ko's are limited to specific packages, the issue isn't > as general as *all* packages having to handle split debug information. How different are .ko files from .o files?
(In reply to H.J. Lu from comment #10) > (In reply to Carlos O'Donell from comment #9) > > (In reply to H.J. Lu from comment #8) > > > (In reply to Carlos O'Donell from comment #7) > > > > (In reply to H.J. Lu from comment #5) > > > > > Who are the consumers of this kind of special debug files? > > > > > > > > Split debug files are used by debuggers. > > > > > > > > The problem is that tooling to build packages has to manipulate lists of > > > > files and distinguish between different types of files in a robust manner to > > > > build installed file lists. > > > > > > Since debuggers can exam special debug files, will a single ET_GNU_DEBUG > > > be sufficient? > > > > Yes, I think that would suffice. > > > > When the ELF file requires completely different semantics for processing > > then my opinion is that the ET_* value should change. > > Let's make a proposal for ET_DEBUG to gABI. If it is rejected, we send > ET_GNU_DEBUG to gnu-gABI. Sounds good to me. > > Likewise I think we need ET_GNU_KMOD, but given that we haven't seen any > > problem there, and *.ko's are limited to specific packages, the issue isn't > > as general as *all* packages having to handle split debug information. > > How different are .ko files from .o files? No different really, some special sections, but that's all. They are already marked ET_REL, so the problem of "correct identification" is academic. I don't consider *.ko to be a problem since ldd and other tools won't confuse them with a complete ELF shared library or program that can be correctly interpreted.
(In reply to Carlos O'Donell from comment #11) > > > > Let's make a proposal for ET_DEBUG to gABI. If it is rejected, we send > > ET_GNU_DEBUG to gnu-gABI. > > Sounds good to me. > Done.