A gentle ping on this issue?

This probably is something that elfutils should transparently support for libabigail. But currently elfutils libdw doesn't support GNU DebugFission dwp/split-dwarf. Parts of it are in the process of being standardized now for DWARF5. But the standard hasn't been finalized and code for elfutils is not yet there (and after DWARF5 public beta was released some details were changed).

For now I would recommend not using -gsplit-dwarf and dwp. If you are looking for smaller or "split" DWARF then yse using eu-strip -o and dwz to create debug multi-files. That is supported.

Hi Mark - 

Thank you for clarifying. It sounds from your description like this will eventually work automatically, which is great.

My actual goal for using DebugFission was more for the build time improvements, rather than debug symbol packaging. My use of libabigail/abidw is somewhat unusual, and also focused on build time improvements:

https://sourceware.org/bugzilla/show_bug.cgi?id=18838#c4
http://scons-users.scons.narkive.com/E9RiycaY/replacement-for-sharedlibrarysignatureoverride

I had hoped to be able to use both DebugFission and abidw driven re-link pruning simultaneously, but it appears that this will not work, for the moment.

I have experimented with dwz as well, with the aim of reducing the size of our debug symbol packages, but so far without success: https://bugzilla.redhat.com/show_bug.cgi?id=1399866. If you have any additional thoughts on that ticket it would be greatly appreciated. I still don't have an explanation for why dwz cannot process the output of ld.gold.

I'm also unclear on the intended relationship between dwp and dwz. When I tried running dwz on dwp files, my recollection is that it didn't work, at all. I'd presume that the process would be to collect the .dwo files associated with each binary into a .dwp file for that binary with the dwp tool, and then run dwz across all the dwp files to extract common symbols into a shared file, along with any other optimization/compression. I would then hope that abidw run on the binary would be able to present the full ABI, extracting information from both the associated .dwp file, and any common underling file produced by dwz.

Or, perhaps the functionality of dwz should be folded directly into dwp? The documentation on these tools and the intended direction for development is somewhat unclear to me.

(In reply to andrew.c.morrow from comment #3)
> Thank you for clarifying. It sounds from your description like this will
> eventually work automatically, which is great.

It will however take some work, it is not really trivial to support. Also while GNU DebugFission is being standardized in the upcoming DWARFv5 there have been various changes. So it isn't clear yet which tools will support which version.

> I have experimented with dwz as well, with the aim of reducing the size of
> our debug symbol packages, but so far without success:
> https://bugzilla.redhat.com/show_bug.cgi?id=1399866. If you have any
> additional thoughts on that ticket it would be greatly appreciated. I still
> don't have an explanation for why dwz cannot process the output of ld.gold.

I commented on that bug report. It looks like the linker is creating a bogus .bss section, but there is not really enough information in the bug report to know for sure.

> I'm also unclear on the intended relationship between dwp and dwz. When I
> tried running dwz on dwp files, my recollection is that it didn't work, at
> all. I'd presume that the process would be to collect the .dwo files
> associated with each binary into a .dwp file for that binary with the dwp
> tool, and then run dwz across all the dwp files to extract common symbols
> into a shared file, along with any other optimization/compression. I would
> then hope that abidw run on the binary would be able to present the full
> ABI, extracting information from both the associated .dwp file, and any
> common underling file produced by dwz.
> 
> Or, perhaps the functionality of dwz should be folded directly into dwp? The
> documentation on these tools and the intended direction for development is
> somewhat unclear to me.

The have been developed completely independently and I don't think anybody has really thought about how to combine them. They also serve somewhat different purposes. And not many tools support both DWARF debuginfo data/file variants.

This recently ended up on my radar again. Is there a ticket tracker for elfutils where I could file an issue, or an already filed issue I should watch? My understanding is that DWARF5 is now an issued standard, and it would be great to see tools like abidw which are built above elfutils be able to reach out to .dwo or .dwp files when -gsplit-dwarf (or its DWARF5 equivalent) is in play.

elfutils 0.171 was recently released with support for DWARF5 and split-dwarf (including GNU DebugFission as extension to older DWARF).

https://sourceware.org/ml/elfutils-devel/2018-q2/msg00141.html

For now .dwo files are supported, but not yet .dwp files.
(Also the binutils dwp tools doesn't work with DWARF5 split dwarf.)
It will take a bit more programming to also add .dwp support, but hopefully the structure of the elfutils code is now so that this isn't major surgery.

Although most things should work as is, libabigail will need to use a few new interfaces to properly handle split dwarf, iterate through CUs using the new dwarf_get_units, and change from using offsets and dwarf_dieoffset/dwarf_offdie to using the new dwarf_die_addr_die interface (because DIEs can now come from multiple files, so using offsets alone isn't enough to identify a DIE).

While we update libabigail for this could you try out the new elfutils-0.171 tools to see if they work correctly in your setup?

And could you indicate how important .dwp support is? Would a libabigail that just works with .dwo files be enough for now?

The interfaces should be generic enough that when .dwp support is added to elfutils libdw, then they should work as is and libabigail shouldn't really care whether the split dwarf units come from the .dwo files or the .dwp package file.

(In reply to Mark Wielaard from comment #6)
> elfutils 0.171 was recently released with support for DWARF5 and split-dwarf
> (including GNU DebugFission as extension to older DWARF).
> 
> https://sourceware.org/ml/elfutils-devel/2018-q2/msg00141.html

That is great news, thanks.

> 
> For now .dwo files are supported, but not yet .dwp files.
> (Also the binutils dwp tools doesn't work with DWARF5 split dwarf.)
> It will take a bit more programming to also add .dwp support, but hopefully
> the structure of the elfutils code is now so that this isn't major surgery.
>

Support for .dwo files is currently more important for me than .dwp.

 
> Although most things should work as is, libabigail will need to use a few
> new interfaces to properly handle split dwarf, iterate through CUs using the
> new dwarf_get_units, and change from using offsets and
> dwarf_dieoffset/dwarf_offdie to using the new dwarf_die_addr_die interface
> (because DIEs can now come from multiple files, so using offsets alone isn't
> enough to identify a DIE).
> 
> While we update libabigail for this could you try out the new elfutils-0.171
> tools to see if they work correctly in your setup?

A quick and dirty test suggests that it is working:

$ g++ -g -gsplit-dwarf -c -o hello_world.o ./hello_world.cpp -std=c++14
$ g++ -o hello_world hello_world.o
$ strace -o trace ~/opt/bin/eu-readelf --debug-dump=info+ --all ./hello_world
$ grep dwo trace
write(1, "           GNU_dwo_name         "..., 57) = 57
write(1, "           GNU_dwo_id           "..., 59) = 59
openat(AT_FDCWD, "/home/andrew/hello_world.dwo", O_RDONLY) = 4
readlink("/proc/27201/fd/4", "/home/andrew/hello_world.dwo", 4095) = 28
lstat("/home/andrew/hello_world.dwo", {st_mode=S_IFREG|0664, st_size=21696, ...}) = 0
write(1, "           GNU_dwo_id           "..., 59) = 59 

More complete testing will need to wait on the libabigail support.

> 
> And could you indicate how important .dwp support is? Would a libabigail
> that just works with .dwo files be enough for now?

I can definitely make use of a libabigail that only supports .dwo, perhaps indefinitely. Thanks!

So I wanted to use libabigail on GCC '-gsplit-dwarf' object files ('*.o'/'*.dwo'), and found that it doesn't work.  Then found this existing PR; confirming that status is still as per Andrew's 2018-06-16 comment: 'eu-readelf'/elfutils (Ubuntu 0.176-1.1build1 in my case) appears to work, but 'abidw'/libabigail (current master branch) doesn't even open the '*.dwo' file, per 'strace'.

If I wanted to try help implement that (but no promises how far I'll get...), are Mark's 2018-06-05 comments here re "libabigail will need to use a few new interfaces" still applicable, to get started?

Additionally, in PR25042 I saw Mark's 2019-09-30 comment that "logic [there] is broken, [...] when dealing with split-dwarf DWO files", so I suppose that needs to be reworked, too?  Per Dodji's 2019-10-01 comment, "this optimization will be unnecessary, hopefully".

Hi Thomas,

(In reply to Thomas Schwinge from comment #8)
> So I wanted to use libabigail on GCC '-gsplit-dwarf' object files
> ('*.o'/'*.dwo'), and found that it doesn't work.  Then found this existing
> PR; confirming that status is still as per Andrew's 2018-06-16 comment:
> 'eu-readelf'/elfutils (Ubuntu 0.176-1.1build1 in my case) appears to work,
> but 'abidw'/libabigail (current master branch) doesn't even open the '*.dwo'
> file, per 'strace'.
> 
> If I wanted to try help implement that (but no promises how far I'll
> get...), are Mark's 2018-06-05 comments here re "libabigail will need to use
> a few new interfaces" still applicable, to get started?

Yes, that is still applicable.

Where libabigail now uses dwarf_next_unit, it should use dwarf_get_units. dwarf_get_units provides the type of the unit, if it is a type unit or split unit, it will not only provide the CU DIE, but also the "Sub DIE", that is the Type DIE (for type units) or Split DIE (for split units). It can then follow the Split DIE to get the actual DIE tree for that unit (the CU DIE is just the Skeleton DIE in that case).

These DIEs might come from different files (Dwarf objects) than the main Dwarf. So where libabigail now uses Dwarf_Off as a (unique) identifier of a DIE (and uses dwarf_offdie or dwarf_offdie_types to reconstruct a Dwarf_Die from a Dwarf_Off), it should now use the Dwarf_Die void *addr field as unique identifier (and use dwarf_die_addr_die to reconstruct the Dwarf_Die).

See libdw.h for more documentation. Specifically:

/* Gets the next Dwarf_CU (unit), version, unit type and if available
   the CU DIE and sub (type) DIE of the unit.  Returns 0 on success,
   -1 on error or 1 if there are no more units.  To start iterating
   provide NULL for CU.  If version < 5 the unit type is set from the
   CU DIE if available (DW_UT_compile for DW_TAG_compile_unit,
   DW_UT_type for DW_TAG_type_unit or DW_UT_partial for
   DW_TAG_partial_unit), otherwise it is set to zero.  If unavailable
   (the version or unit type is unknown) the CU DIE is cleared.
   Likewise if the sub DIE isn't isn't available (the unit type is not
   DW_UT_type or DW_UT_split_type) the sub DIE tag is cleared.  */
extern int dwarf_get_units (Dwarf *dwarf, Dwarf_CU *cu, Dwarf_CU **next_cu,
                            Dwarf_Half *version, uint8_t *unit_type,
                            Dwarf_Die *cudie, Dwarf_Die *subdie)
     __nonnull_attribute__ (3);

/* Given a Dwarf_Die addr returns a (reconstructed) Dwarf_Die, or NULL
   if the given addr didn't come from a valid Dwarf_Die.  In particular
   it will make sure that the correct Dwarf_CU pointer is set for the
   Dwarf_Die, the Dwarf_Abbrev pointer will not be set up yet (it will
   only be once the Dwarf_Die is used to read attributes, children or
   siblings).  This functions can be used to keep a reference to a
   Dwarf_Die which you want to refer to later.  The addr, and the result
   of this function, is only valid while the associated Dwarf is valid.  */
extern Dwarf_Die *dwarf_die_addr_die (Dwarf *dbg, void *addr,
                                      Dwarf_Die *result)
     __nonnull_attribute__ (3);

One note to keep in mind. Each .dwo is also represented as a Dwarf handle, and a Dwarf_Die from a split dwarf file will have that as Dwarf handle. But the above functions work on the main Dwarf handle (which owns the sub Dwarf handles and will clean them up when you drop the main Dwarf handle).

Cheers,

Mark

(In reply to Thomas Schwinge from comment #8)
> Additionally, in PR25042 I saw Mark's 2019-09-30 comment that "logic [there]
> is broken, [...] when dealing with split-dwarf DWO files", so I suppose that
> needs to be reworked, too?  Per Dodji's 2019-10-01 comment, "this
> optimization will be unnecessary, hopefully".

libdw is designed so you don't need to know the specific DW_FORM that an attribute is encoded in. For strings it expects your simply call dwarf_formstring which provides you with a char * (and libdw takes care of figuring out where precisely this string comes from for you).

libabigail however takes advantage of the fact that normally all strings come from one section .debug_str which is a unique string pool. That allows some optimizations when comparing strings.

But especially with split Dwarf (but also with small strings embedded in .debug_info) there is no longer one .debug_str pool that contains all unique strings (but one per .dwo file).

So you might have to go through the code to see where/how strings are compared to make sure it still does the right thing given multiple string pools.

Maybe if string comparisons are really that important to optimize you'll have to create a new string pool so you get string pointers/offsets that get you the most optimal string compare results.