Dodji Seketeli [Wed, 28 Aug 2024 15:51:14 +0000 (17:51 +0200)]
dwarf-reader: Do not fix ELF symbols for virtual destructors
There are cases in DWARF emitted by old producers where some member
function DIEs lack reference to the ELF symbol they relate to. To
address that issue the DWARF reader sets some functions (with linkage
name and no ELF symbol) to the ELF symbol matching that linkage name.
We should not do that destructors however, because of false positives
and because it seems destructors are well handled even by the old
producers that I had access to.
This finishes to fix the self-comparison of the llvm-libs package of fc37.
* src/abg-dwarf-reader.cc (finish_member_function_reading): Do not
schedule virtual destructors for ELF symbol-related function
fixup.
Dodji Seketeli [Wed, 28 Aug 2024 15:04:17 +0000 (17:04 +0200)]
dwarf-reader,ir,writer: Better support for static member variables
To support the duality concrete/abstract instance of static data
member variables in DWARF, the reader needed a little bit of an
overhaul. That overhaul is necessary to self-compare libLLVM.so, from
llvm-libs, in fc37. Note that that self-compare is done using this
command:
So now, with this patch, the IR of a static data member is constructed
only when the concrete instance DIE of the static data member is
encountered. The abstract instance DIE is then read first, the IR is
built from it, and then the concrete instance DIE is read for
additional attributes that might be needed, like the ELF symbol
attribute.
The patch fixes a number of issues related to static data members
representation in the IR, namely, it puts them all in their own array
in the class_or_union, just like what is done for non-static data
members. The ABIXML writer is updated to emit static data member from
their new storage.
* include/abg-ir.h (class_or_union::get_static_data_members):
Declare new method.
* src/abg-dwarf-reader.cc (add_or_update_class_type): Do not
create any IR for the /declaration/ of static data member here.
Wait for its definition from a concrete instance DIE to create the
IR for it.
(build_ir_node_from_die): In the DW_TAG_variable case, if we are
looking at the concrete instance of a static variable (one that
has a DW_AT_abstract_origin pointing to a static data member) then
build the IR from it and add it to its class scope.
* src/abg-ir-priv.h (class_or_union::priv::static_data_members_):
Define new data member.
(class_or_union::priv::priv): Stick the static data members that
are in class_or_union::priv::data_members_ into
class_or_union::priv::static_data_members_.
* src/abg-ir.cc (maybe_adjust_canonical_type): Make sure static
data members of the canonical type has the same symbols set as the
data members of the canonicalized type.
(class_or_union::add_data_member): Update the new
class_or_union::priv::static_data_members_ when a data member is
added.
(class_or_union::get_non_static_data_members): Fix comments.
(class_or_union::get_static_data_members): Define new member
functions.
(set_member_is_static): Update the new
class_or_union::priv::static_data_members_ when static-ness is
changed.
* src/abg-writer.cc (write_decl_in_scope): Support writing pure
declarations (not types).
(write_translation_unit): Don't forget to write remaining global
variables.
(write_class_decl): Make sure to write static data members first,
before writing the non-static data members.
* tests/data/test-abidiff-exit/test-ld-2.28-210.so--ld-2.28-211.so.txt:
Adjust.
* tests/data/test-annotate/libtest23.so.abi: Likewise.
* tests/data/test-annotate/libtest24-drop-fns-2.so.abi: Likewise.
* tests/data/test-annotate/libtest24-drop-fns.so.abi: Likewise.
* tests/data/test-annotate/test14-pr18893.so.abi: Likewise.
* tests/data/test-annotate/test15-pr18892.so.abi: Likewise.
* tests/data/test-annotate/test17-pr19027.so.abi: Likewise.
* tests/data/test-annotate/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-annotate/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-annotate/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-diff-dwarf-abixml/PR25409-librte_bus_dpaa.so.20.0.abi:
Likewise.
* tests/data/test-diff-dwarf-abixml/test0-pr19026-libvtkIOSQL-6.1.so.1.abi:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report1.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report2.txt:
Likewise.
* tests/data/test-read-dwarf/PR22015-libboost_iostreams.so.abi:
Likewise.
* tests/data/test-read-dwarf/PR22122-libftdc.so.abi: Likewise.
* tests/data/test-read-dwarf/libtest23.so.abi: Likewise.
* tests/data/test-read-dwarf/libtest24-drop-fns-2.so.abi:
Likewise.
* tests/data/test-read-dwarf/libtest24-drop-fns.so.abi: Likewise.
* tests/data/test-read-dwarf/test-libandroid.so.abi: Likewise.
* tests/data/test-read-dwarf/test10-pr18818-gcc.so.abi: Likewise.
* tests/data/test-read-dwarf/test11-pr18828.so.abi: Likewise.
* tests/data/test-read-dwarf/test12-pr18844.so.abi: Likewise.
* tests/data/test-read-dwarf/test14-pr18893.so.abi: Likewise.
* tests/data/test-read-dwarf/test15-pr18892.so.abi: Likewise.
* tests/data/test-read-dwarf/test16-pr18904.so.abi: Likewise.
* tests/data/test-read-dwarf/test17-pr19027.so.abi: Likewise.
* tests/data/test-read-dwarf/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-read-dwarf/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test22-pr19097-libstdc++.so.6.0.17.so.abi:
Likewise.
* tests/data/test-read-dwarf/test9-pr18818-clang.so.abi: Likewise.
* tests/data/test-read-write/test10.xml: Likewise.
Dodji Seketeli [Tue, 27 Aug 2024 15:45:08 +0000 (17:45 +0200)]
dwarf-reader,reader.cc: Fix function virtuality setting
When setting the virtual-ness of a function (whether it's virtual or
not and its vtable offset), one must set the vtable offset before
setting the is_virtual property because setting the later triggers a
fix-up that uses the former.
It turns out some parts of the code got the order wrong. To fix that,
this patch introduce a new set_member_function_virtuality that ensures
things are done in the proper order.
SET_member_function_vtable_offset and set_member_function_is_virtual
are now static functions that are used only by the new
set_member_function_virtuality.
* include/abg-fwd.h (set_member_function_vtable_offset)
(set_member_function_is_virtual): Remove these function
declarations.
(set_member_function_virtuality): Declare new function.
* include/abg-ir.h (decl_base::get_context_rel): Make this public.
(set_member_function_is_virtual)
(set_member_function_vtable_offset): Remove these friend
declaration for the decl_base class.
* src/abg-dwarf-reader.cc (finish_member_function_reading): Use
the new set_member_function_virtuality in lieu of the previous
set_member_function_is_virtual and
set_member_function_vtable_offset.
* src/abg-ir.cc (set_member_function_vtable_offset)
(set_member_function_is_virtual): Make these functions static.
(set_member_function_virtuality): Define new functions.
(class_or_union::add_member_function): Use the new
set_member_function_virtuality in lieu of
set_member_function_is_virtual and
set_member_function_vtable_offset.
* src/abg-reader.cc (build_class_decl): Likewise.
Dodji Seketeli [Tue, 27 Aug 2024 13:09:48 +0000 (15:09 +0200)]
abipkgdiff: Extract devel and main packages in the same directory
abipkgdiff extracts the two devel packages in directories named
"devel_package{1,2}, whereas the main packages are extracted in
directories named package{1,2}. This is wrong because a devel package
should be extracted into the same directory tree as the main package.
For instance, there can be a foo.so symbolic link (coming from the
devel package) that points to the foo-1.0.0.so (coming from the main
package) below usr/lib64/ in the extracted directory. If the devel
and main packages are extracted in different directories then the
symbolic link is going to be broken and some binaries won't be
ABI-compared.
This patch fixes the issue by extracting devel packages into the same
directory as the main one. Then, it makes sure the devel package and
the main one are extracted sequentially, not in parallel since they
are not being extracted into the same directory.
By fixing this, a number of previously hidden issues or even change
output are now exposed. The patch updates these.
* tests/data/test-diff-pkg/gcc-debuginfo-14.2.1-1.fc40.x86_64.rpm:
Add missing debug info package.
* tests/data/Makefile.am: Add it to source distribution.
* tests/test-diff-pkg.cc (in_out_specs): Add the debuginfo package
to the libgm2 test.
* tools/abipkgdiff.cc (main): Extract the devel packages into the
"package1" and "package2" directories.
(extract_package_and_map_its_content): Extract the main and devel
packages sequentially, not in parallel.
(self_compare_prepared_userspace_package): Add more logs.
* tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt:
Adjust.
* tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt:
Likewise
* tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt:
Likewise
* tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt:
Likewise
* tests/data/test-fedabipkgdiff/test4-glib-0.100.2-2.fc20.x86_64.rpm-glib-0.106-1.fc23.x86_64.rpm-report-0.txt:
Likewise
* tests/data/test-fedabipkgdiff/test7-self-compare-from-fc23-dbus-glib-report-0.txt:
Likewise
* tests/data/test-fedabipkgdiff/vte291-0.39.1-1.fc22.x86_64--vte291-0.39.90-1.fc22.x86_64-report-0.txt:
Likewise
Dodji Seketeli [Fri, 23 Aug 2024 14:07:30 +0000 (16:07 +0200)]
dwarf-reader,ir: Fix endless loop while analyzing DWARF from Modula-2
While reading the DWARF of libm2iso.so.19.0.0 from the package
libgm2-14.2.1-1.fc40.x86_64.rpm (Modula-2 library for the gcc-gm2
package), abidw crashes due to a stack overflow, because of an endless
loop while trying to print the type names for DIEs in the DWARF
reader.
This is due to this kind of recursive construct of an anonymous type
at offset 0xf1:
It's a pointer to a type described by the DIE at offset ... 0xf1.
That is a cycle in the graph of that anonymous type, right there.
To name an anonymous struct like this, the DWARF reader constructs its
"flat representation". Namely, the idea is to represent the struct by
a string that looks like:
"struct {type1 data1, type2 data2}"
For that, the DWARF reader walks the DIE tree and emit the flat
representation recursively.
But here, because the DIE tree contains a cycle, the code goes into an
endless loop.
There is a similar issue with function types
Note that later, when the libabigail IR is built, we face a similar
problem.
This patch introduces cycle detection & avoidance in the code that
emit type names in the DWARF reader and in the Libabigail IR. It adds
the libgm2 RPM to the testsuite and self-compares it using abipkgdiff,
in the test-diff-pkg test harness.
* src/abg-dwarf-reader.cc (die_qualified_type_name)
(die_qualified_decl_name, die_qualified_type_name_empty)
(die_function_signature, die_pretty_print_type)
(die_pretty_print_decl, die_pretty_print)
(reader::{get_die_qualified_type_name}): Take a new guard
parameter. Pass it to all the type name construction function,
all the way down to die_class_flat_representation and
die_return_and_parm_names_from_fn_type_die.
(die_qualified_name, die_type_name)
(die_class_or_enum_flat_representation)
(die_class_or_enum_flat_representation)
(reader::{get_die_pretty_type_representation,
get_die_pretty_representation}): Add an overload that takes a
guard parameter and rewrite the previous overload in terms of the
new one.
(die_class_flat_representation)
(die_return_and_parm_names_from_fn_type_die): Take a new guard
parameter. Use it to detect and avoid cycles during the
construction of the type name.
* src/abg-ir-priv.h
(class_or_union::priv::is_printing_flat_representation_): Define
new data member.
(class_or_union::priv::{set,unset,is}_printing_flat_representation):
Define member functions.
(function_type::priv::is_pretty_printing_): Define new data
member.
(function_type::priv::{set,unset,}_is_pretty_printing): Define
member functions.
* src/abg-ir.cc (get_class_or_union_flat_representation)
(add_outer_pointer_to_fn_type_expr): Implement cycle detection and
avoidance for recursive classes and function types.
* tests/data/test-diff-pkg/libgm2-14.2.1-1.fc40.x86_64-self-check-report.txt:
New test reference output.
* tests/data/test-diff-pkg/libgm2-14.2.1-1.fc40.x86_64.rpm: New
input RPM.
* tests/data/test-read-dwarf/test-libandroid.so.abi: Adjust.
* tests/data/Makefile.am: Add the new test material to source
distribution.
* tests/test-diff-pkg.cc (in_out_specs): Add the new test inputs
to the test harness.
Dodji Seketeli [Thu, 22 Aug 2024 15:37:25 +0000 (17:37 +0200)]
abidw: Add a --kmi-stablelist option alongside existing --kmi-whitelist
* doc/manuals/abidw.rst: Update the documentation.
* tools/abidw.cc (display_usage): Add a help string for the new
--kmi-stablelist.
(parse_command): Parse it.
Dodji Seketeli [Thu, 22 Aug 2024 15:17:26 +0000 (17:17 +0200)]
dwarf-reader,tools-utils: Add statistics about built/suppressed functions
When running abidw --verbose, it's useful to have information about
the number of functions and variable that were actually constructed or
suppressed. This is useful to see how effective a suppression
specification that instructs libabigail to drop certains
functions/variables actually is.
This patch emits statistics about the amount of functions/variables
represented in the IR and dropped on the floor.
* src/abg-dwarf-reader.cc (reader::stats): Define new struct.
(reader::stats_): Define a new data member of type stats.
(build_or_get_fn_decl_if_not_suppressed)
(build_or_get_var_decl_if_not_suppressed): Increments the stats.
(reader::read_debug_info_into_corpus): Emit the stats about number
of suppressed/kept functions and variables at the end of the
corpus building.
* src/abg-tools-utils.cc (load_vmlinux_corpus): Emit the status
about the number of functions/variables in the corpus.
Dodji Seketeli [Wed, 21 Aug 2024 10:49:25 +0000 (12:49 +0200)]
abilint: Support --verbose option
This new option is useful to show where time is spent, just like what
abidw --verbose does.
* doc/manuals/abilint.rst: Document the new --verbose option.
* tools/abilint.cc (options::do_log): New data member.
(options::options): Initialize it.
(display_usage): Document it.
(set_reader_options): New static function to set the options of
the reader from user-provided options.
(main): Use the new set_reader_options.
Dodji Seketeli [Wed, 21 Aug 2024 10:34:09 +0000 (12:34 +0200)]
{btf,ctf,dwarf}-reader, ir: Fix self-comparison debugging for corpus groups
When debugging the self comparison of a corpus group, libabigail fails
to detect that two ABI artifacts belong to the same corpus group.
This is because it only takes into account each individual corpus, not
the fact that several corpora can be grouped into one corpus group.
This patch introduces a new function type_originates_from_corpus which
can detect that a type originates from a corpus or a from a corpus
group. That function is used to detect that the type being
canonicalized belongs to the ABIXML corpus group and so its canonical
type must match the one of its originating ELF type.
* src/abg-btf-reader.cc (reader::read_debug_info_into_corpus):
When reading an ELF binary, which is part of a corpus group, for
the purpose of self-comparison, set that corpus group as the
comparison input. Otherwise, if the ELF binary is not part of the
corpus group, set its corpus as the comparison input. This is
done only when we are debugging the self-comparison process by
doing abidw --debug-abidiff.
* src/abg-ctf-reader.cc (reader::read_corpus): Likewise.
* src/abg-dwarf-reader.cc (reader::read_debug_info_into_corpus):
Likewise.
* src/abg-ir-priv.h (type_originates_from_corpus): Declare ...
* src/abg-ir.cc (type_originates_from_corpus): ... new function.
(type_base::get_canonical_type_for): When debugging the
self-comparison process, use the new type_originates_from_corpus
to know if the current type being canonicalized originates from
the second corpus.
Dodji Seketeli [Wed, 21 Aug 2024 10:22:38 +0000 (12:22 +0200)]
{dwarf,btf,ctf}-reader: Set the origin of the corpus group
Apart from the ABIXML reader, readers don't set the origin of a corpus
group when its read. Oops. Fixed thus.
* src/abg-btf-reader.cc (reader::read_corpus): Set the origin of
the corpus group being read.
* src/abg-ctf-reader.cc (reader::read_corpus): Likewise.
* src/abg-dwarf-reader.cc (reader::read_debug_info_into_corpus):
Likewise.
Dodji Seketeli [Wed, 21 Aug 2024 10:15:43 +0000 (12:15 +0200)]
dwarf-reader: Fix building of void, void* and variadic parm types
This patch adds the IR nodes of void, void* and variadic parm types to
their scope just once. Also, it schedules them for (sorting, hashing
and) canonicalization just like other types.
* src/abg-dwarf-reader.cc (build_ir_node_for_void_type)
(build_ir_node_for_void_pointer_type)
(build_ir_node_for_variadic_parameter_type): Add the type node to
its scope just once. Use
reader::schedule_type_for_late_canonicalization to canonicalize
it.
Dodji Seketeli [Fri, 16 Aug 2024 17:40:40 +0000 (19:40 +0200)]
dwarf-reader: Support LLVM's lingo of declaration-ness
In the DWARF emitted by LLVM, a type DIE can have the
DW_AT_declaration attribute present (saying that it's a declaration)
and yet have the DW_AT_byte_size present, implying that the type is
defined, at the same time. OK, in all fairness, a definition is a
declaration too, but then, the DW_AT_declaration attribute becomes a
tad useless on a definition.
Note that in the DWARF emitted by GCC, when a DIE has the
DW_AT_declaration set, if no size attribute is present then Libabigail
considers that it's a declaration, not a definition.
This state of things makes libabigail wrongly think that some types
(especially some enums) are defined, when they are just declarations.
And that leads to self-comparison issues down the road.
For instance, consider this DIE sub-tree from the libLLVM.so of
llvm-libs-17.0.6-5.el9.x86_64:
This "enum Colors" DIE here is a just a declaration, even though it
has a DW_AT_byte_size attribute set. We see that it has no children
DIE.
But then, later, we can see this DIE sub-tree:
[484843] namespace abbrev: 39
name (GNU_strp_alt) "llvm"
[484848] class_type abbrev: 90
containing_type (ref_udata) [484848]
calling_convention (data1) pass_by_reference (4)
name (GNU_strp_alt) "raw_ostream"
byte_size (data1) 64
decl_file (data1) raw_ostream.h (1)
decl_line (data1) 52
[...]
[4848b9] enumeration_type abbrev: 125
name (GNU_strp_alt) "Colors"
byte_size (data1) 4
declaration (flag_present) yes
[4848bf] member abbrev: 108
name (GNU_strp_alt) "RED"
type (ref_udata) [485234]
decl_file (data1) raw_ostream.h (1)
decl_line (data1) 110
external (flag_present) yes
declaration (flag_present) yes
accessibility (data1) public (1)
const_value (sdata) 1 (1)
[4848ca] member abbrev: 108
name (GNU_strp_alt) "GREEN"
type (ref_udata) [485234]
decl_file (data1) raw_ostream.h (1)
decl_line (data1) 111
external (flag_present) yes
declaration (flag_present) yes
accessibility (data1) public (1)
const_value (sdata) 2 (2)
[...]
Here, we see the actual definition of the "enum Colors" represented
before by DIE 0x1745f6. The definition is not linked in any way to
the previous declaration. So the DWARF reader now thinks that both
enum Colors (at DIE 0x1745f6 and 0x4848b9) are two different enums
with the same name. And of course, that can lead to self-comparison
issues down the road.
In this patch, the DWARF reader is instructed to detect that DIE
0x1745f6 is a declaration, by noticing that it has no children DIE.
Once that was done, I noticed that the pass that resolves declarations
to their definitions is broken.
The pass is broken because declarations are schedule to be processed
by that pass too early, before they got added to their lexical scope.
At that point, the declarations have non-qualified names. When the
declaration-resolution time comes (after the ABI corpus is fully
constructed), the definitions of these declarations are added to their
lexical scope and thus have fully-qualified names. So the
declaration-resolution pass looks for definitions by looking up their
non-qualified names (as the declarations had non-qualified names, due
to their early scheduling), and it never finds any because they all
have full-qualified names!
So the patch fixes this by deferring the scheduling of declarations to
the declaration-resolution pass, for after they are added to their
lexical scope.
* src/abg-dwarf-reader.cc (die_is_declaration_only): A DIE is a
declaration if it has the DW_AT_declaration property and if it
either has no size attribute or no children DIE.
(reader:: maybe_schedule_decl_only_type_for_resolution): Define
new function.
(build_enum_type, add_or_update_class_type)
(add_or_update_union_type): Do not schedule a declaration for
declaration-to-definition-resolution pass here.
(build_typedef_type): If the typedef is a naming typedef, schedule
the underlying type for the declaration-to-definition-resolution
pass here.
(build_ir_node_from_die): Schedule the declaration for
declaration-to-definition-resolution pass here, after it's been
added to its scope.
* tests/data/test-diff-filter/test41-report-0.txt: Adjust.
* tests/data/test-read-dwarf/PR22015-libboost_iostreams.so.abi:
Likewise.
* tests/data/test-read-dwarf/PR22122-libftdc.so.abi: Likewise.
* tests/data/test-read-dwarf/test-libandroid.so.abi: Likewise.
Dodji Seketeli [Fri, 16 Aug 2024 17:13:22 +0000 (19:13 +0200)]
ir: Fix name setting of a ptr-to-mbr-type
The name of a pointer to member type is currently set at the time of
construction of the type.
This can be problematic because at the time of construction of the
type, the member type might not be yet fully constructed. That would
result in a name (for the pointer to member type) that is incorrect
and stays incorrect for the lifetime of the ptr-to-mbr-type.
To be correct, the name of the pointer to member type must always
reflect the current state of the member and containing types.
This patch does just that.
The patch turns decl_base::get_name into a virtual member function.
It then creates a ptr_to_mbr_type::get_name virtual member function
that dynamically overloads decl_base::get_name. Then,
ptr_to_mbr_type::get_name just builds and returns the qualified name
of the pointer to member type.
* include/abg-ir.h (decl_base::{g,s}et_name): Make these member
functions virtual.
(ptr_to_mbr_type::get_name): Add new virtual overload.
* src/abg-ir.cc (ptr_to_mbr_type::ptr_to_mbr_type): Do not set the
name of the type here. Just say that it's not anonymous.
(ptr_to_mbr_type::get_name): Define new (virtual) member function.
Dodji Seketeli [Wed, 14 Aug 2024 15:50:44 +0000 (17:50 +0200)]
dwarf-reader: Speed-up decl-only resolution
After the IR is constructed, decl-only classes and enums are resolved
to their definition, for cases where the link between the declaration
and the definition is not obviously stated in the DWARF
representation.
During that resolution, it often happens that types are compared. As
that comparison happens before type canonicalization, the comparison
is structural. So it takes time and it can take forever.
As pre-canonicalization type comparison happens between artifacts that
are from the same binary, we can consider that two types of the same
name and kind having the same source location are equivalent.
This optimization speeds up things A LOT. So much so that
self-comparing the usr/libexec/gcc/s390x-redhat-linux/12/gnat1 binary
(from the gcc-gnat package) now takes less than 4 minutes, down from
more than one hour!
* src/abg-dwarf-reader.cc
(reader::compare_before_canonicalisation): If the two types being
compared are of the same kind and name, and are declared at the
same source location, then they ought to be equivalent.
* tests/data/test-annotate/test14-pr18893.so.abi: Adjust.
* tests/data/test-read-dwarf/test14-pr18893.so.abi: Likewise.
* tests/data/test-diff-pkg/spice-server-0.12.4-19.el7.x86_64-0.12.8-1.el7.x86_64-report-2.txt:
Likewise.
Dodji Seketeli [Tue, 13 Aug 2024 15:20:22 +0000 (17:20 +0200)]
reader: Avoid duplicating recursive types
Sometimes, building a sub-type of a complex type can trigger the
building of the complex type itself. This is in the case of a
recursive complex type where one of its sub-types has the complex type
as a sub-type.
In those cases, the complex type should not be duplicated. This patch
ensures that.
* src/abg-reader.cc (build_qualified_type_decl)
(build_pointer_type_def, build_reference_type_def)
(build_ptr_to_mbr_type, build_subrange_type, build_array_type_def)
(build_enum_type_decl, build_typedef_decl): After a sub-type is
built, check if the complex type we are looking at is built too.
If it is, then return it.
Dodji Seketeli [Tue, 6 Aug 2024 09:21:22 +0000 (11:21 +0200)]
dwarf-reader: Avoid duplicating anonymous member types
Sometimes, anonymous member types are duplicated in some classes read
by the DWARF reader. This is because the DWARF reader doesn't lookup
anonymous member types in their class scope before adding them into
their scope.
This patch teaches the DWARF reader how to name an anonymous class,
union or enum by using its flat pretty representation, e.g:
struct {int blah; char meh;}
That flat representation is used as the name of the anonymous type to
look it up in a given class scope before adding it to that scope. If
the scope already contains the type, then the type is not added.
Now that class scopes have their proper anonymous member types, it
appeared that the filtering of change reports needed a number of
adaptations because we are now seeing things that we were not seeing
before. For instance two new change categories
NON_COMPATIBLE_DISTINCT_CHANGE_CATEGORY and
NON_COMPATIBLE_NAME_CHANGE_CATEGORY are added. The former is for a
change where a given type becomes another type of a different kind in
an non-compatible way. For instance, a struct type becomes an enum
type. The later category is for a change in a name of the type,
resulting in a non-compatible change. Thus, new non-compatible
changes can now be categorized and reported as such.
* include/abg-comp-filter.h (has_void_ptr_to_ptr_change)
(has_harmless_enum_to_int_change)
(has_benign_array_of_unknown_size_change): Declare new functions.
* include/abg-comparison.h (enum diff_category): Add
NON_COMPATIBLE_DISTINCT_CHANGE_CATEGORY,
NON_COMPATIBLE_NAME_CHANGE_CATEGORY enumerators. Update the other
enumerator values. Also, update the EVERYTHING_CATEGORY
enumerator value.
* include/abg-ir.h (class_decl::find_base_class): Document the
name of the parameter.
* src/abg-comp-filter.cc (type_diff_has_cv_qual_change_only)
(is_non_compatible_distinct_change, is_void_ptr_to_ptr): Define
new static functions.
(has_subtype_changes): Remove static function.
(class_diff_has_only_harmless_changes): Do not bother checking if
the class_diff has subtype changes. If there are, the category of
those changes is going to be propagated upward to this current
diff node. Update the comments.
(has_harmless_name_change): A diff node carrying compatible types
or types that differ only from their CV qualifiers should qualify
as having a harmless name change.
(has_harmless_enum_to_int_change): Make this function be
non-static.
(type_diff_has_cv_qual_change_only): Introduce an overload that
takes two ABI artifacts instead of one diff node. Make the diff
node overload use the new one. Simplify logic by essentially
peeling off qualified or typedefs first. Then if what remain is
pointer or array types, look at their underlying types. If the
remaining underlying types are equal then return true.
(has_void_ptr_to_ptr_change): Use the new is_void_ptr_to_ptr
function. This allows the function to detect void* -> pointer and
pointer -> void* changes. And it simplifies the logic.
* src/abg-comparison.cc (distinct_diff::compatible_child_diff):
Build a compatible child diff node iff the two diff subject are
actually compatible meaning, they are equal modulo a typedef.
(get_default_harmful_categories_bitmap): Add the new
abigail::comparison::{NON_COMPATIBLE_DISTINCT_CHANGE_CATEGORY,NON_COMPATIBLE_NAME_CHANGE_CATEGORY}
enumerators to the bitmap of harmful categories.
(operator<<(ostream& o, diff_category c)): Update this to support
emitting the new
abigail::comparison::{NON_COMPATIBLE_DISTINCT_CHANGE_CATEGORY,NON_COMPATIBLE_NAME_CHANGE_CATEGORY}
enumerators.
(class_or_union_diff::priv::count_filtered_{subtype_}changed_dm):
Fix logic for counting local changes.
diff::has_local_changes_to_be_reported is not reliable so do not
rely on it. We might want to remove that function in the end.
(category_propagation_visitor::visit_end): If the current diff
node has a harmless void pointer to pointer, enum to int, or a
benign array of unknown size change, do not propagate the
NON_COMPATIBLE_{NAME,DISTINCT}_CHANGE_CATEGORY that have
necessarily bubbled up from some of their distant children nodes.
* src/abg-dwarf-reader.cc
(maybe_set_member_type_access_specifier)
(get_next_member_sibling_die): Declare this pre-existing static
function.
(get_internal_anonymous_die_name)
(lookup_class_typedef_or_enum_type_from_corpus)
(lookup_class_typedef_or_enum_type_from_corpus): Remove.
(die_return_and_parm_names_from_fn_type_die): Add is_method_type
and return_type_name parameters. Fix the representation of member
functions.
(die_function_signature): Add qualified_name parameter. Adjust
call to die_return_and_parm_names_from_fn_type_die.
(die_type_name, die_enum_flat_representation)
(die_class_flat_representation)
(die_class_or_enum_flat_representation): Define new static
functions.
(reader::get_die_qualified_type_name, die_qualified_type_name):
For anonymous class or enum types, use the new
die_class_or_enum_flat_representation. Adjust the call to
die_return_and_parm_names_from_fn_type_die. For function types fix
the name representation.
(die_pretty_print_type): Adjust call to
die_return_and_parm_names_from_fn_type_die.
(die_pretty_print_decl): Take qualified_name and include_fns
parameters. Use the new die_type_name for variable type names.
Adjust call to die_function_signature.
(die_pretty_print): Adjust call die_pretty_print_decl.
(get_member_child_die): Remove useless vertical space.
(build_enum_underlying_type): Sort this enum underlying type (and
potentially hash it) before canonicalizing it.
(add_or_update_class_type): If an anonymous type is in the global
scope, take that into account when building its internal anonymous
die name. Lookup anonymous member types before adding them to the
class scope to avoid duplicating them in their scope.
(add_or_update_union_type): Likewise, lookup anonymous member
types before adding them to the union scope to avoid duplicating
them in their scope.
(build_subranges_from_array_type_die): Const-ify the input reader.
Adjust call to build_ir_node_from_die. Associate the subrange DIE
to the IR node built.
(build_ir_node_from_die): Get the current corpus from the reader
because it might be set for scope passed to this function, or the
scope might be nullptr. Schedule base types for canonicalization
like all the other types. Also, lookup all class/union types
before adding them to their class scope.
* src/abg-leaf-reporter.cc (leaf_reporter::report): In the
overload for class_or_union_diff, report changed data members and
their sub-types.
* tests/Makefile.am: Do not XFAIL the test runtestabidiffexit. No
more tests are XFAILED. All tests should now pass.
* tests/data/test-abidiff-exit/PR30503/libsdl/libsdl-1.2.60-1.2.64-report.txt:
Adjust.
* tests/data/test-abidiff-exit/PR31513/reported/PR31513-reported-report-1.txt:
Likewise.
* tests/data/test-abidiff-exit/qualifier-typedef-array-report-1.txt:
Likewise.
* tests/data/test-annotate/libtest24-drop-fns-2.so.abi: Likewise.
* tests/data/test-annotate/libtest24-drop-fns.so.abi: Likewise.
* tests/data/test-annotate/test0.abi: Likewise.
* tests/data/test-annotate/test1.abi: Likewise.
* tests/data/test-annotate/test15-pr18892.so.abi: Likewise.
* tests/data/test-annotate/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-annotate/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-annotate/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-annotate/test21-pr19092.so.abi: Likewise.
* tests/data/test-diff-dwarf/PR25058-liblttng-ctl-report-1.txt:
Likewise.
* tests/data/test-diff-dwarf/test4-report.txt: Likewise.
* tests/data/test-diff-dwarf/test5-report.txt: Likewise.
* tests/data/test-diff-filter/test-PR26739-2-report-0.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report0.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report1.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report2.txt:
Likewise.
* tests/data/test-diff-filter/test31-pr18535-libstdc++-report-0.txt:
Likewise.
* tests/data/test-diff-filter/test31-pr18535-libstdc++-report-1.txt:
Likewise.
* tests/data/test-diff-filter/test35-pr18754-no-added-syms-report-0.txt:
Likewise.
* tests/data/test-diff-filter/test41-report-0.txt: Likewise.
* tests/data/test-diff-filter/test5-report.txt: Likewise.
* tests/data/test-diff-pkg/libICE-1.0.6-1.el6.x86_64.rpm--libICE-1.0.9-2.el7.x86_64.rpm-report-0.txt:
Likewise.
* tests/data/test-diff-pkg/spice-server-0.12.4-19.el7.x86_64-0.12.8-1.el7.x86_64-report-2.txt:
Likewise.
* tests/data/test-diff-pkg/tbb-4.1-9.20130314.fc22.x86_64--tbb-4.3-3.20141204.fc23.x86_64-report-0.txt:
Likewise.
* tests/data/test-diff-pkg/tbb-4.1-9.20130314.fc22.x86_64--tbb-4.3-3.20141204.fc23.x86_64-report-1.txt:
Likewise.
* tests/data/test-read-dwarf/PR22015-libboost_iostreams.so.abi:
Likewise.
* tests/data/test-read-dwarf/PR22122-libftdc.so.abi: Likewise.
* tests/data/test-read-dwarf/PR24378-fn-is-not-scope.abi:
Likewise.
* tests/data/test-read-dwarf/PR25007-sdhci.ko.abi: Likewise.
* tests/data/test-read-dwarf/libtest24-drop-fns-2.so.abi:
Likewise.
* tests/data/test-read-dwarf/libtest24-drop-fns.so.abi: Likewise.
* tests/data/test-read-dwarf/test-libaaudio.so.abi: Likewise.
* tests/data/test-read-dwarf/test-libandroid.so.abi: Likewise.
* tests/data/test-read-dwarf/test0.abi: Likewise.
* tests/data/test-read-dwarf/test0.hash.abi: Likewise.
* tests/data/test-read-dwarf/test1.abi: Likewise.
* tests/data/test-read-dwarf/test1.hash.abi: Likewise.
* tests/data/test-read-dwarf/test10-pr18818-gcc.so.abi: Likewise.
* tests/data/test-read-dwarf/test11-pr18828.so.abi: Likewise.
* tests/data/test-read-dwarf/test12-pr18844.so.abi: Likewise.
* tests/data/test-read-dwarf/test15-pr18892.so.abi: Likewise.
* tests/data/test-read-dwarf/test16-pr18904.so.abi: Likewise.
* tests/data/test-read-dwarf/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-read-dwarf/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test21-pr19092.so.abi: Likewise.
* tests/data/test-read-dwarf/test22-pr19097-libstdc++.so.6.0.17.so.abi:
Likewise.
* tests/data/test-read-dwarf/test9-pr18818-clang.so.abi: Likewise.
Dodji Seketeli [Thu, 8 Aug 2024 12:13:27 +0000 (14:13 +0200)]
ir: Cache the result of scope_decl::get_sorted_member_types
Speed up ABIXML emitting by caching the result of
scope_decl::get_sorted_member_types when it's called on a
canonicalized type.
* include/abg-ir.h
(environment::canonicalization_started): Define new member
functions.
* src/abg-ir-priv.h
(environment::priv::canonicalization_started_): Define new data
member.
(canonicalize_types): Call
environment::canonicalization_started(true) right before starting
type canonicalization and call
environment::canonicalization_started(false) right after type
canonicalization is done.
* src/abg-ir.cc (environment::canonicalization_started): Define
new methods.
(environment::canonicalization_is_done): Set the
new canonicalization_started flag to false when the
canonicalization is done.
(scope_decl::priv::clear_sorted_member_types_cache_): Declare new
data member.
(scope_decl::{add_member_decl, insert_member_type}): If we are
adding a type to the scope, then flag the scope as needing to
clear the sorted member types cache.
(scope_decl::get_sorted_member_types): If type canonicalization is
not yet started, then the sorted member types cache needs to be
cleared. If the sorted member types cache is to be cleared, then
clear it. If the cache is empty, fill it. Then return the
content of the cache.
Dodji Seketeli [Thu, 8 Aug 2024 11:12:49 +0000 (13:12 +0200)]
reader: Avoid duplicating member types
When a member type is already present in a class scope, do not add it
again.
* src/abg-reader.cc (read_name): New static function.
(build_class_decl, build_union_decl): If a member type already
exists in the class, do not add it.
Dodji Seketeli [Tue, 14 May 2024 10:40:54 +0000 (12:40 +0200)]
Don't strip typedefs in fn names when pretty-printing and comparing
This removes a kludge that was needed to avoid spurious changes back
when we didn't have hashing based comparison for type
canonicalization.
If, in the future, we remove the type name based comparison speed
optimization during canonicalization, this patch will be even more
relevant. For now, it greatly simplifies the code base already and it
fixes several issues from the type hashing patch, so much so that
runtestdiffpkg and runtestdifffilter now PASS. runtestabidiffexit
still fails so it's still XFAILed.
* src/abg-ir.cc (get_function_type_name, get_method_type_name): Do
not peel typedef from return type.
(equals): In the overload for function_type, do not peel typedef
before comparing return types. In the overload for
function_decl::parameter, do not peel typedef before comparing the
parameter type.
(stream_pretty_representation_of_fn_parms): Do not strip typedef
from parameter types.
(add_outer_pointer_to_fn_type_expr): Likewise for return type.
* tests/Makefile.am: XFAIL the test runtestabidiffexit.
* tests/data/test-abidiff-exit/PR30503/libsdl/libsdl-1.2.60-1.2.64-report.txt:
Adjust.
* tests/data/test-abidiff-exit/PR31513/reported/PR31513-reported-report-1.txt:
Likewise.
* tests/data/test-abidiff-exit/pointer-to-member/test-ptr-to-mbr2-output-1.txt:
Likewise.
* tests/data/test-annotate/test13-pr18894.so.abi: Likewise.
* tests/data/test-annotate/test14-pr18893.so.abi: Likewise.
* tests/data/test-annotate/test15-pr18892.so.abi: Likewise.
* tests/data/test-annotate/test17-pr19027.so.abi: Likewise.
* tests/data/test-annotate/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-annotate/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-annotate/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-annotate/test21-pr19092.so.abi: Likewise.
* tests/data/test-diff-dwarf-abixml/PR25409-librte_bus_dpaa.so.20.0.abi:
Likewise.
* tests/data/test-diff-dwarf-abixml/test0-pr19026-libvtkIOSQL-6.1.so.1.abi:
Likewise.
* tests/data/test-diff-pkg/spice-server-0.12.4-19.el7.x86_64-0.12.8-1.el7.x86_64-report-2.txt:
Likewise.
* tests/data/test-read-dwarf/PR22015-libboost_iostreams.so.abi:
Likewise.
* tests/data/test-read-dwarf/PR22122-libftdc.so.abi: Likewise.
* tests/data/test-read-dwarf/PR25007-sdhci.ko.abi: Likewise.
* tests/data/test-read-dwarf/PR25042-libgdbm-clang-dwarf5.so.6.0.0.abi:
Likewise.
* tests/data/test-read-dwarf/test-libaaudio.so.abi: Likewise.
* tests/data/test-read-dwarf/test-libandroid.so.abi: Likewise.
* tests/data/test-read-dwarf/test10-pr18818-gcc.so.abi: Likewise.
* tests/data/test-read-dwarf/test11-pr18828.so.abi: Likewise.
* tests/data/test-read-dwarf/test12-pr18844.so.abi: Likewise.
* tests/data/test-read-dwarf/test13-pr18894.so.abi: Likewise.
* tests/data/test-read-dwarf/test14-pr18893.so.abi: Likewise.
* tests/data/test-read-dwarf/test15-pr18892.so.abi: Likewise.
* tests/data/test-read-dwarf/test16-pr18904.so.abi: Likewise.
* tests/data/test-read-dwarf/test17-pr19027.so.abi: Likewise.
* tests/data/test-read-dwarf/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-read-dwarf/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test21-pr19092.so.abi: Likewise.
* tests/data/test-read-dwarf/test22-pr19097-libstdc++.so.6.0.17.so.abi:
Likewise.
* tests/data/test-read-dwarf/test9-pr18818-clang.so.abi: Likewise.
Dodji Seketeli [Sun, 17 Dec 2023 15:56:12 +0000 (16:56 +0100)]
comp-filter: Ignore ptr size when detecting void ptr to ptr change
Note that the output of the test runtestdifffilter has been
ameliorated as void pointer to pointer changes are now completely
ignored.
There are still 2 tests that fail after this patch so they are still
XFAILed: runtestdiffpkg and runtestabidiffexit. They
are going to be addressed by subsequent patches.
* src/abg-comp-filter.cc (has_void_ptr_to_ptr_change): Ignore
pointer size when detecting void pointer to pointer change.
* tests/data/test-diff-filter/test47-filter-void-ptr-change-report-0.txt:
Adjust.
Dodji Seketeli [Sun, 17 Dec 2023 15:58:13 +0000 (16:58 +0100)]
comp-filter: Consider lvalue-ness changes on references as harmful
This patch detects an lvalue-ness change on a reference type,
categorizes it as REFERENCE_LVALUENESS_CHANGE_CATEGORY and considers
it as harmful by default.
Note that this patch fixes the parts of runtestdifffilter that have to
do with lvalue-ness changes to reference types, but there is still one
test failing from the testsuite: runtestabidiffexit. This test has
been XFAILed in the test suite.
* include/abg-comparison.h (enum diff_category): Add the new
REFERENCE_LVALUENESS_CHANGE_CATEGORY enumerator. Adjust the value
of the other enumerators.
* src/abg-comparison.cc (get_default_harmful_categories_bitmap):
Consider the new REFERENCE_LVALUENESS_CHANGE_CATEGORY as being
part of the default harmful categories bitmap.
(operator<<(ostream& o, diff_category c)): Support the new
REFERENCE_LVALUENESS_CHANGE_CATEGORY for me.
* include/abg-comp-filter.h (has_lvalue_reference_ness_change):
Declare new function.
* src/abg-comp-filter.cc (has_lvalue_reference_ness_change):
Define new function.
(categorize_harmful_diff_node): Use the new
has_lvalue_reference_ness_change to set the new
REFERENCE_LVALUENESS_CHANGE_CATEGORY.
* tests/Makefile.am: XFAIL runtestabidiffexit.
* tests/data/test-diff-filter/test3-report.txt: Adjust.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report0.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report1.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report2.txt:
Likewise.
* tests/data/test-diff-filter/test35-pr18754-no-added-syms-report-0.txt:
Likewise.
* tests/data/test-diff-filter/test35-pr18754-no-added-syms-report-1.txt:
Likewise.
* tests/data/test-diff-filter/test47-filter-void-ptr-change-report-0.txt:
Likewise.
Dodji Seketeli [Wed, 27 Mar 2024 00:11:02 +0000 (01:11 +0100)]
ir: decl-only classes don't equal fully defined classes under ODR
Until now, when the One Definition Rule is in effect (when analyzing
C++, for instance) we assume that a decl-only class equals all the
fully-defined classes of the same name. Now that we are using type
hashing to determine type equality during type canonicalization, we
cannot keep that assumption, because the hash of a decl-only class
does not necessarily equals the hash of its definition. Actually,
decl-only classes don't have hashes. So this patch removes that
assumption.
Note that even with this patch, "make check" still yields 3 failures
that are still XFAILed in the test suite.
* src/abg-ir.cc (equals): In language supporting the ODR (just
like in all other languages), consider that a decl-only class is
different from its defined class counterpart.
* tests/data/test-diff-dwarf/test28-vtable-changes-report-0.txt:
Adjust.
* tests/data/test-diff-filter/test41-report-0.txt: Likewise.
Dodji Seketeli [Wed, 21 Aug 2024 12:43:00 +0000 (14:43 +0200)]
ir: Remove the now useless type propagation optimization
Now that the type hashing patch is in, I figure we can do away with
the overly complicated type propagation optimization. The type
propagating optimization is disabled by the type hashing patch anyway
so this is overdue now.
The nice thing about this is that we are dropping a big complicated
code with no noticeable speed penalty \o/
* configure.ac: Remove the now useless --enable-ct-propagation
option.
* include/abg-ir.h (environment::do_on_the_fly_canonicalization):
Remove member function declaration.
* src/abg-ir-priv.h (type_base::priv::{depends_on_recursive_type_,
canonical_type_propagated_,
propagated_canonical_type_confirmed_}): Remove data members.
(type_base::priv::priv): Update.
(type_base::priv::{depends_on_recursive_type,
set_depends_on_recursive_type,
set_does_not_depend_on_recursive_type, canonical_type_propagated,
set_canonical_type_propagated,
propagated_canonical_type_confirmed,
set_propagated_canonical_type_confirmed,
clear_propagated_canonical_type}): Remove member functions.
(environment::priv::{types_with_non_confirmed_propagated_ct_,
recursive_types_, types_with_cleared_propagated_ct_,
do_on_the_fly_canonicalization_}): Remove data members.
(environment::priv::priv): Adjust.
(environment::priv::{mark_dependant_types,
mark_dependant_types_compared_until}): Update comment.
(environment::priv::{mark_dependant_types,
mark_dependant_types_compared_until, is_recursive_type,
propagate_ct, confirm_ct_propagation_for_types_dependant_on,
confirm_ct_propagation, confirm_ct_propagation,
types_with_cleared_propagated_ct,
types_with_cleared_propagated_ct,
record_type_with_cleared_propagated_canonical_type,
erase_type_with_cleared_propagated_canonical_type,
collect_types_that_depends_on,
cancel_ct_propagation_for_types_dependant_on,
cancel_ct_propagation, clear_propagated_canonical_type,
add_to_types_with_non_confirmed_propagated_ct,
remove_from_types_with_non_confirmed_propagated_ct,
cancel_all_non_confirmed_propagated_canonical_types}): Remove.
(canonicalize_types): Remove canonical type propagation debugging
code.
* src/abg-ir.cc (mark_dependant_types_compared_until): Remove function.
(RETURN_TRUE_IF_COMPARISON_CYCLE_DETECTED): Adjust, do not use the
removed mark_dependant_types_compared_until.
(return_comparison_result): Remove handling of the canonical type
propagation optimization.
(environment::do_on_the_fly_canonicalization): Remove member
functions.
(maybe_propagate_canonical_type, maybe_propagate_canonical_type)
(maybe_cancel_propagated_canonical_type, ): Remove functions.
(compare_canonical_type_against_candidate, canonicalize): Remove
handling of the canonical type propagation optimization.
(equals): In the overload for class_or_union, adjust call to
return_comparison_result. In the overload for class_decl, remove
call to maybe_cancel_propagated_canonical_type.
(OnTheFlyCanonicalization): Remove this doxygen documentation
module describing the canonical type propagation optimization.
Dodji Seketeli [Tue, 26 Mar 2024 16:36:27 +0000 (17:36 +0100)]
Implement type hashing
The order in which types of the same name & kind (aka homonym types)
are canonicalized does have an impact on the final canonicalization
result. This variation is at the root of the self-comparison error
happening on the gcc-gnat package in f37. The command to reproduce
the issue is:
$ fedabipkgdiff -a --self-compare --from fc37 gcc-gnat
Note that the binary on which self-comparison is failing is gnat1 for
the s390 arch:
That test is still failing with this patch, but is going to pass with
subsequent patches in the series.
Having persistent hashes that can be serialized along with the types
into ABIXML does help to suppress that variation.
This patch implements that idea.
Learning from the teachings of the DWZ tool, the patch uses the xxhash
algorithm for primitive hashing and hash combination. The patch
prevents cycling while hashing type graphs by maintaining a hashing
state in objects being hashed, just like what DWZ does.
The patch cleans out the relics of the previous (failed) attempt of
using hashing in the IR. Now, a virtual
type_or_decl_base::hash_value() is declared and implemented by
decl_base, type_base and descendants of type_base.
Each IR node has a "private" hashing function that knows how to hash
the specific parts of the IR node and combine the hashes of its
sub-type IR children nodes to come up with a hashing value for the
whole IR node, while preventing cycling. There are some subtleties
introduced during the hashing. For instance, a typedef hash is the
same as the hash of its underlying type. This helps the hash value of
a pointer or reference to a typedef of class to be the same as the
hash value of a pointer or reference to the class. Another subtlety
is to never hash the declaration of a class, if a definition is
present; rather, always hash the definition. In general, types that
are not canonicalized (for which is_non_canonicalized_type returns
true) are not hashed.
That private hashing function is thus used by the ::hash_value()
of the IR node to compute the hash value for the first time and cache
it for it subsequent invocations.
After building the IR, the patch first sorts all types, then hashes
them and then canonicalizes them. During the canonicalization phase,
two types with different hashes are deemed different. Two types with
the same hash are compared structurally.
During ABIXML de-serialization, libabigail reads the hash value back
from each IR node and uses that instead of re-calculating the hash.
Note that calculating the hash of a node read from ABIXML could lead
to a different result than calculating the hash for the same node from
DWARF, because the order in which hashes are calculated counts,
especially for recursive types. That is one of the reasons why hashes
are not re-calculated when reading from abixml.
In all fairness, I think we would need to test this hypothesis again,
because we now sort types before hashing them, so in theory, the
hashing should yield the same result as when done from ELF.
Regardless, just reading the hash values from ABIXML is faster than
re-calculating it. We might want to re-calculate the hashes for
debugging purposes, however. This is left as an exercise for the
astute reader of this patch.
If the hash of an ABIXML node equals the hash of an ELF node, then
during type canonicalization, the two nodes are compared structurally.
If the hashes are different, then the nodes compare different during
canonicalization. That speeds up canonicalization at ABIXML reading
time. But this implies that each change in a hashing scheme should
result in a change in the major ABIXML version number, rendering newer
ABIXML files incompatible with previous ones. It even makes them
incompatible with newer libabigail code altogether. In that case, we
might want to teach abidiff about ABIXML versioning, even when
comparing an ABIXML input against an ELF one.
Note that the patch uses the hashing during type canonicalization in
general; that is, if the hashes are present and different, then the
types are different. That helps to do away with the need for the
"canonical type propagation" optimization that I suspect is causing
issues on redundant types.
It turns out that not using that optimization doesn't incur any
noticeable speed penalty so in a subsequent patch, that canonical type
propagation optimization code is going to be entirely removed.
The patch uses an optional value for hashes, to make the difference
between no-hash (for recursive sub-types and types supposed to have no
hash/canonical types) and a hash value of zero which is a valid hash
value.
Note that the patch emits hashes for function-decl elements. This is
the hash of the type of the function. When parsing ABIXML, the reader
sets that function-decl hash to the function_type built from it.
Also, note that for types having the same representation (homonym
types) and yet are canonically different, this patch emits a distinct
canonical type index (aka CTI). That CTI is appended to the hash
value, following a "#" sign.
Later, during type canonicalization, two homonym types T and T' (where
T originates from ELF and T' originates from ABIXML) having the same
hash and CTI will be structurally compared. If T equals T', then T
and T' will be considered canonically equivalent. In other words,
CTIs help to match homonym types originating from ABIXML against
homonym types originating from ELF.
Note that 2 tests are still failing. They are fall-outs that are
addressed in subsequent patches. This patch XFAIL them until they get
fixed by subsequent patches of the series.
* configure.ac: Bump ABIXML version to 4.0. Detect the new xxhash
dependency and require the 0.8.0 version at minimum.
* include/abg-fwd.h (look_through_decl_only_type): Declare new
functions.
(is_scope_decl): Const-ify the parameter and
the returned value. Also, do not include "abg-hash.h" here.
* include/abg-hash.h: Make this include cstdint (for uint64_t) and
abg-ir.h, instead of stdint.h.
(enum hashing_state): Define new enum in the abigail::hashing
namespace.
(combine_hashes, fnv_has, hash, get_hashing_state)
(set_hashing_state, is_recursive_artefact): Declare new functions
in the abigail::hashing namespace.
(struct {decl_base, type_base, type_decl, qualified_type_def,
pointer_type_def, reference_type_def, ptr_to_mbr_type,
array_type_def, enum_type_decl, typedef_decl, function_type,
method_type, member_base, class_or_union, class_decl::base_spec,
class_decl, union_decl}::hash): Declare new hash functors in the
abigail::ir namespace.
* include/abg-ir.h: Remove the inclusion of abg-hash.h from this
header.
(typedef hash_t): Define new type.
(peek_hash_value): Declare new function.
(get_canonical_types): Return a pointer to const vector.
({type_or_decl_base, type_base, qualified_type_def,
pointer_type_def, reference_type_def, ptr_to_mbr_type,
array_type_def::subrange_type, typedef_decl, function_type,
method_type, class_or_union, class_decl, class_decl::base_spec,
union_decl}::hash_value): Add new virtual member functions.
(type_or_decl_base::set_hash_value): Add new member function.
(type_or_decl_base::priv_): Make this data member public.
(peek_hash_value, set_or_get_cached_hash_value): Declare these
functions as friends of class type_or_decl_base.
({decl_base, scope_decl, var_decl, function_decl,
function_decl::parameter}::get_hash): Remove member functions.
* src/abg-comparison-priv.h (types_or_decls_hash::operator()):
Adjust to using the new hash_t type.
* src/abg-ctf-reader.cc
(reader::{additional_types_to_canonicalize, types}): Add new data
members.
(reader::add_type(): Add new member function.
(reader::canonicalize_all_types): Use the new types vector data
member to stash types to be canonicalized and pass them to
ir::hash_and_canonicalize_types for canonicalization.
(process_ctf_base_type)
(build_ir_node_for_variadic_parameter_type)
(build_ir_node_for_void_type, build_ir_node_for_void_pointer_type)
(build_array_ctf_range, process_ctf_enum_type): Do not use
canonicalize anymore. Rather, rely on reader::add_type to
schedule types for canonicalization, doing the generic sorting and
hashing before doing the actually canonicalization.
* src/abg-dwarf-reader.cc ({dwarf_offset_pair_hash,offset_hash,
offset_pair_hash}::operator()): Use the new hashing function.
(reader::read_debug_info_into_corpus): Improve logging. Use
ir::hash_and_canonicalize_types in lieu of ir::canonicalize_types.
(reader::types_to_canonicalize): Add
non-const overload.
(reader::canonicalize_types_scheduled): Add better logs & misc
obvious cleanup.
(maybe_canonicalize_type): Force scheduling canonicalization of
all types at the end of the DWARF processing.
* src/abg-hash.cc (combine_hashes, hash, get_hashing_state)
(set_hashing_state, is_recursive_artefact): Define new functions.
(MAYBE_RETURN_EARLY_FROM_HASHING_TO_AVOID_CYCLES)
(MAYBE_FLAG_TYPE_AS_RECURSIVE)
(MAYBE_RETURN_EARLY_IF_HASH_EXISTS): Define new macros.
(struct {decl_base, type_base, type_decl, qualified_type_def,
pointer_type_def, reference_type_def, ptr_to_mbr_type,
array_type_def, ptr_to_mbr_type, enum_type_decl, typedef_decl,
function_decl, function_type, method_type, member_base,
class_or_union, class_decl::base_spec, class_decl,
union_decl}::hash::operator): Define new hash functors.
({template_parameter, template_decl, non_type_tparameter,
template_tparameter, type_composition, type_composition,
function_tdecl}::{hash, dynamic_hash, shared_ptr_hash): Remove.
* src/abg-ir-priv.h: Don't include abg-ir.h anymore, rather
include abg-hash.h.
(struct type_or_decl_base::priv): Move this here, from abg-ir.cc.
(type_or_decl_base::priv::{hashing_state_, hash_value_,
is_recursive_artefact_}): Define new data members.
(type_or_decl_base::priv::{get_hashing_state, set_hashing_state,
set_hash_value, force_set_hash_value, is_recursive_artefact}):
Define new member functions.
(struct sort_for_hash_functor): Define new functor.
(do_hash_value, set_or_get_cached_hash_value)
(hash_and_canonicalize_types, sort_and_canonicalize_types): Define
new function templates.
(type_is_suitable_for_hash_computing)
(sort_types_for_hash_computing_and_c14n)
(get_canonical_type_index, get_decl_name_for_comparison): Declare
new functions.
(type_base::priv::canonical_type_index): New data member.
(type_base::priv::{priv, clear_propagated_canonical_type}):
Initialize it.
(uint64_t_pair_hash::operator()): Adjust.
(environment::priv::number_of_canonical_types): New data member.
(environment::priv::priv): Initialize it.
(environment::priv::get_new_canonical_type_index): New member
function.
(environment::priv::propagate_ct): Propagate the CTI too.
(environment::priv::{confirm_ct_propagation_for_types_dependant_on,
confirm_ct_propagation}): Assert that canonical type has been
propagated and thus we have a canonical type.
(struct type_topo_comp::operator()): Beef up sorting of types.
Take into account the absolute path of the TU, the hash value, the
CTI)
(struct sort_for_hash_functor): Define new functor.
(sort_types_for_hash_computing_and_c14n): Declare new function
template and new overload.
(canonicalize_types): Take new do_log and show_stats parameters.
Improve logging. Do not sort types in here.
(hash_and_canonicalize_types): Define new function template. This
one does the sorting before the hashing and the canonicalization.
(sort_and_canonicalize_types): Likewise, but this one does no
hashing.
(cache_type_comparison_result): Cache the result of the comparison
now, unconditionally. As we don't do canonical type propagation
anymore, we should not get canonical types and equality to
disagree anymore.
(* src/abg-ir.cc (try_canonical_compare): If hash values are
present and different then the two types are different.
(environment::get_canonical_types): Constify return value.
(environment::get_canonical_type): Adjust.
(struct type_or_decl_base::priv): Move this to abg-ir-priv.h.
(type_or_decl_base::hashing_started): Remove.
({decl_base, scope_decl, var_decl, function_decl,
function_decl::parameter, class_decl::base_spec,
non_type_tparameter, type_composition}::get_hash): Likewise.
({template_parameter}::get_hashing_has_started)
(template_parameter::set_hashing_has_started): Likewise.
(type_or_decl_base::{hash_value, set_hash_value})
({type_base, type_decl, qualified_type_def, pointer_type_def,
reference_type_def, ptr_to_mbr_type,
array_type_def::subrange_type, array_type_def, enum_type_decl,
typedef_decl, function_type, method_type, class_or_union,
class_decl::base_spec, class_decl, union_decl}::hash_value):
Define new member functions.
(type_or_decl_base::set_hash_value): Likewise.
(get_decl_name_for_comparison): Make this non-static.
(is_scope_decl): Constify.
(type_is_suitable_for_hash_computing)
(peek_hash_value, read_type_hash, read_hash_and_stash)
(look_through_decl_only_type)
(candidate_matches_a_canonical_type_hash)
(sort_types_for_hash_computing_and_c14n): Define new functions.
(lookup_pointer_type, lookup_reference_type)
(pointer_type_def::get_qualified_name)
(reference_type_def::get_qualified_name)
(reference_type_def::get_pretty_representation): Adjust.
(get_type_name): Better handle naming of anonymous type decls,
used for enums.
(get_debug_representation): Adjust to emit hashes & CTI of types
as well as member types while debugging.
(look_through_decl_only_type): Rename look_through_decl_only into
this.
(lookup_pointer_type, lookup_reference_type)
(reference_type_def::get_qualified_name): Adjust.
(compare_canonical_type_against_candidate): Stop doing canonical
type propagation during type canonicalization.
(type_base::get_canonical_type_for): Use the new
candidate_matches_a_canonical_type_hash. Set the canonical type
index for homophone canonical types. In debug mode, check that if
a type equals a canonical type, their hash value must match.
(canonicalize): Take new do_log and show_stats parameter. Improve
logging. Also, a type must have the same CTI as its canonical
type.
(hash_type_or_decl): Adjust to use the new type
hash_t.
(type_topo_comp::operator()): Add an overload for type_base_wptr.
If the two pointers are equal, get out early. Otherwise, if
everything else is equal, sort using the absolute path of the
containing translation unit.
(maybe_propagate_canonical_type): Propagate canonical type only if
their CTI match.
* src/abg-reader.cc (maybe_canonicalize_type): Improve logging.
Schedule all types for late canonicalization.
(reader::perform_type_canonicalization): Improve logging. Call
hash_and_canonicalize_types to hash and canonicalize types.
(read_type_hash_and_cti, read_hash_and_stash): Define new static
functions.
(build_function_decl, build_type_decl)
(build_qualified_type_decl)
(build_pointer_type_def, build_reference_type_def)
(build_ptr_to_mbr_type, build_function_type, build_subrange_type)
(build_array_type_def, build_enum_type_decl, build_typedef_decl)
(build_class_decl, build_union_decl): Read and set the hash value
from ABIXML.
(build_function_decl, build_qualified_type_decl)
(build_pointer_type_def, build_reference_type_def)
(build_function_type, build_subrange_type, build_array_type_def)
(build_enum_type_decl, build_typedef_decl, build_class_decl)
(build_union_decl): Read the hash and CTI and set them to the
type.
(build_reference_type_def): Build the referenced type before-hand.
(build_class_tdecl, build_type_tparameter, build_type_composition)
(build_template_tparameter, build_type, handle_type_decl)
(handle_qualified_type_decl, handle_pointer_type_def)
(handle_reference_type_def, handle_function_type)
(handle_array_type_def, handle_enum_type_decl)
(handle_typedef_decl, handle_class_decl, handle_union_decl):
Adjust to the use of the new maybe_canonicalize_type signature.
* src/abg-btf-reader.cc (reader::canonicalize_types): Use the new
hash_and_canonicalize_types defined above. Log the time taken by
type canonicalization.
* src/abg-ctf-reader.cc (reader::{types,
additional_types_to_canonicalize}): New data members
(reader::add_types): New member functions.
(reader::canonicalize_all_types): Use the new
hash_and_canonicalize_types defined above.
(process_ctf_base_type): Do not call canonicalize here.
(build_ir_node_for_variadic_parameter_type)
(build_ir_node_for_void_type)
(build_ir_node_for_void_pointer_type, process_ctf_enum_type):
Likewise, and call reader::add_type instead.
* src/abg-dwarf-reader.cc ({dwarf_offset_pair_hash, offset_hash,
offset_pair_hash}::operator()): Adjust to using the new hash_t
type.
(reader::canonicalize_types_scheduled): Use the new
hash_and_canonicalize_types above.
(maybe_canonicalize_type): Schedule all types for late
canonicalization.
* src/abg-writer.cc (reader::get_id_for_type): Constify the
parameter.
(write_type_hash_and_cti, write_common_type_info)
(write_fn_parm_and_return_types): Define new static functions.
(write_type_decl, write_qualified_type_def)
(write_pointer_type_def, write_reference_type_def)
(write_array_subrange_type, write_array_type_def)
(write_enum_type_decl, write_typedef_decl, write_function_decl)
(write_function_type, write_class_decl_opening_tag)
(write_union_decl_opening_tag): Emit hash value and CTI to the
ABIXML.
* tests/Makefile.am: XFAIL the tests runtestdifffilter and
runtestabidiffexit for now.
* tests/data/test-abidiff-exit/PR30329/PR30329-report-1.txt:
Adjust.
* tests/data/test-annotate/PR29443-missing-xx.o.annotated.abi:
Likewise.
* tests/data/test-annotate/libtest23.so.abi: Likewise.
* tests/data/test-annotate/libtest24-drop-fns-2.so.abi: Likewise.
* tests/data/test-annotate/libtest24-drop-fns.so.abi: Likewise.
* tests/data/test-annotate/test-anonymous-members-0.o.abi:
Likewise.
* tests/data/test-annotate/test-pointer-to-member-1.o.annotated.abi:
Likewise.
* tests/data/test-annotate/test0.abi: Likewise.
* tests/data/test-annotate/test1.abi: Likewise.
* tests/data/test-annotate/test13-pr18894.so.abi: Likewise.
* tests/data/test-annotate/test14-pr18893.so.abi: Likewise.
* tests/data/test-annotate/test15-pr18892.so.abi: Likewise.
* tests/data/test-annotate/test17-pr19027.so.abi: Likewise.
* tests/data/test-annotate/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-annotate/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-annotate/test2.so.abi: Likewise.
* tests/data/test-annotate/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-annotate/test21-pr19092.so.abi: Likewise.
* tests/data/test-annotate/test3.so.abi: Likewise.
* tests/data/test-annotate/test4.so.abi: Likewise.
* tests/data/test-annotate/test5.o.abi: Likewise.
* tests/data/test-annotate/test6.so.abi: Likewise.
* tests/data/test-annotate/test7.so.abi: Likewise.
* tests/data/test-annotate/test8-qualified-this-pointer.so.abi:
Likewise.
* tests/data/test-diff-dwarf-abixml/PR25409-librte_bus_dpaa.so.20.0.abi:
Likewise.
* tests/data/test-diff-dwarf-abixml/test0-pr19026-libvtkIOSQL-6.1.so.1.abi:
Likewise.
* tests/data/test-diff-filter/test-PR26739-2-report-0.txt:
Likewise.
* tests/data/test-diff-filter/test3-report.txt: Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report0.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report1.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report2.txt:
Likewise.
* tests/data/test-diff-filter/test31-pr18535-libstdc++-report-0.txt:
Likewise.
* tests/data/test-diff-filter/test31-pr18535-libstdc++-report-1.txt:
Likewise.
* tests/data/test-diff-filter/test35-pr18754-no-added-syms-report-0.txt:
Likewise.
* tests/data/test-diff-filter/test35-pr18754-no-added-syms-report-1.txt:
Likewise.
* tests/data/test-diff-filter/test43-decl-only-def-change-leaf-report-0.txt:
Likewise.
* tests/data/test-diff-pkg-ctf/gmp-6.x.x86_64-report-0.txt:
Likewise.
* tests/data/test-diff-pkg/tbb-4.1-9.20130314.fc22.x86_64--tbb-4.3-3.20141204.fc23.x86_64-report-0.txt:
Likewise.
* tests/data/test-diff-pkg/tbb-4.1-9.20130314.fc22.x86_64--tbb-4.3-3.20141204.fc23.x86_64-report-1.txt:
Likewise.
* tests/data/test-read-btf/test0.o.abi: Likewise.
* tests/data/test-read-btf/test1.o.abi: Likewise.
* tests/data/test-read-ctf/PR27700/test-PR27700.abi: Likewise.
* tests/data/test-read-ctf/test-PR26568-1.o.abi: Likewise.
* tests/data/test-read-ctf/test-PR26568-2.o.abi: Likewise.
* tests/data/test-read-ctf/test-alias.o.abi: Likewise.
* tests/data/test-read-ctf/test-ambiguous-struct-A.o.hash.abi:
Likewise.
* tests/data/test-read-ctf/test-ambiguous-struct-B.o.hash.abi:
Likewise.
* tests/data/test-read-ctf/test-anonymous-fields.o.abi: Likewise.
* tests/data/test-read-ctf/test-array-mdimension.abi: Likewise.
* tests/data/test-read-ctf/test-array-of-pointers.abi: Likewise.
* tests/data/test-read-ctf/test-array-size.abi: Likewise.
* tests/data/test-read-ctf/test-bitfield-enum.abi: Likewise.
* tests/data/test-read-ctf/test-bitfield.abi: Likewise.
* tests/data/test-read-ctf/test-callback.abi: Likewise.
* tests/data/test-read-ctf/test-callback2.abi: Likewise.
* tests/data/test-read-ctf/test-conflicting-type-syms-a.o.hash.abi:
Likewise.
* tests/data/test-read-ctf/test-conflicting-type-syms-b.o.hash.abi:
Likewise.
* tests/data/test-read-ctf/test-const-array.abi: Likewise.
* tests/data/test-read-ctf/test-dynamic-array.o.abi: Likewise.
* tests/data/test-read-ctf/test-enum-many.o.hash.abi: Likewise.
* tests/data/test-read-ctf/test-enum-symbol.o.hash.abi: Likewise.
* tests/data/test-read-ctf/test-enum.o.abi: Likewise.
* tests/data/test-read-ctf/test-fallback.abi: Likewise.
* tests/data/test-read-ctf/test-forward-type-decl.abi: Likewise.
* tests/data/test-read-ctf/test-functions-declaration.abi:
Likewise.
* tests/data/test-read-ctf/test-linux-module.abi: Likewise.
* tests/data/test-read-ctf/test-list-struct.abi: Likewise.
* tests/data/test-read-ctf/test0.abi: Likewise.
* tests/data/test-read-ctf/test0.hash.abi: Likewise.
* tests/data/test-read-ctf/test1.so.abi: Likewise.
* tests/data/test-read-ctf/test1.so.hash.abi: Likewise.
* tests/data/test-read-ctf/test2.so.abi: Likewise.
* tests/data/test-read-ctf/test2.so.hash.abi: Likewise.
* tests/data/test-read-ctf/test3.so.abi: Likewise.
* tests/data/test-read-ctf/test3.so.hash.abi: Likewise.
* tests/data/test-read-ctf/test4.so.abi: Likewise.
* tests/data/test-read-ctf/test4.so.hash.abi: Likewise.
* tests/data/test-read-ctf/test5.o.abi: Likewise.
* tests/data/test-read-ctf/test7.o.abi: Likewise.
* tests/data/test-read-ctf/test8.o.abi: Likewise.
* tests/data/test-read-ctf/test9.o.abi: Likewise.
* tests/data/test-read-dwarf/PR22015-libboost_iostreams.so.abi:
Likewise.
* tests/data/test-read-dwarf/PR22122-libftdc.so.abi: Likewise.
* tests/data/test-read-dwarf/PR24378-fn-is-not-scope.abi:
Likewise.
* tests/data/test-read-dwarf/PR25007-sdhci.ko.abi: Likewise.
* tests/data/test-read-dwarf/PR25042-libgdbm-clang-dwarf5.so.6.0.0.abi:
Likewise.
* tests/data/test-read-dwarf/PR26261/PR26261-exe.abi: Likewise.
* tests/data/test-read-dwarf/PR27700/test-PR27700.abi: Likewise.
* tests/data/test-read-dwarf/PR28584/PR28584-smv.clang.o.abi:
Likewise.
* tests/data/test-read-dwarf/PR29443-missing-xx.o.abi: Likewise.
* tests/data/test-read-dwarf/PR29692-kdelibs3-libkjava.so.1.0.0.abi:
Likewise.
* tests/data/test-read-dwarf/libtest23.so.abi: Likewise.
* tests/data/test-read-dwarf/libtest24-drop-fns-2.so.abi:
Likewise.
* tests/data/test-read-dwarf/libtest24-drop-fns.so.abi: Likewise.
* tests/data/test-read-dwarf/test-PR26568-1.o.abi: Likewise.
* tests/data/test-read-dwarf/test-PR26568-2.o.abi: Likewise.
* tests/data/test-read-dwarf/test-fallback.abi: Likewise.
* tests/data/test-read-dwarf/test-libaaudio.so.abi: Likewise.
* tests/data/test-read-dwarf/test-libandroid.so.abi: Likewise.
* tests/data/test-read-dwarf/test-pointer-to-member-1.o.abi:
Likewise.
* tests/data/test-read-dwarf/test-suppressed-alias.o.abi:
Likewise.
* tests/data/test-read-dwarf/test0.abi: Likewise.
* tests/data/test-read-dwarf/test0.hash.abi: Likewise.
* tests/data/test-read-dwarf/test1.abi: Likewise.
* tests/data/test-read-dwarf/test1.hash.abi: Likewise.
* tests/data/test-read-dwarf/test10-pr18818-gcc.so.abi: Likewise.
* tests/data/test-read-dwarf/test11-pr18828.so.abi: Likewise.
* tests/data/test-read-dwarf/test12-pr18844.so.abi: Likewise.
* tests/data/test-read-dwarf/test13-pr18894.so.abi: Likewise.
* tests/data/test-read-dwarf/test14-pr18893.so.abi: Likewise.
* tests/data/test-read-dwarf/test15-pr18892.so.abi: Likewise.
* tests/data/test-read-dwarf/test16-pr18904.so.abi: Likewise.
* tests/data/test-read-dwarf/test17-pr19027.so.abi: Likewise.
* tests/data/test-read-dwarf/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-read-dwarf/test2.so.abi: Likewise.
* tests/data/test-read-dwarf/test2.so.hash.abi: Likewise.
* tests/data/test-read-dwarf/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test21-pr19092.so.abi: Likewise.
* tests/data/test-read-dwarf/test22-pr19097-libstdc++.so.6.0.17.so.abi:
Likewise.
* tests/data/test-read-dwarf/test3-alias-1.so.hash.abi: Likewise.
* tests/data/test-read-dwarf/test3-alias-2.so.hash.abi: Likewise.
* tests/data/test-read-dwarf/test3-alias-3.so.hash.abi: Likewise.
* tests/data/test-read-dwarf/test3-alias-4.so.hash.abi: Likewise.
* tests/data/test-read-dwarf/test3.so.abi: Likewise.
* tests/data/test-read-dwarf/test3.so.hash.abi: Likewise.
* tests/data/test-read-dwarf/test4.so.abi: Likewise.
* tests/data/test-read-dwarf/test4.so.hash.abi: Likewise.
* tests/data/test-read-dwarf/test5.o.abi: Likewise.
* tests/data/test-read-dwarf/test5.o.hash.abi: Likewise.
* tests/data/test-read-dwarf/test6.so.abi: Likewise.
* tests/data/test-read-dwarf/test6.so.hash.abi: Likewise.
* tests/data/test-read-dwarf/test7.so.abi: Likewise.
* tests/data/test-read-dwarf/test7.so.hash.abi: Likewise.
* tests/data/test-read-dwarf/test8-qualified-this-pointer.so.abi:
Likewise.
* tests/data/test-read-dwarf/test8-qualified-this-pointer.so.hash.abi:
Likewise.
* tests/data/test-read-dwarf/test9-pr18818-clang.so.abi: Likewise.
* tests/data/test-read-write/test-crc.xml: Likewise.
* tests/data/test-read-write/test0.xml: Likewise.
* tests/data/test-read-write/test1.xml: Likewise.
* tests/data/test-read-write/test10.xml: Likewise.
* tests/data/test-read-write/test11.xml: Likewise.
* tests/data/test-read-write/test12.xml: Likewise.
* tests/data/test-read-write/test13.xml: Likewise.
* tests/data/test-read-write/test14.xml: Likewise.
* tests/data/test-read-write/test15.xml: Likewise.
* tests/data/test-read-write/test16.xml: Likewise.
* tests/data/test-read-write/test17.xml: Likewise.
* tests/data/test-read-write/test18.xml: Likewise.
* tests/data/test-read-write/test19.xml: Likewise.
* tests/data/test-read-write/test2.xml: Likewise.
* tests/data/test-read-write/test20.xml: Likewise.
* tests/data/test-read-write/test21.xml: Likewise.
* tests/data/test-read-write/test22.xml: Likewise.
* tests/data/test-read-write/test23.xml: Likewise.
* tests/data/test-read-write/test24.xml: Likewise.
* tests/data/test-read-write/test25.xml: Likewise.
* tests/data/test-read-write/test26.xml: Likewise.
* tests/data/test-read-write/test27.xml: Likewise.
* tests/data/test-read-write/test28-without-std-fns-ref.xml:
Likewise.
* tests/data/test-read-write/test28-without-std-vars-ref.xml:
Likewise.
* tests/data/test-read-write/test3.xml: Likewise.
* tests/data/test-read-write/test4.xml: Likewise.
* tests/data/test-read-write/test5.xml: Likewise.
* tests/data/test-read-write/test6.xml: Likewise.
* tests/data/test-read-write/test7.xml: Likewise.
* tests/data/test-read-write/test8.xml: Likewise.
* tests/data/test-read-write/test9.xml: Likewise.
Dodji Seketeli [Thu, 22 Aug 2024 15:04:26 +0000 (17:04 +0200)]
dwarf-reader: Fix support of suppression specifications
While looking at something else, I realized that when functions and
variables are to be dropped on the floor by a suppression
specification, there are time where the function is suppressed but not
dropped from the ABI corpus. This is due to some thinkos in the DWARF
reader code. Fixed thus.
* src/abg-dwarf-reader.cc (function_is_suppressed)
(variable_is_suppressed): Do not return too early when the
function doesn't seem suppressed, yet.
* tests/data/test-diff-suppr/test31-report-0.txt: Adjust.
* tests/data/test-diff-suppr/test32-report-1.txt: Adjust.
Dodji Seketeli [Wed, 28 Aug 2024 12:25:43 +0000 (14:25 +0200)]
dwarf-reader: Better support concrete instance functions DIEs
abidiff wrongly reports this change while comparing two binaries[1]:
[C] 'function void g_cclosure_marshal_VOID__BOOLEAN(GClosure*, GValue*, guint, const GValue*, gpointer, gpointer)' at gmarshal.c:188:1 has some indirect sub-type changes:
parameter 2 of type 'GValue*' changed:
entity changed from 'GValue*' to 'typedef guint' at gtypes.h:61:1
type size changed from 64 to 32 (in bits)
parameter 3 of type 'typedef guint' changed:
entity changed from 'typedef guint' to 'GValue*'
type size changed from 32 to 64 (in bits)
parameter 4 of type 'const GValue*' changed:
in pointed to type 'const GValue' at gclosure.h:77:1:
entity changed from 'const GValue' to 'typedef GClosure' at gclosure.h:77:1
type size changed from 192 to 256 (in bits)
parameter 6 of type 'typedef gpointer' changed:
typedef name changed from gpointer to guint at gtypes.h:61:1
underlying type 'void*' changed:
entity changed from 'void*' to 'unsigned int'
type size changed from 0 to 32 (in bits)
parameter 7 of type 'const GValue*' was added
parameter 8 of type 'typedef gpointer' was added
parameter 9 of type 'typedef gpointer' was added
Let's look at the DWARF of the newer binary:
The abstract DIE for the g_cclosure_marshal_VOID__BOOLEAN is:
[ 62c87] subprogram abbrev: 100
external (flag_present) yes
name (strp) "g_cclosure_marshal_VOID__BOOLEAN"
decl_file (implicit_const) gmarshal.c (19)
decl_line (data1) 188
decl_column (implicit_const) 1
prototyped (flag_present) yes
sibling (ref_udata) [ 62d20]
[ 62c8f] formal_parameter abbrev: 28
name (GNU_strp_alt) "closure"
decl_file (data1) gmarshal.c (19)
decl_line (data1) 188
decl_column (data1) 49
type (ref_addr) [ b10]
[ 62c9b] formal_parameter abbrev: 28
name (GNU_strp_alt) "return_value"
decl_file (data1) gmarshal.c (19)
decl_line (data1) 189
decl_column (data1) 49
type (ref_addr) [ 9db]
[ 62ca7] formal_parameter abbrev: 28
name (GNU_strp_alt) "n_param_values"
decl_file (data1) gmarshal.c (19)
decl_line (data1) 190
decl_column (data1) 49
type (ref_addr) [ 4e8]
[ 62cb3] formal_parameter abbrev: 28
name (GNU_strp_alt) "param_values"
decl_file (data1) gmarshal.c (19)
decl_line (data1) 191
decl_column (data1) 49
type (ref_addr) [ 9dd]
[ 62cbf] formal_parameter abbrev: 28
name (GNU_strp_alt) "invocation_hint"
decl_file (data1) gmarshal.c (19)
decl_line (data1) 192
decl_column (data1) 49
type (ref_addr) [ 3a0]
[ 62ccb] formal_parameter abbrev: 28
name (GNU_strp_alt) "marshal_data"
decl_file (data1) gmarshal.c (19)
decl_line (data1) 193
decl_column (data1) 49
type (ref_addr) [ 3a0]
Let's focus on its second parameter. Its DIE is:
[ 62c9b] formal_parameter abbrev: 28
name (GNU_strp_alt) "return_value"
decl_file (data1) gmarshal.c (19)
decl_line (data1) 189
decl_column (data1) 49
type (ref_addr) [ 9db]
Note how the formal parameters of that concrete instance are all in a
different order than the ones of the concrete instance ...
In this case, when reading the concrete instance, we need to get the
IR of the function from the abstract instance DIE and then only read
the potential complementary attributes from this concrete instance.
The DWARF reader mistakenly builds the IR for the function from the
concrete instance in this case, hence the wrong change reported down
the road.
This patch fixes that.
Note that artifacts coming from that bug, reported on IRC have been
added to the libabigail-tests.git test suite.
* src/abg-dwarf-reader.cc (build_ir_node_from_die): If the DIE is
a concrete instance of an abstract one, then get the IR from the
abstract one as it has all the types right.
* tests/data/test-annotate/test14-pr18893.so.abi: Adjust.
* tests/data/test-diff-pkg/PR24690/PR24690-report-0.txt: Likewise.
* tests/data/test-diff-pkg/spice-server-0.12.4-19.el7.x86_64-0.12.8-1.el7.x86_64-report-2.txt:
Likewise.
* tests/data/test-diff-pkg/spice-server-0.12.4-19.el7.x86_64-0.12.8-1.el7.x86_64-report-3.txt:
Likewise.
* tests/data/test-read-dwarf/test-libandroid.so.abi: Likewise.
* tests/data/test-read-dwarf/test14-pr18893.so.abi: Likewise.
Dodji Seketeli [Thu, 8 Aug 2024 09:47:06 +0000 (11:47 +0200)]
ctf-reader: Enumerate dicts in the archive rather than using their name
ctf::reader::process_ctf_archive calls ctf_dict_open with the name of
the (parent) dictionary to open. If that parent dictionary is set to
an unexpected name, then the call to ctf_dict_open fails. This can
happen for instance when at link time, the name of the parent
dictionary is set to an arbitrary name using the
"ctf_link_set_memb_name_changer" function.
This patch enumerates the dictionaries of the archive to avoid having
to know the name of the parent dictionary. The enumeration is done
using the ctf_archive_next function of libctf.
There is currently no binary with an unexpected dictionary name in the
test suite so this patch cannot be tested for that particular case.
I'd be glad to have such binaries added to the test suite.
In the mean time this patch has been tested successfully using "make
fullcheck" on the existing test suite.
* src/abg-ctf-reader.cc (reader::process_ctf_archive): Do not use
ctf_dict_open to open the dictionary by name. Rather, enumerate
the dictionaries of the current archive by using ctf_archive_next
just like what lookup_symbol_in_ctf_archive does. Set the
argument of the skip_parent parameter to "false" to ensure we get
the parent dictionary.
(lookup_symbol_in_ctf_archive): Clean this up to properly
initialize the parameters and to document the arguments to the
parameters of ctf_archive_next. Use nullptr instead of NULL.
* tests/data/test-diff-pkg-ctf/test-rpm-report-1.txt: Adjust.
* tests/data/test-diff-pkg-ctf/test-rpm-report-2.txt: Adjust.
Dodji Seketeli [Fri, 16 Aug 2024 17:02:46 +0000 (19:02 +0200)]
ir: Handle ptr to fn type member with empty void return type
If a pointer to member type points to a function type that has an
empty return type, sometimes that can lead to a crash. This patch
considers that the empty return type is a void type.
* src/abg-ir.cc (add_outer_ptr_to_mbr_type_expr): If the function
type has no return type, consider it as void type.
Dodji Seketeli [Fri, 16 Aug 2024 12:48:39 +0000 (14:48 +0200)]
reader: Avoid empty return type node for a function type IR
Sometimes, the function type IR can have an empty node as return type,
to represent void. This can wreak havoc on some part of the code that
don't expect that. This patch uses a proper void type node for that
instead.
* src/abg-reader.cc (build_function_type): If the return type node
is empty, use a void type node.
Dodji Seketeli [Tue, 6 Aug 2024 08:03:09 +0000 (10:03 +0200)]
ir: Speed up enum comparison
When two enums are naively equal, comparing then member-wise should be
fast. This patch does just that. This speeds up things a lot, in the
context of self-comparison and type canonicalization (from DWARF) when
there are huge enums in the binary.
* src/abg-ir.cc (equals): In the overload for enums, compare
enumerators naively for the case both enums are naively equal.
Dodji Seketeli [Mon, 5 Aug 2024 15:47:01 +0000 (17:47 +0200)]
ir: Support comparing a class_decl against a class_or_union
When a class_decl is compared against a class_or_union type using
class_decl::operator==, that operator systematically fails. This
patch fixes that by comparing the common parts between the two
objects.
* src/abg-ir.cc (class_decl::operator==): Support comparing
against a class_or_union.
Dodji Seketeli [Mon, 5 Aug 2024 14:32:45 +0000 (16:32 +0200)]
ir: Support canonical type adjustments for decl-only class types
When a decl-only class 'D' type (which has an associated class type) is
canonicalized and its canonical type is being adjusted to make sure it
contains all the member functions of 'D', the member functions are not
being taken into account. Fixed thus.
* src/abg-ir.cc (maybe_adjust_canonical_type): Look through
decl-only classes to get their definition.
Dodji Seketeli [Thu, 1 Aug 2024 11:22:34 +0000 (13:22 +0200)]
ir: Fix getting the translation unit for an ABI artifact
Sometimes when the current translation unit is not yet set for a given
artifact, we can infer it by getting the translation unit of the scope
of the artifact. This patch performs that inference when the
translation unit is not yet set for the current artifact.
* include/abg-fwd.h (get_translation_unit): Take a
type_or_decl_base parameter, not a decl_base.
* src/abg-ir.cc (get_translation_unit): Likewise. If no
translation has yet been associated with the ABI artifact, get it
from its scope.
(type_or_decl_base::get_corpus): Use the new get_translation_unit.
(maybe_set_translation_unit): Assert that the translation unit is
non-nil.
Use smart pointers for variables exported from the ABI corpus
It turned out that for some applications using the library, being able
to share the variables returned by
libabigail::ir::corpus::lookup_variable is useful. This patch changes
the API to use shared pointer for variables exported from the ABI
corpus. For the sake of consistency, we should do the same for
function decls, but given the churn, I thought I'd do it for variables
first and see the impact. If it's OK then I'll do it for function
decls too.
* include/abg-comparison.h (string_var_ptr_map): Use a
var_decl_sptr rather than a var_decl*.
* include/abg-corpus.h (corpus::{variable, variables_set}):
Likewise.
(corpus::lookup_variable): Return a var_decl_sptr not a var_decl*.
(corpus::exported_decls_builder::maybe_add_var_to_exported_vars):
Take a var_decl_sptr, not a var_decl*.
* include/abg-fe-iface.h
(fe_iface::add_var_to_exported_or_undefined_decls): Likewise.
* include/abg-ir.h (istring_var_decl_ptr_map_type): Use a
var_decl_sptr rather than a var_decl*.
* src/abg-btf-reader.cc (reader::build_ir_node_from_btf_type):
Adjust call to add_var_to_exported_or_undefined_decls.
* src/abg-comparison-priv.h (var_comp::operator()()): Add a new
overload for var_decl_sptr.
(corpus_diff::priv::{deleted_variable_is_suppressed,
deleted_variable_is_suppressed}): Take a var_decl_sptr not a
var_decl*.
(sort_string_var_ptr_map): Take ...
* src/abg-comparison.cc (sort_string_var_ptr_map): ... a
vector<var_decl_sptr> rather than a vector<var_decl*>.
(corpus_diff::priv::ensure_lookup_tables_populated): Adjust.
(variable_is_suppressed): Likewise.
(corpus_diff::priv::{deleted_variable_is_suppressed,
added_variable_is_suppressed}): Likewise.
* src/abg-corpus-priv.h (str_var_ptr_map_type)
(istr_var_ptr_map_type): Use a var_decl_sptr rather than a
var_decl*.
(corpus::exported_decls_builder::priv::{add_var_to_map,
keep_wrt_id_of_vars_to_keep, keep_wrt_regex_of_vars_to_suppress,
keep_wrt_regex_of_vars_to_keep}): Likewise.
(corpus::exported_decls_builder::priv::vars): Likewise.
* src/abg-corpus.cc
(corpus::exported_decls_builder::maybe_add_var_to_exported_vars):
Likewise.
(var_comp::operator()()): Add an overload for var_decl_sptr.
(corpus::lookup_variable): Return var_decl_sptr rather than
var_decl*.
* src/abg-ctf-reader.cc (reader::process_ctf_archive): Adjust.
* src/abg-dwarf-reader.cc (build_ir_node_from_die): Adjust.
* src/abg-fe-iface.cc
(fe_iface::add_var_to_exported_or_undefined_decls): Take a
var_decl_sptr rather than a var_decl*.
* src/abg-reader.cc (build_class_decl, handle_var_decl): Adjust.
* src/abg-writer.cc (write_translation_unit): Likewise.
* tools/abicompat.cc (var_change::decl, var_change::var_change):
Likewise.
(compare_expected_against_provided_variables): Likewise.
configure: Support the optional 'big-tests' sub-directory
If the 'big-tests' sub-directory (or a symbolic link to it) is present
then this patch configures it.
By default, the patch defines a new git sub-module into the
'big-tests' sub-directory. That git sub-module is the repository
git://sourceware.org/git/libabigail-tests. Users can now add the
--recurse-submodules option to git clone when they clone
git://sourceware.org/git/libabigail, so that it clones git big-tests
submodule as well.
The patch adds a new 'big-tests' target (to the top-most Makefile.am)
which runs "make check" in the big-tests sub-directory.
The patch also adds a new 'full-check' target that runs targets check,
check-self-compare and big-tests.
* Makefile.am: Add big-tests,clean-big-tests and full-check
targets.
* configure.ac: Add an --enable-big-tests option. For now, this
option is mandatory to handle the optional 'big-tests' support.
If the --enable-big-tests option is provided and if the
'big-tests' sub-directory is present, configure the big-tests/
sub-package.
* .gitmodules: New sub-module configuration to get
git://sourceware.org/git/libabigail-tests and stick into ...
* big-tests: ... this directory as a sub-module.
* CONTRIBUTING: Update this to explain how to get the Big Tests
going.
* tests/Makefile.am: Update the test summary for the
check-self-compare target.
abidw: Support the --abidiff option for Linux Kernel trees
It turned out abidw --abidiff (self-comparison) doesn't work when
analyzing a Linux Kernel tree. In other words, abidw cannot
self-compare a Linux Kernel tree. Embarrassing. Fixed thus.
* include/abg-corpus.h (is_corpus_group): Declare ...
* src/abg-corpus.cc (is_corpus_group): ... new function.
* src/abg-ctf-reader.cc (create_reader): Support --debug-abidiff
when the package is configured with --enable-debug-self-comparison.
* tools/abidw.cc (load_corpus_and_write_abixml): Factorize self
comparison code out into ...
(perform_self_comparison): ... this.
(load_kernel_corpus_group_and_write_abixml): Use the new
perform_self_comparison to perform self comparison when the
--abidiff option is provided. Also, support --debug-abidiff when
the package is configured with --enable-debug-self-comparison.
ir,comparison,default-reporter: Consider sub-ranges in array diffs
It appears that the comparison engine does not take changes in the
underlying types of sub-ranges into account when looking for sub-range
changes.
This patch fixes that issue.
Then the patch amends the diff model of array_diff to make it take
into account sub-range diffs as children diff nodes. Then it updates
redundancy propagation to make sure changes to sub-ranges are always
reported even when they are redundant. The patch filters out harmless
name changes to integral types, by default.
* include/abg-comparison.h (array_diff::array_diff): Take a vector
of sub-range diffs as children diff nodes.
(array_diff::{subrange_diffs, any_subrange_diff_to_be_reported}):
Declare new methods.
(is_anonymous_subrange_diff): Declare new function.
* src/abg-comp-filter.cc (integral_type_has_harmless_name_change):
Define new function.
(has_harmless_name_change): Use the new
integral_type_has_harmless_name_change.
* src/abg-comparison-priv.h (array_diff::priv::subrange_diffs_):
Define new data member.
* src/abg-comparison.cc (is_anonymous_class_or_union_diff): Fix
comment.
(is_anonymous_subrange_diff): Define new function.
(array_diff::chain_into_hierarchy): Append sub-range diffs as
children nodes of the array_diff node.
(array_diff::array_diff): Take a vector of sub-range diffs as
children diff nodes.
(array_diff::{subrange_diffs, any_subrange_diff_to_be_reported}):
Define new methods.
(array_diff::has_changes): Take sub-range diffs into account.
(compute_diff): In the overload for array_type_def, compute diffs
for sub-ranges as part of the diff for array_type_defs.
(leaf_diff_node_marker_visitor::visit_begin): Do not consider an
anonymous sub-range diff node as being a leaf diff node to report
about.
(redundancy_marking_visitor::visit_end): Report all sub-range diff
nodes changes; do not propagate their (potential) redundancy to
their array_diff node parent.
* src/abg-default-reporter.cc (default_reporter::report): In the
overload for array_diff, if there is a sub-range change to be
reported, then report it.
* src/abg-ir-priv.h (real_type::base_type): Add SIZE_BASE_TYPE,
SSIZE_BASE_TYPE, BIT_SIZE_BASE_TYPE, SBIT_SIZE_BASE_TYPE,
ARRAY_SIZE_BASE_TYPE enumerators to this enum.
* src/abg-ir.cc (parse_base_real_type): Support parsing real type
names that are __ARRAY_SIZE_TYPE__, sizetype, ssizetype,
bitsizetype and sbitsizetype.
(real_type::to_string): Support IZE_BASE_TYPE, SSIZE_BASE_TYPE,
BIT_SIZE_BASE_TYPE, SBIT_SIZE_BASE_TYPE and ARRAY_SIZE_BASE_TYPE
enumerators.
(equals): In the overload for array_type_def::subrange_type,
compare the underlying types of the sub-ranged. In the overload
for array_type_def, consider sub-range changes mismatch as a local
non-type change. This is so that changes to sub-ranges don't get
filtered out because changes to element types are redundant.
* src/abg-reporter-priv.cc (represent): In the overload for
subrange_diff, clean up reporting of change of bound values.
Also, add reports of changes from non-finite to finite size.
* tests/data/test-abidiff-exit/ada-subrange/test1-ada-subrange/test1-ada-subrange-report-1.txt:
Adjust.
* Tests/data/test-abidiff-exit/ada-subrange/test1-ada-subrange/test1-ada-subrange-report-2.txt:
Likewise.
* tests/data/test-abidiff-exit/ada-subrange/test2-ada-subrange-redundant/test2-ada-subrange-redundant-report-1.txt:
Likewise.
* tests/data/test-abidiff-exit/ada-subrange/test2-ada-subrange-redundant/test2-ada-subrange-redundant-report-2.txt:
Likewise.
* tests/data/test-abidiff-exit/test-allow-type-array-v0--v1-report-1.txt:
Likewise.
* tests/data/test-abidiff-exit/test-allow-type-array-v0--v3-report-1.txt:
Likewise.
* tests/data/test-abidiff/test-PR27985-report.txt: Likewise.
* tests/data/test-diff-dwarf/test10-report.txt: Likewise.
* tests/data/test-diff-dwarf/test11-report.txt: Likewise.
* tests/data/test-diff-dwarf/test35-pr19173-libfoo-long-clang-report-0.txt:
Likewise.
* tests/data/test-diff-dwarf/test35-pr19173-libfoo-long-gcc-report-0.txt:
Likewise.
* tests/data/test-diff-filter/PR24430-fold-qualified-array-report-0.txt:
Likewise.
* tests/data/test-diff-filter/test-PR29811-0-report-1.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report0.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report1.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report2.txt:
Likewise.
* tests/data/test-diff-filter/test35-pr18754-no-added-syms-report-0.txt:
Likewise.
* tests/data/test-diff-pkg/libICE-1.0.6-1.el6.x86_64.rpm--libICE-1.0.9-2.el7.x86_64.rpm-report-0.txt:
Likewise.
* tests/data/test-diff-pkg/nss-3.23.0-1.0.fc23.x86_64-report-0.txt:
Likewise.
* tests/data/test-diff-pkg/tbb-4.1-9.20130314.fc22.x86_64--tbb-4.3-3.20141204.fc23.x86_64-report-0.txt:
Likewise.
* tests/data/test-diff-pkg/tbb-4.1-9.20130314.fc22.x86_64--tbb-4.3-3.20141204.fc23.x86_64-report-1.txt:
Likewise.
* tests/data/test-diff-suppr/test-has-data-member-inserted-at-1-report-1.txt:
Likewise.
* tests/data/test-diff-suppr/test-has-strict-flexible-array-data-member-conversion-report-2.txt:
Likewise.
It turns out what we call integral_type in the libabigail IR is
actually a real_type because it includes real types (float and
double). This patch rights that wrong.
* include/abg-fwd.h (is_real_type): Declare new function.
* src/abg-ctf-reader.cc (process_ctf_base_type): Adjust.
* src/abg-dwarf-reader.cc (die_qualified_type_name)
(build_type_decl): Adjust.
* src/abg-ir-priv.h (class real_type): Rename class integral_type
into this.
* src/abg-ir.cc (get_internal_real_type_name, operator|)
(operator&, operator~, operator|=, operator&=)
(parse_real_type_modifier, parse_base_real_type, parse_real_type)
(real_type::{real_type, get_base_type, get_modifiers,
set_modifiers, to_string, operator string()}):
s/integral_type/real_type.
(type_decl::{type_decl, get_qualified_name,
get_pretty_representation}): Adjust.
(is_real_type): Define new function.
(is_integral_type): Use the new is_real_type.
ir: Don't strip typedefs from parms and return type when comparing fns
As we are now sorting types so that they are
always compared in the same order during canonicalization,
irrespective of the front-end, I believe stripping typedefs from
function parameteres and return types during comparison is no more
necessary. And also, that removes so spurious changes reports.
* src/abg-ir.cc (equals): In the overloads from function_type and
function_decl::parameter, do not strip typedefs off of types
becore comparing them.
* tests/data/test-abidiff-exit/PR30329/PR30329-report-1.txt:
Adjust.
* tests/data/test-abidiff-exit/PR30503/libsdl/libsdl-1.2.60-1.2.64-report.txt:
Adjust.
* tests/data/test-abidiff-exit/PR31513/reported/PR31513-reported-report-1.txt:
Adjust.
* tests/data/test-abidiff-exit/PR31513/reported/PR31513-reported-report-2.txt:
Adjust.
* tests/data/test-annotate/test14-pr18893.so.abi: Adjust.
* tests/data/test-annotate/test15-pr18892.so.abi: Adjust.
* tests/data/test-annotate/test17-pr19027.so.abi: Adjust.
* tests/data/test-annotate/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Adjust.
* tests/data/test-diff-dwarf/test2-report.txt: Adjust.
* tests/data/test-diff-filter/test3-report.txt: Adjust.
* tests/data/test-diff-pkg-ctf/gmp-6.x.x86_64-report-0.txt:
Adjust.
* tests/data/test-read-dwarf/PR25007-sdhci.ko.abi: Adjust.
* tests/data/test-read-dwarf/test-libandroid.so.abi: Adjust.
* tests/data/test-read-dwarf/test14-pr18893.so.abi: Adjust.
* tests/data/test-read-dwarf/test15-pr18892.so.abi: Adjust.
* tests/data/test-read-dwarf/test17-pr19027.so.abi: Adjust.
* tests/data/test-read-dwarf/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Adjust.
* tests/data/test-read-dwarf/test22-pr19097-libstdc++.so.6.0.17.so.abi:
Adjust.
variadic parameter type is not added to its proper scope, leading to
issues down the road. Fixed thus.
* src/abg-ir-priv.h (type_topo_comp::operator()): Do not compare
types using their location anymore. It's unnecessary (now that
types are sorted before canonicalization) and it wreaks havoc with
the new properly constructed variadic parameter types.
* src/abg-reader.cc (build_ir_node_for_variadic_parameter_type):
Define new function. Add variadic parameter type to the global
scope of the current translation unit.
(build_function_parameter, build_type_decl): Use the new
build_ir_node_for_variadic_parameter_type.
* tests/data/test-read-write/test17.xml: Adjust.
* tests/data/test-read-write/test19.xml: Likewise.
* tests/data/test-read-write/test20.xml: Likewise.
* tests/data/test-read-write/test21.xml: Likewise.
* tests/data/test-read-write/test22.xml: Likewise.
* tests/data/test-read-write/test23.xml: Likewise.
* tests/data/test-read-write/test25.xml: Likewise.
* tests/data/test-read-write/test26.xml: Likewise.
* tests/data/test-read-write/test28-without-std-fns-ref.xml:
Likewise.
* tests/data/test-read-write/test28-without-std-vars-ref.xml:
Likewise.
reader: Fix building of void and void pointer types
build_ir_node_for_void_pointer_type and build_ir_node_for_void_type
get the type IR from the environment, add it to the global scope of
the current translation unit and schedule it for type
canonicalization.
This patch change the code of these functions so that the adding to
the scope and the canonicalization are done only on the first
invocation of these functions. Subsequent invocations just return the
type IR.
Otherwise, subsequent invocations were unnecessarily trying to add the
type IR to yet another global scope and were trying to canonicalize it
again.
* src/abg-reader.cc (build_ir_node_for_void_pointer_type)
(build_ir_node_for_void_type): Add the type to its scope and
schedule it for canonicalization just once.
The ABIXML reader currently canonicalizes some (simple) types directly
and schedules canonicalization of other types for after the corpus has
been built. This dual mode of operation was used back then when
ABIXML types were not aggressively de-duplicated as they are now.
Today, type canonicalization needs to happen in the same way for all
readers.
This patch thus schedules type canonicalization for after the creation
of the corpus, in the ABIXML reader.
* src/abg-reader.cc (reader::maybe_canonicalize_type): Remove.
(reader::schedule_type_for_canonicalization): Do not schedule
nullptr types for canonicalization.
(reader::perform_type_canonicalization): Rename
perform_late_type_canonicalizing into this.
(reader::{read_corpus, build_or_get_type_decl}): Adjust call to
maybe_canonicalize_type into schedule_type_for_canonicalization.
(read_translation_unit_from_file)
(read_translation_unit_from_buffer, read_translation_unit)
(build_function_decl, build_ir_node_for_void_type)
(build_array_type_def, build_ir_node_for_void_pointer_type)
(build_class_decl, build_union_decl)
(build_class_tdecl, build_type_tparameter, build_type_composition)
(build_template_tparameter, build_type, handle_type_decl)
(handle_qualified_type_decl, handle_pointer_type_def)
(handle_reference_type_def, handle_function_type)
(handle_array_type_def, handle_enum_type_decl)
(handle_typedef_decl, handle_class_decl, handle_union_decl):
Likewise.
Reading of corpus group is broken in the ABIXML reader. Only the main
corpus is read and then, abixml::reader::read_corpus_group_from_input
fails to read subsequent corpora.
This patch fixes abixml::reader::read_corpus_group_from_input and
makes it call a new abixml::reader::initialize function, much like
what the ELF based readers are doing.
* src/abg-fe-iface.cc (fe_iface::priv::initialize): Do not clear
the corpus group upon initialization.
* src/abg-reader.cc (reader::initialize): Define new function.
Make it call fe_iface::initialize.
(reader::clear_per_corpus_data): Remove.
(reader::read_corpus): Remove call to clear_per_corpus_data.
(reader::read_corpus_group_from_input): After calling
reader::read_corpus and adding the resulting corpus to the group,
get the next sibling XML element node. If it's an "abi-corpus"
node then re-initialize the reader by invoking the new
reader::initialize and move on to read it by invoking
reader::read_corpus again.
* tests/data/test-diff-suppr/test45-abi-report-1.txt: Adjust.
Analyzing a single kernel binary has been broken for a long time, it
seems. For the record, I mean this:
$ abidw --ctf vmlinux > vmlinux.ctf.abi
Doesn't work.
This patch fixes ctf::reader::slurp_elf_info to make it not crash on
binaries that have no CTF sections because there is a .ctfa file in
the same directory.
Also, it fixes ctf::reader::read_corpus to support loading the CTF
archive for individual kernel binaries and not just for entire trees
(corpus groups). In the case of individual kernel binaries, the
archive can be built from the CTF sections of the binary.
* src/abg-ctf-reader.cc (reader::slurp_elf_info): Do not crash on
a kernel binary that has no CTF section because the CTF
information is in a .ctfa file in the same directory.
(reader::read_corpus): Support loading a CTF archive for a single
kernel binary. Also, support loading the CTF archive from the CTF
sections in the binary itself not necessarily from a .ctfa file in
the same directory.
Dodji Seketeli [Fri, 28 Jun 2024 16:33:04 +0000 (18:33 +0200)]
ctf-reader: During re-initialization, only clear canonicalize-able types
The patch
https://sourceware.org/git/?p=libabigail.git;a=commit;h=666555665cc4fcfc8ae19661c489822e0df00ae3
introduced a mistake: It frees the CTF archive object during the
re-initialization of the reader between each binary (kernel or
module). The archive contains CTF type information for a kernel and
all its modules. So it should be kept around until all those binaries
are analyzed. Instead, that patch frees the CTF archive object after
handling each binary. Oops.
This patch fixes that problem by free-ing the CTF archive only when
the reader itself is freed, presumably, after analyzing all binaries.
Similarly, the type map maintained by the reader contains types for
the kernel and all its modules. So it should not be freed at
re-initialization time. Rather, what should be freed is only the
types that are to be canonicalized at the end of processing of a given
binary. So the patch makes the reader maintain types to be
canonicalized in a vector and clears that vector at each
re-initialization.
At re-initialization time, the patch also avoids resetting the corpus
group that the current reader feeds.
* src/abg-ctf-reader.cc (reader::types_to_canonicalize): Add data
member.
(reader::add_type): Add the new type to the vector of types to be
canonicalized. Update comment.
(reader::canonicalize_all_types): Now that the reader maintains
the vector of types to be canonicalized, just pass that vector to
canonicalize_types. reader::types_map contains all the types that
have been created in all the binaries processed by this reader so
far. Many of these types have already been canonicalized at the
end of the analysis of binaries that have already been processed.
Only the types created during the processing of the current binary
have not yet be canonicalized and reader::types_to_canonicalize is
where they are maintained. So reader::types_map should be left
alone by this function.
(reader::initialize): Do not close the CTF archive here. Do not
clear reader::{types_map, corpus_group} either.
(reader::read_corpus): Use nullptr, not NULL. Make sure to not
re-open an archive that has already been opened.
(reader::~reader): Mark a closed archive. This is more cosmetic
than anything else.
corpus: Support adding translation units with empty path
The CTF front-end produces an artificial translation unit with an
empty path for all the types in a given ABI corpus. Adding such a
translation unit to its corpus is conditioned on it having a non-empty
path. Oops. Fixed thus.
* src/abg-corpus.cc (corpus::add): Do not require that the path of
the translation unit be non-empty.
tools-utils,btf-reader: Take modules into account for corpus group
It turns out the BTF front-end won't take kernel modules into account
when analyzing a linux kernel build tree.
This is because it doesn't support split and base BTF objects.
Please note that the BTF API is not yet documented in
https://docs.kernel.org/bpf/btf.html, so my understanding of how it
works comes the source code of the bpftool program available at
https://github.com/libbpf/bpftool and https://github.com/libbpf/libbpf.
When analyzing the vmlinux binary, the btf__parse function returns a
"base BTF object" which contains the type information contained in the
.BTF section of that binary. The memory of the BTF types describing
the types of vmlinux lives in the base BTF object. So the base BTF
object must be kept around for the entire time of the analysis of the
Linux kernel tree (vmlinux + modules).
When analyzing a kernel module however, it's the btf__parse_split
function that is used. That function takes into parameter the "base
BTF object" representing the type information of the corresponding
vmlinux binary and returns a "split BTF object" which contains the
type information contained in the .BTF section of the kernel module.
Please note that the split BTF object does reference the base BTF
object as well. The memory of the BTF types describing the types the
kernel module lives in the split BTF object. Because some types and
decls might be exported by a kernel module and used by another one,
the memory of a given split BTF object might be needed to be kept
alive for the entire time of the analysis of the Linux kernel tree as
well.
It's this base/split model of BTF that is not supported by the BTF
front-end.
This patch introduces support for that model.
At any point in time, the BTF front-end now has one handle to the base
BTF object and one handle to the current split BTF object if we are
looking at kernel module. The base and split BTF objects are kept
around for the entire lifetime of the front-end.
For a given binary, each type is identified by a type ID. The
validity of a given type ID is effective only for that binary. The
same type ID in another binary might designate another type. So
rather than maintaining a map that associates a type ID with a libabigail
IR artifact, the front-end now maintains a map that associates a
*type* with libabigail IR artifact.
With this changes, abidw can now analyze and emit the ABIXML for a
linux kernel tree in 28 seconds, using BTF. It can self-compare a
kernel tree in 36 seconds.
* src/abg-btf-reader.cc (btf_type_to_abi_artifact_map_type):
Rename btf_type_id_to_abi_artifact_map_type into this. Make it
associate a btf_type* to a libabigail IR, rather than a type ID to
a libabigail IR.
(reader::{base_btf_handle_, base_btf_file_name_,
split_btf_handle_, split_btf_file_name_, split_btfs_to_free_}):
Define new data members.
(reader::{btf_handle_}): Remove.
(reader::base_btf_handle): Renamed btf_handle into this. Create
the BTF object using btf__parse. Store the name of the binary
into reader::base_btf_file_name_.
(reader::read_split_btf): Define new member function.
(reader::btf_handle): This is new code for an existing member
function name. Return the current split BTF object handle if
applicable, otherwise create the base BTF object handle.
(reader::btf_type_to_artifacts_): Renamed
btf_type_id_to_artifacts into this.
(reader::associate_artifact_to_btf_type): Renamed
associate_artifact_to_btf_type_id into this.
(reader::schedule_type_for_canonicalization): Do not schedule a
type that has already been canonicalized.
(reader::nr_btf_types): Take a BTF* parameter.
(reader::~reader): Free all the allocated split BTF objects.
(reader::read_debug_info_into_corpus): If we are looking at a
split BTF, then set the first type ID as the one right after the
last type ID of the base BTF.
(reader::build_ir_node_from_btf_type): Clean-up the logic for
building type void type.
(reader::{build_ir_node_for_void_type,
build_ir_node_for_void_pointer_type,
build_ir_node_for_variadic_parameter_type,
build_enum_underlying_type, build_array_type}): Schedule type for
canonicalization at the end of the IR construction, not right
away.
* src/abg-tools-utils.cc (create_best_elf_based_reader): Pass
show_all_types and linux_kernel_mode arguments to
btf::create_reader.
elf-based-reader: Clean up logic of elf_based_reader::read_and_add_corpus_to_group
While looking at something else, I noticed the logic of
elf_based_reader::read_and_add_corpus_to_group wasn't clear. The
corpus should be read (thus created) first, then added to the reader.
Fixed thus.
Ross Burton [Wed, 10 Jul 2024 10:38:05 +0000 (11:38 +0100)]
configure.ac: improve fts checks
The current code checks explicitly for musl and if so uses pkg-config to
find fts-standalone. This is suboptimal because musl may not be the only
libc to need a standalone fts implementation, and fts-standalone is an
old gentoo-specific name[1] so libabigail won't build on any other system
against musl.
Instead, use AC_SEARCH_LIBS to look for fts_open in libc then libfts. Do
this before the check for behaviour just in case someone has a standalone
and broken implementation. I assume that the standalone fts is installed
on the standard search path, which should be a fair assumption.
As we're not using pkg-config anymore the required link flags are now in
LIBS, so remove all instances of FTS_CFLAGS and FTS_LIBS.
* configure.ac: Use AC_SEARCH_LIBS to look for fts_open in libc
and in libfts if necessary. Remove the use of FTS_LIBS which is
now unnecessary.
* src/Makefile.am: Do not use FTF_LIBS as it's now unnecessary.
* tools/Makefile.am: Likewise.
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
Dodji Seketeli [Fri, 21 Jun 2024 15:04:40 +0000 (17:04 +0200)]
btf-reader: Add missing data members reset to reader::initialize
btf::reader::initialize forgets to reset a few data members used
during binary type information loading. This resetting is key to get
the reader ready for being re-used to load a new binary, like a kernel
module. Otherwise, the working set just keeps growing unnecessarily.
* src/abg-btf-reader.cc (btf::reader::initialize): Clear
types_to_canonicalize_, btf_type_id_to_artifacts_ and cur_tu_.
Dodji Seketeli [Fri, 21 Jun 2024 14:49:29 +0000 (16:49 +0200)]
dwarf-reader: Fix reader::initialize to clear per corpus data
After cleaning up re-initialization sequence of the the CTF reader ,
Claudiu Ianculescu suggested that I look into the DWARF reader's
initialization sequence as well. Lo and behold, the
dwarf::reader::read_debug_info_into_corpus calls
reader::clear_per_corpus_data while reader::initialize does not.
This patch moves the call to reader::clear_per_corpus_data into
reader::initialize. It makes the code consistent with what we are now
doing in the CTF reader.
* src/abg-dwarf-reader.cc (reader::initialize): Call
clear_per_corpus_data.
(reader::read_debug_info_into_corpus): Remove the call to
clear_per_corpus_data from here.
Dodji Seketeli [Fri, 21 Jun 2024 10:17:25 +0000 (12:17 +0200)]
ir: Cache the pretty representation used during pre-canonicalization type sorting
Profiling showed that during pre-canonicalization types sorting (which
uses type_topo_comp and decl_topo_comp sorting functors) we spend a
lot of time computing the pretty representation of types and decls
invoked from type_topo_comp::operator() and
decl_topo_comp::operator().
This patch thus uses type_base::get_cached_pretty_representation and
introduces a new similar decl_base::get_cached_pretty_representation
to cache the pretty representation of types and decls and re-use that
representation instead of computing it over and over.
Together with the previous patch of this series, the time spent to
analyze a Linux Kernel tree went from ~ 55 minutes to around 1m15s
using a non-optimized build of libabigail.
* include/abg-ir.h (decl_base::get_cached_pretty_representation):
Declare new member function.
* src/abg-ir-priv.h ({type,decl}_topo_comp::operator()): Use
{type,decl}_base::get_cached_pretty_representation instead of
get_pretty_representation.:
* src/abg-ir.cc (decl_base::priv::{internal_cached_repr_,
cached_repr}): Define new data members.
(decl_base::get_cached_pretty_representation):
Define new member function.
(type_base::get_cached_pretty_representation): Cache pretty
representation even for non-canonicalized types. Fix comments.
Dodji Seketeli [Fri, 21 Jun 2024 10:00:26 +0000 (12:00 +0200)]
ctf-reader: Fix re-initialization of the CTF reader
When analyzing a Linux Kernel tree made of vmlinux and loadable module
binaries, the same (CTF) binary reader is re-used to load every single
binaries in a loop. That can be seen, for instance, by reading the
code of load_vmlinux_corpus in abg-tools-utils.cc.
As part of that process, abigail::ctf::reader::initialize is invoked
prior to using the reader to read type information from each binary.
But then, looking at things a bit closer, I realized that
ctf::reader::initialize is failing to reset the reader::ctfa data
member. That leads to a memory leak that was making things grow
out of proportion.
Also, the resetting code fails to actually clear out the map of types
that are to be sorted and canonicalized. That leads to unnecessarily
sorting huge amounts of types.
The patch address the two points above.
Apart from the obvious gain in code integrity, this patch
significantly reduces the time taken to analyze a Linux Kernel tree.
Please note that the subsequent patch coming after this one will
reduce that time even further.
* src/abg-ctf-reader.cc (reader::reader): Initialize ctfa,
ctf_sect, symtab_sect and strtab_sect data members.
(reader::initialize): In the overload taking no argument, make
sure to free the ctfa data member before setting it to nullptr.
In the overload that takes arguments, make sure to invoke
reader::initialize() to really free the data used. Also, improve
the comments.
ctf-reader: Optimize calling sorting function for functions and variables.
Sorting the functions and variables is done twice. First in the
process_ctf_archive() and second in read_corpus(). However, we need
to call them only once when we do the canonicalization of all types
and functions. Thus, move canonicalization of types in read_corpus()
and perfom only once sorting the functions and types.
* abg-ctf-reader.cc (process_ctf_archive()): Move
canonicalize_all_types(), sort_functions() and sort_variables()...
(read_corpus()): ...here, just after calling process_ctf_archive().
Dodji Seketeli [Fri, 31 May 2024 13:29:48 +0000 (15:29 +0200)]
btf-reader: Fix re-use of the BTF reader for several binaries in a row
When analyzing a Linux Kernel tree, the BTF reader needs to be reset
after each binary (vmlinux or module) read. It turns out the reset
was not being done properly. Fixed thus. Also, I noticed that
reader::read_debug_info_into_corpus would not return an empty corpus
when no BTF information was found in the binary. Fixed as well.
* src/abg-btf-reader.cc (reader::initialize): Free the btf handle
first thing as part of the re-initialization.
(reader::~reader): Once the BTF handle has been freed, set it to
nil to show that it's been deleted.
(reader::read_debug_info_into_corpus): If no BTF handle could be
retrieved then it means no BTF data was found on the binary.
Thus, return an empty corpus.
Dodji Seketeli [Fri, 31 May 2024 12:03:23 +0000 (14:03 +0200)]
abidw: make the --lt option support --btf
When building the ABI for a Linux kernel that has BTF and DWARF debug
info, the --btf option doesn't select the BTF debug info if DWARF is
also present.
Fixed thus.
* tools/abidw.cc (load_kernel_corpus_group_and_write_abixml):
Support --btf.
Dodji Seketeli [Fri, 31 May 2024 11:56:52 +0000 (13:56 +0200)]
tools-utils.cc: Support collecting kernel binary paths build from sources
When 'abidw --lt /some/dir' runs with /some/dir being a linux kernel
tree built from source (as opposed to coming from a distribution),
get_binary_paths_from_kernel_dist misbehaves because it expects to
find modules somewhere under/lib/modules. This patch makes it support
linux kernel trees built from source by looking for modules all over
the place, wherever they might be.
* src/abg-tools-utils.cc (get_binary_paths_from_kernel_dist): Look
for kernel modules under the root directory by default.
Dodji Seketeli [Wed, 15 May 2024 10:40:18 +0000 (12:40 +0200)]
{btf,dwarf,ctf,abixml}-reader: Fix size of subrange type
While looking at something else, it appeared that the process of
setting the size of the array_type_def::subrange_type is wrong for all
front-ends.
For this patch, I went back to read the DWARF 5 specification at
"5.13/ Subrange Type Entries".
Basically, the size of the subrange type as described by the
DW_TAG_subrange_type DIE is set either:
* by the size of the base type designated by the DW_AT_type
attribute.
* or by the DW_AT_byte_size or DW_AT_bit_size attribute that might
be present on the DIE.
That is different from the length property carried by the subrange
type.
The patch updates the four front-rends (DWARF, BTF, CTF and ABIXML) to
fix the setting process of the size of the subrange type and adjusts
the testsuite accordingly.
* src/abg-btf-reader.cc (reader::build_array_type): The size of
the subrange is not defined specifically. Let's set it to the
default address size of the translation unit.
* src/abg-ctf-reader.cc (build_array_ctf_range): If no basis type
was found, then set the size of the subrange type to the default
address size of the translation unit.
* src/abg-dwarf-reader.cc (build_subrange_type): Consider the
DW_AT_byte_size and DW_AT_bit_size attribute of
DW_TAG_subrange_type if present. Otherwise, if the basis type is
present, its size as the size of the subrange type. Otherwise,
use the default address size of the translation unit as the size
of the subrange type.
* src/abg-ir.cc (array_type_def::subrange_type::subrange_type): If
the underlying (basis) type is present, then use its size as the
size of the subrange type. Otherwise the size is zero.
(array_type_def::subrange_type::set_underlying_type): Update the
size of the subrange when setting a new underlying (basis) type.
* src/abg-reader.cc (build_subrange_type): Read and set the size
of the subrange.
* tests/data/test-abidiff-exit/ada-subrange/test1-ada-subrange/test1-ada-subrange-report-1.txt:
Adjust.
* tests/data/test-abidiff-exit/ada-subrange/test1-ada-subrange/test1-ada-subrange-report-2.txt:
Likewise.
* tests/data/test-abidiff-exit/ada-subrange/test2-ada-subrange-redundant/test2-ada-subrange-redundant-report-1.txt:
Likewise.
* tests/data/test-abidiff-exit/ada-subrange/test2-ada-subrange-redundant/test2-ada-subrange-redundant-report-2.txt:
Likewise.
* tests/data/test-diff-pkg/GtkAda-gl-2.24.2-29.fc29.x86_64--2.24.2-30.fc30.x86_64-report-0.txt:
Likewise.
Dodji Seketeli [Wed, 15 May 2024 11:33:20 +0000 (13:33 +0200)]
configure: Fix detection of BTF header to enable the BTF front-end
Cestmir Kalina reported that running configure with --enable-btf with
no BTF header present on the system wrongly leads to the BTF
front-end of libabigail being enabled, and later leads to
compilation/installation errors.
This patch fixes that by ensuring that the ENABLE_BTF variable set by
the --enable-btf option handling code is not reused later during
actual btf.h header file detection code.
* configure.ac: Use a new HAS_BTF_HEADERS to store the detection
status for the btf.h header. Emit a better notice message when
the BTF header file is found.
Dodji Seketeli [Fri, 10 May 2024 11:52:22 +0000 (13:52 +0200)]
btf-reader: Ignore BTF nodes that ought to be ignored
For now, the BTF nodes BTF_KIND_DECL_TAG, BTF_KIND_DATASEC, and
BTF_KIND_UNKN are ignored by libabigail, in practice. We should thus
not abort when we encounter them. Fixed thus. Many thanks to Cestmir
Kalina for reporting this.
* src/abg-btf-reader.cc (build_ir_node_from_btf_type): Ignore and
return nil for node of kind BTF_KIND_DECL_TAG, BTF_KIND_DATASEC,
and BTF_KIND_UNKN. For all other nodes, abort.
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
# Veuillez saisir le message de validation pour vos
modifications. Les lignes # commençant par '#' seront ignorées, et
un message vide abandonne la validation. # # Sur la branche
master # Votre branche est à jour avec 'origin/master'. # #
Modifications qui seront validées : # modifié :
src/abg-btf-reader.cc # # Fichiers non suivis: #
doc/api/libabigail.doxy.bak # patches/ # prtests/ # #
------------------------ >8 ------------------------ # Ne touchez
pas à la ligne ci-dessus. # Tout ce qui suit sera éliminé.
diff --git a/src/abg-btf-reader.cc b/src/abg-btf-reader.cc
index f22e92bf..67206bb4 100644
--- a/src/abg-btf-reader.cc
+++ b/src/abg-btf-reader.cc
@@ -486,12 +486,16 @@ public:
#ifdef WITH_BTF_KIND_TYPE_TAG
case BTF_KIND_TYPE_TAG/* Type Tag */:
+ break;
#endif
#ifdef WITH_BTF_KIND_DECL_TAG
case BTF_KIND_DECL_TAG/* Decl Tag */:
+ break;
#endif
case BTF_KIND_DATASEC/* Section */:
+ break;
case BTF_KIND_UNKN/* Unknown */:
+ break;
default:
ABG_ASSERT_NOT_REACHED;
break;
Dodji Seketeli [Tue, 7 May 2024 01:06:24 +0000 (03:06 +0200)]
dwarf-reader,ir: Unify type sorting & sort types before c14n
This fixes the temporary breakage I have introduced by committing too
soon the patch below that should have been committed after the type
hashing patch:
d26bd3d2 dwarf-reader,ir: Merge member fns of classes
Oops.
That patch introduces some instability in the ABIXML output due to the
fact that sorting of function types in particular is not deterministic
enough. I believe that because the sorting is using the internal
pretty representation of function types, it can introduce instability
in the sorting of two types that are equivalent modulo one typedef,
because the internal pretty representation strips typedefs off of
function return and parameter types. This patch fixes that by using
the type_topo_comp operator used in the IR for sorting types, instead
of having a different sorting routine in the ABIXML writer. Another
instability is due to the fact that method types where there is no
parameter beside the implicit "this" parameter and that are different
due to const-ness of the enclosing class could not be told apart. The
patch fixes that as well.
The patch also sorts types before canonicalizing them in the ABIXML
reader, so that type canonicalization happens in the same order (in
DWARF and ABIXML) to minimize changes in canonicalization results
between DWARF and ABIXML. This should hopefully help minimize
self-comparison errors.
Note that sorting types (based on their pretty representation) before
canonicalizing them might incur some (temporary) slowness that I
believe should be handled by the coming type hashing patch set. If we
see that the slowness is not bearable, we might remove that sorting at
some point. For now, let's keep it for testing purposes.
* src/abg-ctf-reader.cc (reader::canonicalize_all_types): Use a
temporary vector as input for ir::canonicalize_types because it
needs an ordered container as it now sorts types before
canonicalizing them.
* src/abg-dwarf-reader.cc (reader::types_to_canonicalize): Add a
non-const overload for this accessor.
* src/abg-ir-priv.h (compare_using_locations)
(is_ptr_ref_or_qual_type): Add a declaration for these functions
defined in abg-ir.cc.
(struct decl_topo_comp, struct type_topo_comp): Move these here,
from abg-ir.cc.
(canonicalize_types): Sort the types before canonicalizing them.
* src/abg-ir.cc (compare_using_locations): Turn this from static
to non-static, now that it's declared in abg-ir-priv.h.
(struct decl_topo_comp, struct type_topo_comp): Move these into
abg-ir-priv.h.
* src/abg-reader.cc: Include abg-ir-priv.h.
(reader::perform_late_type_canonicalizing): Use
ir::canonicalize_types, rather than canonicalizing types one by
one.
* src/abg-writer.cc: Include abg-ir-priv.h.
(struct type_topo_comp): Remove.
(write_context::sort_types): Use type_topo_comp from abg-ir-priv.h
instead of type_topo_comp that was removed.
* tests/data/test-annotate/test13-pr18894.so.abi: Adjust.
* tests/data/test-annotate/test14-pr18893.so.abi: Likewise.
* tests/data/test-annotate/test15-pr18892.so.abi: Likewise.
* tests/data/test-annotate/test17-pr19027.so.abi: Likewise.
* tests/data/test-annotate/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-annotate/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-annotate/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-annotate/test21-pr19092.so.abi: Likewise.
* tests/data/test-read-dwarf/PR22015-libboost_iostreams.so.abi:
Likewise.
* tests/data/test-read-dwarf/PR22122-libftdc.so.abi: Likewise.
* tests/data/test-read-dwarf/PR25007-sdhci.ko.abi: Likewise.
* tests/data/test-read-dwarf/test-libandroid.so.abi: Likewise.
* tests/data/test-read-dwarf/test10-pr18818-gcc.so.abi: Likewise.
* tests/data/test-read-dwarf/test11-pr18828.so.abi: Likewise.
* tests/data/test-read-dwarf/test12-pr18844.so.abi: Likewise.
* tests/data/test-read-dwarf/test13-pr18894.so.abi: Likewise.
* tests/data/test-read-dwarf/test14-pr18893.so.abi: Likewise.
* tests/data/test-read-dwarf/test15-pr18892.so.abi: Likewise.
* tests/data/test-read-dwarf/test16-pr18904.so.abi: Likewise.
* tests/data/test-read-dwarf/test17-pr19027.so.abi: Likewise.
* tests/data/test-read-dwarf/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-read-dwarf/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test21-pr19092.so.abi: Likewise.
* tests/data/test-read-dwarf/test22-pr19097-libstdc++.so.6.0.17.so.abi:
Likewise.
* tests/data/test-read-dwarf/test9-pr18818-clang.so.abi: Likewise.
* tests/data/test-read-write/test28-without-std-fns-ref.xml:
Likewise.
* tests/data/test-read-write/test28-without-std-vars-ref.xml:
Likewise.
Dodji Seketeli [Tue, 26 Mar 2024 15:20:18 +0000 (16:20 +0100)]
test-read-common: Fix error message
The error message in test_task::run_abidw in test-read-common.cc is
wrong. Fixed thus. Also, emit the error message directly to the standard
error as the error is detected.
* tests/test-read-common.cc (test_task::run_abidw): Fix error
message and emit it to stderr directly.
Dodji Seketeli [Tue, 26 Mar 2024 23:43:15 +0000 (00:43 +0100)]
dwarf-reader,ir: Merge member fns of classes
When looking at something else, I realized that in DWARF, two classes
C and C' of the same name and coming from the same corpus might not
have the same member functions that are defined. This is probably
because for a given translation unit, some compilers emit debug info
only for the member functions that are used somehow.
So, depending on which of C or C' is the canonical class in the end,
we will not necessarily have the same member functions emitted in
ABIXML. That is annoying. So there was a kludge in
maybe_adjust_canonical_type to ensure that the canonical type has all
the member functions of the types belonging to same class of
equivalence.
This patch adds a pass before type canonicalization in the DWARF
reader to ensure that all classes of the same name in a given corpus
have the same member functions. The patch thus removes the now
unnecessary kludge from maybe_adjust_canonical_type and asserts the
impossibility of the member function discrepancy.
* src/abg-dwarf-reader.cc (reader::{copy_missing_member_functions,
contains_anonymous_class, merge_member_functions_of_classes,
merge_member_functions_in_classes_of_same_names}): Define new
member functions.
(reader::read_debug_info_into_corpus): Call the reader::new
merge_member_functions_in_classes_of_same_names.
* src/abg-ir.cc (maybe_adjust_canonical_type): When looking at the
canonical type C of a type T, if they both come from the same
corpus, assert that C has all the member functions of T. This
because the DWARF reader now ensures that all types have the same
name in a given corpus have the same member functions.
* tests/data/test-annotate/test14-pr18893.so.abi: Adjust.
* tests/data/test-annotate/test15-pr18892.so.abi: Likewise.
* tests/data/test-annotate/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-annotate/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-annotate/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test-libandroid.so.abi: Likewise.
* tests/data/test-read-dwarf/test12-pr18844.so.abi: Likewise.
* tests/data/test-read-dwarf/test14-pr18893.so.abi: Likewise.
* tests/data/test-read-dwarf/test15-pr18892.so.abi: Likewise.
* tests/data/test-read-dwarf/test16-pr18904.so.abi: Likewise.
* tests/data/test-read-dwarf/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-read-dwarf/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test22-pr19097-libstdc++.so.6.0.17.so.abi:
Likewise.
* tests/data/test-read-dwarf/test9-pr18818-clang.so.abi: Likewise.
Dodji Seketeli [Fri, 3 May 2024 21:05:26 +0000 (23:05 +0200)]
dwarf-reader,ir: Improve detection of method types & implicit parms
There is a confusion in the DWARF reader about static member functions
and non-static member functions. With recent versions of DWARF, a
member function with the DW_AT_object_pointer attribute is known to
have a this-pointer and thus designates a non-static member function.
DWARF emitted by gcc 4.4.x and previous compiler didn't emit the
DW_AT_object_pointer so we need to look at the first parameter of the
member function, make sure it's artificial and pointers to a pointer
to the same class as the class of the current member function.
This patch avoids to confuse static and non-static member functions by
telling apart types of static member functions, non-static member
function and non member functions. This fixes the representation of
several method types and function declaration signatures across the
test suite.
* include/abg-ir.h (method_type::get_is_for_static_method): Add
new method.
* src/abg-dwarf-reader.cc (fn_die_first_parameter_die)
(member_fn_die_has_this_pointer, die_peel_typedef): Define new
static functions.
(die_this_pointer_is_const): Make this support DIEs representing
const pointers.
(die_function_type_is_method_type): Use the new
member_fn_die_has_this_pointer function rather than (wrongly) open
coding its functionality. If the (member) function DIE has no
this pointer DIE and if the function is at class scope, then we
are looking a static member function.
(finish_member_function_reading): Use the new
method::get_is_for_static_method method rather than open-coding
its functionality.
* src/abg-ir.cc (method_type::get_is_for_static_method): Define
new method.
(type_topo_comp::operator()): In this comparison
operator for types, if two method types have the same string
representation then if one method type is for a static method,
then make the non-static method type come first.
(function_type::get_first_non_implicit_parm): The first parameter
of a function can only carry the "this pointer" if it's
artificial.
* tests/data/test-abidiff-exit/PR31513/reported/PR31513-reported-report-1.txt:
Adjust.
* tests/data/test-abidiff/test-struct1-report.txt: Likewise.
* tests/data/test-abidiff/test-struct1-v0.cc.bi: Likewise.
* tests/data/test-abidiff/test-struct1-v1.cc.bi: Likewise.
* tests/data/test-annotate/test14-pr18893.so.abi: Likewise.
* tests/data/test-annotate/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-annotate/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-annotate/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-diff-dwarf-abixml/PR25409-librte_bus_dpaa.so.20.0.abi:
Likewise.
* tests/data/test-diff-dwarf-abixml/test0-pr19026-libvtkIOSQL-6.1.so.1.abi:
Likewise.
* tests/data/test-diff-dwarf/test42-PR21296-clanggcc-report0.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report0.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report1.txt:
Likewise.
* tests/data/test-diff-filter/test30-pr18904-rvalueref-report2.txt:
Likewise.
* tests/data/test-diff-filter/test35-pr18754-no-added-syms-report-0.txt:
Likewise.
* tests/data/test-diff-filter/test35-pr18754-no-added-syms-report-1.txt:
Likewise.
* tests/data/test-diff-filter/test41-report-0.txt: Likewise.
* tests/data/test-read-dwarf/PR22015-libboost_iostreams.so.abi:
Likewise.
* tests/data/test-read-dwarf/PR22122-libftdc.so.abi: Likewise.
* tests/data/test-read-dwarf/test-libandroid.so.abi: Likewise.
* tests/data/test-read-dwarf/test11-pr18828.so.abi: Likewise.
* tests/data/test-read-dwarf/test12-pr18844.so.abi: Likewise.
* tests/data/test-read-dwarf/test14-pr18893.so.abi: Likewise.
* tests/data/test-read-dwarf/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test19-pr19023-libtcmalloc_and_profiler.so.abi:
Likewise.
* tests/data/test-read-dwarf/test20-pr19025-libvtkParallelCore-6.1.so.abi:
Likewise.
* tests/data/test-read-dwarf/test9-pr18818-clang.so.abi: Likewise.
Dodji Seketeli [Tue, 26 Mar 2024 22:44:54 +0000 (23:44 +0100)]
writer: Fix control of emitting parm names in function types
Emitting parameter names in function types should be controlled by the
set_write_parameter_names function. It turns out it is not. Fixed
thus and adjust testannotate and testreadwarf to avoid emitting
parameter names altogether. This removes some rare randomness in the
tests output.
Dodji Seketeli [Fri, 3 May 2024 13:12:37 +0000 (15:12 +0200)]
ir: Make IR node visitor aware of non-canonicalized types
is_non_canonicalized_type is the function that decides which type
should be canonicalized or not. So, code that expects a type to be
canonicalized should comply with the rule of
is_non_canonicalized_type, aka The Rule. It turns out, the IR node
visitor code fails to comply with The Rule. Fixed thus. Note that
with this commit, the program build/tests/testirwalker now works fine
on itself. Many thanks to Cestmir Kalina who noticed and reported the
issue.
* src/abg-ir.cc (ir_node_visitor::{mark_type_node_as_visited,
type_node_has_been_visited}): Invoke is_non_canonicalized_type to
know if we should expect a type to have a canonical type or not.
ir,writer: Fix function type naming & fn type annotation in ABIXML
While looking at something else, it occurred to me that function type
annotation in ABIXML (as emitted by abidw --annotate) is wrong,
especially when it comes to pointer to member function types. Now
that we support pointer to member function types naming, just using
ir::get_type_name in the annotate function should be enough to do the
trick. And this is what this patch does.
Also, I realized that get_function_type_name was wrongly invoking
get_pretty_representation on the return type of the function type,
instead of just emitting its type name. This patch fixes that as
well.
* src/abg-ir.cc (get_function_type_name, get_method_type_name):
Use get_type_name in lieu of get_pretty_representation.
* src/abg-writer.cc (annotate): In the overload for function_type,
use get_function_name instead of open coding function type
representations.
* tests/data/test-annotate/test-pointer-to-member-1.o.annotated.abi: Adjust.
* tests/data/test-annotate/test13-pr18894.so.abi: Likewise.
* tests/data/test-annotate/test14-pr18893.so.abi: Likewise.
* tests/data/test-annotate/test15-pr18892.so.abi: Likewise.
* tests/data/test-annotate/test17-pr19027.so.abi: Likewise.
* tests/data/test-annotate/test18-pr19037-libvtkRenderingLIC-6.1.so.abi: Likewise.
* tests/data/test-annotate/test19-pr19023-libtcmalloc_and_profiler.so.abi: Likewise.
* tests/data/test-annotate/test20-pr19025-libvtkParallelCore-6.1.so.abi: Likewise.
* tests/data/test-annotate/test21-pr19092.so.abi: Likewise.
* tests/data/test-diff-dwarf/PR25058-liblttng-ctl-report-1.txt: Likewise.
* tests/data/test-diff-pkg-ctf/gmp-6.x.x86_64-report-0.txt: Likewise.
* tests/data/test-diff-pkg/spice-server-0.12.4-19.el7.x86_64-0.12.8-1.el7.x86_64-report-2.txt:
Likewise.
* tests/data/test-read-dwarf/PR25007-sdhci.ko.abi: Likewise.
* tests/data/test-read-dwarf/test-libandroid.so.abi: Likewise.
* tests/data/test-read-dwarf/test11-pr18828.so.abi: Likewise.
* tests/data/test-read-dwarf/test12-pr18844.so.abi: Likewise.
Sam James [Fri, 19 Apr 2024 02:23:11 +0000 (03:23 +0100)]
tests/runtestabidb?.sh.in: Use bash shebang
'function x() { ... }' is a bashism, but /bin/sh may be another shell,
like dash. Just use an explicit /bin/bash shebang as the file relies on
a few other bashisms like 'type'.
* tests/runtestabidb[1-2].sh.in: Use bash shebang.
Signed-off-by: Sam James <sam@gentoo.org> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
The output of fedabipkgidiff doesn't clearly say when two
(sub)packages have no ABI change.
This patch fixes that. The patch also trims unnecessary white spaces
from the output of fedabipkgdiff. As a result, the patch adjusts the
expected reference output files.
* tools/fedabipkgdiff: Adjust copyright year
(abipkgdiff): Strip out trailing spaces from the stdout
string. Emit a clear message when no ABI change was found.
* tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt:
Adjust.
* tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt:
Adjust.
* tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt:
Adjust.
* tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt:
Adjust.
* tests/data/test-fedabipkgdiff/test4-glib-0.100.2-2.fc20.x86_64.rpm-glib-0.106-1.fc23.x86_64.rpm-report-0.txt:
Adjust.
* tests/data/test-fedabipkgdiff/test5-same-dir-dbus-glib-0.100.2-2.fc20.x86_64--dbus-glib-0.106-1.fc23.x86_64-report-0.txt:
Adjust.
* tests/data/test-fedabipkgdiff/test6-nss-util-3.12.6-1.fc14.x86_64--nss-util-3.24.0-2.0.fc25.x86_64-report-0.txt:
Adjust.
* tests/data/test-fedabipkgdiff/test7-self-compare-from-fc23-dbus-glib-report-0.txt:
Adjust.
configure,abidb: Make the libarchive python module optional for abidb
Some distributions don't have the libarchive python module packaged.
This patch makes that module optional and enables abidb to function
without it, as it's only necessary for the support of the '--archive'
command line option of the abidb program.
* configure.ac: Detect the libarchive python module as an optional
dependency for the abidb program. So far, only the git python
module remains a hard dependency.
* tools/abidb (main): Do not support the '--archive' command line
option if the libarchive python module is not present.
While looking at packaging libabigail for EPEL8, it turned out abidb
requires python 3.9 for the type hints on function definitions. Yet,
EPEL8 has an older python.
This patch adds a --disable-abidb option to configure to handle EPEL8.
* configure.ac: Add a --disable-abidb option to configure to
disable the abidb tool and related tests.
git://sourceware.org / libabigail.git / log