[PATCH 00/27] Implement type hashing & fix self-comparing gcc-gnat in fc37
dodji@redhat.com
dodji@redhat.com
Thu Aug 29 15:46:29 GMT 2024
From: Dodji Seketeli <dodji@redhat.com>
This set of patches implements type hashing at the libabigail IR level
before doing type canonicalization. The goal of this scheme is to
hopefully speed-up the type canonicalization process and also to make
it less error prone than what it is today. I believe that the main
source of error of the type canonicalization process is the use of the
"canonical type propagation" optimization. There has been a number of
bugs throughout the years that are related to that optimization and I
believe experience has proven that it becomes increasingly difficult
to maintain.
Hashing types allows the type canonicalization algorithm to quickly
determine when two types are different, as they'd have different hash
values. When the hash values are equal however, structural comparison
is still needed. But that amount of structural comparison can be
drastically reduced.
Once type hashing is implemented and it becomes much easier (and
precise) to determine that two types are different during type
canonicalization, I encountered a number of fixes, while trying to
make to the gcc-gnat package to self-compare. The work that was laid
down also helped tackle the self-comparison of the libLLVM.so binary
as well as many other binaries.
In the end, the 27 patches allow libabigail to be able to self-compare
a lot of binaries than previously. Among the 42 packages that were
failing self-comparison in fc36 at
https://sourceware.org/bugzilla/show_bug.cgi?id=29413#c5, only 5 still
fail today. These are binutils, infinipath-psm, monodoc, OpenMolcas
and qt6-qtbase. I'll probably try to fix self-comparison for these
packages in subsequent fixes now that fundamental things are fixed by
this patch series.
Of course, speed of processing has improved a lot for some packages,
going from near one hour to a matter of minutes in some instances.
Dodji Seketeli (27):
Implement type hashing
ir: Remove the now useless type propagation optimization
ir: decl-only classes don't equal fully defined classes under ODR
comp-filter: Consider lvalue-ness changes on references as harmful
comp-filter: Ignore ptr size when detecting void ptr to ptr change
Don't strip typedefs in fn names when pretty-printing and comparing
reader: Avoid duplicating member types
ir: Cache the result of scope_decl::get_sorted_member_types
dwarf-reader: Avoid duplicating anonymous member types
reader: Avoid duplicating recursive types
dwarf-reader: Speed-up decl-only resolution
ir: Fix name setting of a ptr-to-mbr-type
dwarf-reader: Support LLVM's lingo of declaration-ness
reader: Improve logging in the ABIXML reader
ir: Improve type logging during type canonicalization
tools-utils: Improve logging while reading a Linux kernel
dwarf-reader: Fix building of void, void* and variadic parm types
{dwarf,btf,ctf}-reader: Set the origin of the corpus group
{btf,ctf,dwarf}-reader, ir: Fix self-comparison debugging for corpus groups
abilint: Support --verbose option
big-tests: Update git sub-module
dwarf-reader,tools-utils: Add statistics about built/suppressed functions
abidw: Add a --kmi-stablelist option alongside existing --kmi-whitelist
dwarf-reader,ir: Fix endless loop while analyzing DWARF from Modula-2
abipkgdiff: Extract devel and main packages in the same directory
dwarf-reader,reader.cc: Fix function virtuality setting
dwarf-reader,ir,writer: Better support for static member variables
big-tests | 2 +-
configure.ac | 27 +-
doc/manuals/abidw.rst | 16 +-
doc/manuals/abilint.rst | 4 +
include/abg-comp-filter.h | 12 +
include/abg-comparison.h | 35 +-
include/abg-fwd.h | 28 +-
include/abg-hash.h | 249 +-
include/abg-ir.h | 457 +-
src/abg-btf-reader.cc | 46 +-
src/abg-comp-filter.cc | 245 +-
src/abg-comparison-priv.h | 2 +-
src/abg-comparison.cc | 73 +-
src/abg-ctf-reader.cc | 58 +-
src/abg-dwarf-reader.cc | 2063 +-
src/abg-hash.cc | 1478 +-
src/abg-ir-priv.h | 1329 +-
src/abg-ir.cc | 1602 +-
src/abg-leaf-reporter.cc | 24 +-
src/abg-reader.cc | 401 +-
src/abg-tools-utils.cc | 24 +-
src/abg-writer.cc | 457 +-
tests/data/Makefile.am | 4 +
.../libsdl/libsdl-1.2.60-1.2.64-report.txt | 12 +-
.../test-ptr-to-mbr2-output-1.txt | 1 -
.../qualifier-typedef-array-report-1.txt | 64 +-
.../test-ld-2.28-210.so--ld-2.28-211.so.txt | 2 +-
.../test-abidiff/test-PR18791-report0.txt | 21 +
.../PR29443-missing-xx.o.annotated.abi | 8 +-
tests/data/test-annotate/libtest23.so.abi | 338 +-
.../test-annotate/libtest24-drop-fns-2.so.abi | 377 +-
.../test-annotate/libtest24-drop-fns.so.abi | 377 +-
.../test-anonymous-members-0.o.abi | 34 +-
.../test-pointer-to-member-1.o.annotated.abi | 34 +-
tests/data/test-annotate/test0.abi | 46 +-
tests/data/test-annotate/test1.abi | 44 +-
.../data/test-annotate/test13-pr18894.so.abi | 1264 +-
.../data/test-annotate/test14-pr18893.so.abi | 3435 +-
.../data/test-annotate/test15-pr18892.so.abi | 18646 +++-----
.../data/test-annotate/test17-pr19027.so.abi | 9370 ++--
...st18-pr19037-libvtkRenderingLIC-6.1.so.abi | 10482 +++--
...19-pr19023-libtcmalloc_and_profiler.so.abi | 9933 ++--
tests/data/test-annotate/test2.so.abi | 30 +-
...st20-pr19025-libvtkParallelCore-6.1.so.abi | 9108 ++--
.../data/test-annotate/test21-pr19092.so.abi | 3848 +-
tests/data/test-annotate/test3.so.abi | 4 +-
tests/data/test-annotate/test4.so.abi | 18 +-
tests/data/test-annotate/test5.o.abi | 4 +-
tests/data/test-annotate/test6.so.abi | 18 +-
tests/data/test-annotate/test7.so.abi | 52 +-
.../test8-qualified-this-pointer.so.abi | 16 +-
.../PR25409-librte_bus_dpaa.so.20.0.abi | 2796 +-
.../test0-pr19026-libvtkIOSQL-6.1.so.1.abi | 7781 ++--
.../PR25058-liblttng-ctl-report-1.txt | 15 +-
.../test28-vtable-changes-report-0.txt | 3 +
tests/data/test-diff-dwarf/test4-report.txt | 6 +-
.../test42-PR21296-clanggcc-report0.txt | 6 +-
.../test-diff-dwarf/test46-rust-report-0.txt | 2 -
tests/data/test-diff-dwarf/test5-report.txt | 4 +-
.../test-PR26739-2-report-0.txt | 16 +-
.../test30-pr18904-rvalueref-report0.txt | 56 +-
.../test30-pr18904-rvalueref-report1.txt | 58 +-
.../test30-pr18904-rvalueref-report2.txt | 58 +-
.../test31-pr18535-libstdc++-report-0.txt | 2 +-
.../test31-pr18535-libstdc++-report-1.txt | 2 +-
.../test35-pr18754-no-added-syms-report-0.txt | 56 +-
.../test35-pr18754-no-added-syms-report-1.txt | 2 +-
.../data/test-diff-filter/test41-report-0.txt | 40 +-
...t43-decl-only-def-change-leaf-report-0.txt | 5 +
...test47-filter-void-ptr-change-report-0.txt | 3 +
tests/data/test-diff-filter/test5-report.txt | 9 +-
tests/data/test-diff-filter/test7-report.txt | 1 -
.../gcc-debuginfo-14.2.1-1.fc40.x86_64.rpm | Bin 0 -> 122904364 bytes
...libICE-1.0.9-2.el7.x86_64.rpm-report-0.txt | 13 +-
...14.2.1-1.fc40.x86_64-self-check-report.txt | 5 +
.../libgm2-14.2.1-1.fc40.x86_64.rpm | Bin 0 -> 223952 bytes
.../libgm2-debuginfo-14.2.1-1.fc40.x86_64.rpm | Bin 0 -> 441515 bytes
...l7.x86_64-0.12.8-1.el7.x86_64-report-2.txt | 26 +-
...bb-4.3-3.20141204.fc23.x86_64-report-0.txt | 43 +-
...bb-4.3-3.20141204.fc23.x86_64-report-1.txt | 29 +-
...0-from-fc20-to-fc23-dbus-glib-report-0.txt | 26 +
...dbus-glib-0.106-1.fc23.x86_64-report-0.txt | 11 +
....fc20--dbus-glib-0.106-1.fc23-report-0.txt | 26 +
...--dbus-glib-0.106-1.fc23.i686-report-0.txt | 15 +
...-glib-0.106-1.fc23.x86_64.rpm-report-0.txt | 11 +
...f-compare-from-fc23-dbus-glib-report-0.txt | 2 +
...-vte291-0.39.90-1.fc22.x86_64-report-0.txt | 1 +
tests/data/test-read-btf/test0.o.abi | 74 +-
tests/data/test-read-btf/test1.o.abi | 12 +-
.../test-read-ctf/PR27700/test-PR27700.abi | 10 +-
tests/data/test-read-ctf/test-PR26568-1.o.abi | 18 +-
tests/data/test-read-ctf/test-PR26568-2.o.abi | 16 +-
tests/data/test-read-ctf/test-alias.o.abi | 6 +-
.../test-ambiguous-struct-A.o.hash.abi | 20 +-
.../test-ambiguous-struct-B.o.hash.abi | 10 +-
.../test-read-ctf/test-anonymous-fields.o.abi | 14 +-
.../test-read-ctf/test-array-mdimension.abi | 16 +-
.../test-read-ctf/test-array-of-pointers.abi | 24 +-
tests/data/test-read-ctf/test-array-size.abi | 18 +-
.../data/test-read-ctf/test-bitfield-enum.abi | 8 +-
tests/data/test-read-ctf/test-bitfield.abi | 10 +-
tests/data/test-read-ctf/test-callback.abi | 10 +-
tests/data/test-read-ctf/test-callback2.abi | 12 +-
.../test-conflicting-type-syms-a.o.hash.abi | 10 +-
.../test-conflicting-type-syms-b.o.hash.abi | 8 +-
tests/data/test-read-ctf/test-const-array.abi | 12 +-
.../test-read-ctf/test-dynamic-array.o.abi | 18 +-
.../test-read-ctf/test-enum-many.o.hash.abi | 8 +-
.../test-read-ctf/test-enum-symbol.o.hash.abi | 6 +-
tests/data/test-read-ctf/test-enum.o.abi | 8 +-
tests/data/test-read-ctf/test-fallback.abi | 4 +-
.../test-read-ctf/test-forward-type-decl.abi | 14 +-
.../test-functions-declaration.abi | 14 +-
.../data/test-read-ctf/test-linux-module.abi | 38 +-
tests/data/test-read-ctf/test-list-struct.abi | 8 +-
tests/data/test-read-ctf/test0.abi | 40 +-
tests/data/test-read-ctf/test0.hash.abi | 40 +-
tests/data/test-read-ctf/test1.so.abi | 18 +-
tests/data/test-read-ctf/test1.so.hash.abi | 18 +-
tests/data/test-read-ctf/test2.so.abi | 18 +-
tests/data/test-read-ctf/test2.so.hash.abi | 18 +-
tests/data/test-read-ctf/test3.so.abi | 4 +-
tests/data/test-read-ctf/test3.so.hash.abi | 4 +-
tests/data/test-read-ctf/test4.so.abi | 18 +-
tests/data/test-read-ctf/test4.so.hash.abi | 18 +-
tests/data/test-read-ctf/test5.o.abi | 34 +-
tests/data/test-read-ctf/test7.o.abi | 22 +-
tests/data/test-read-ctf/test8.o.abi | 4 +-
tests/data/test-read-ctf/test9.o.abi | 48 +-
.../PR22015-libboost_iostreams.so.abi | 1579 +-
.../test-read-dwarf/PR22122-libftdc.so.abi | 6146 +--
.../PR24378-fn-is-not-scope.abi | 26 +-
.../data/test-read-dwarf/PR25007-sdhci.ko.abi | 2938 +-
.../PR25042-libgdbm-clang-dwarf5.so.6.0.0.abi | 609 +-
.../test-read-dwarf/PR26261/PR26261-exe.abi | 44 +-
.../test-read-dwarf/PR27700/test-PR27700.abi | 10 +-
.../PR28584/PR28584-smv.clang.o.abi | 6 +-
.../test-read-dwarf/PR29443-missing-xx.o.abi | 8 +-
.../PR29692-kdelibs3-libkjava.so.1.0.0.abi | 2 +-
tests/data/test-read-dwarf/libtest23.so.abi | 316 +-
.../libtest24-drop-fns-2.so.abi | 353 +-
.../test-read-dwarf/libtest24-drop-fns.so.abi | 353 +-
.../data/test-read-dwarf/test-PR26568-1.o.abi | 18 +-
.../data/test-read-dwarf/test-PR26568-2.o.abi | 16 +-
tests/data/test-read-dwarf/test-fallback.abi | 4 +-
.../test-read-dwarf/test-libaaudio.so.abi | 424 +-
.../test-read-dwarf/test-libandroid.so.abi | 21142 +++++----
.../test-pointer-to-member-1.o.abi | 34 +-
.../test-suppressed-alias.o.abi | 6 +-
tests/data/test-read-dwarf/test0.abi | 46 +-
tests/data/test-read-dwarf/test0.hash.abi | 48 +-
tests/data/test-read-dwarf/test1.abi | 44 +-
tests/data/test-read-dwarf/test1.hash.abi | 50 +-
.../test-read-dwarf/test10-pr18818-gcc.so.abi | 4194 +-
.../test-read-dwarf/test11-pr18828.so.abi | 11118 ++---
.../test-read-dwarf/test12-pr18844.so.abi | 21428 ++++-----
.../test-read-dwarf/test13-pr18894.so.abi | 1260 +-
.../test-read-dwarf/test14-pr18893.so.abi | 3091 +-
.../test-read-dwarf/test15-pr18892.so.abi | 20201 ++++-----
.../test-read-dwarf/test16-pr18904.so.abi | 19855 ++++----
.../test-read-dwarf/test17-pr19027.so.abi | 8996 ++--
...st18-pr19037-libvtkRenderingLIC-6.1.so.abi | 9051 ++--
...19-pr19023-libtcmalloc_and_profiler.so.abi | 9371 ++--
tests/data/test-read-dwarf/test2.so.abi | 30 +-
tests/data/test-read-dwarf/test2.so.hash.abi | 30 +-
...st20-pr19025-libvtkParallelCore-6.1.so.abi | 7774 ++--
.../test-read-dwarf/test21-pr19092.so.abi | 3804 +-
.../test22-pr19097-libstdc++.so.6.0.17.so.abi | 37350 ++++++++--------
.../test-read-dwarf/test3-alias-1.so.hash.abi | 4 +-
.../test-read-dwarf/test3-alias-2.so.hash.abi | 4 +-
.../test-read-dwarf/test3-alias-3.so.hash.abi | 4 +-
.../test-read-dwarf/test3-alias-4.so.hash.abi | 4 +-
tests/data/test-read-dwarf/test3.so.abi | 4 +-
tests/data/test-read-dwarf/test3.so.hash.abi | 4 +-
tests/data/test-read-dwarf/test4.so.abi | 18 +-
tests/data/test-read-dwarf/test4.so.hash.abi | 18 +-
tests/data/test-read-dwarf/test5.o.abi | 4 +-
tests/data/test-read-dwarf/test5.o.hash.abi | 4 +-
tests/data/test-read-dwarf/test6.so.abi | 18 +-
tests/data/test-read-dwarf/test6.so.hash.abi | 18 +-
tests/data/test-read-dwarf/test7.so.abi | 52 +-
tests/data/test-read-dwarf/test7.so.hash.abi | 52 +-
.../test8-qualified-this-pointer.so.abi | 16 +-
.../test8-qualified-this-pointer.so.hash.abi | 16 +-
.../test9-pr18818-clang.so.abi | 3051 +-
tests/data/test-read-write/test-crc.xml | 2 +-
tests/data/test-read-write/test0.xml | 2 +-
tests/data/test-read-write/test1.xml | 2 +-
tests/data/test-read-write/test10.xml | 20 +-
tests/data/test-read-write/test11.xml | 4 +-
tests/data/test-read-write/test12.xml | 6 +-
tests/data/test-read-write/test13.xml | 8 +-
tests/data/test-read-write/test14.xml | 8 +-
tests/data/test-read-write/test15.xml | 6 +-
tests/data/test-read-write/test16.xml | 4 +-
tests/data/test-read-write/test17.xml | 54 +-
tests/data/test-read-write/test18.xml | 16 +-
tests/data/test-read-write/test19.xml | 22 +-
tests/data/test-read-write/test2.xml | 8 +-
tests/data/test-read-write/test20.xml | 88 +-
tests/data/test-read-write/test21.xml | 26 +-
tests/data/test-read-write/test22.xml | 12 +-
tests/data/test-read-write/test23.xml | 12 +-
tests/data/test-read-write/test24.xml | 16 +-
tests/data/test-read-write/test25.xml | 62 +-
tests/data/test-read-write/test26.xml | 22 +-
tests/data/test-read-write/test27.xml | 16 +-
.../test28-without-std-fns-ref.xml | 1190 +-
.../test28-without-std-vars-ref.xml | 1116 +-
tests/data/test-read-write/test3.xml | 8 +-
tests/data/test-read-write/test4.xml | 4 +-
tests/data/test-read-write/test5.xml | 4 +-
tests/data/test-read-write/test6.xml | 10 +-
tests/data/test-read-write/test7.xml | 4 +-
tests/data/test-read-write/test8.xml | 2 +-
tests/data/test-read-write/test9.xml | 8 +-
tests/test-diff-pkg.cc | 13 +
tools/abidw.cc | 3 +-
tools/abilint.cc | 30 +-
tools/abipkgdiff.cc | 38 +-
220 files changed, 140901 insertions(+), 145092 deletions(-)
create mode 100644 tests/data/test-diff-pkg/gcc-debuginfo-14.2.1-1.fc40.x86_64.rpm
create mode 100644 tests/data/test-diff-pkg/libgm2-14.2.1-1.fc40.x86_64-self-check-report.txt
create mode 100644 tests/data/test-diff-pkg/libgm2-14.2.1-1.fc40.x86_64.rpm
create mode 100644 tests/data/test-diff-pkg/libgm2-debuginfo-14.2.1-1.fc40.x86_64.rpm
--
2.43.5
More information about the Libabigail
mailing list