ir: Avoid cancelling a "confirmed" propagated canonical type
When canonicalizing a type T at the IR level, the canonical type of T
(aka C(T)) can be deduced in two ways:
1/ Either T structurally (aka member-wise) compares equal to another
T' which has a canonical type C(T'). In that case we deduce that
C(T) equals C(T').
2/ Or, during the canonicalization of another type Y of which T is a
sub-type, the comparison engine comes to compare T against a type
T' that is already canonicalized and finds that T structurally
compares equal to T'. In that case we deduce that C(T) equals
C(T'), even though we were canonicalizing Y in the first place.
In 2/ we C(T) is stored and so when comes the time to canonicalize T,
we already know C(T).
What happens in 2/ is called "canonical type propagation". Because the
canonical type of T' is propagated to T while canonicalizing Y.
There can be cases however where the propagated canonical type has to
be "cancelled". This is described in the source code in abg-ir.cc in
the "@ref OnTheFlyCanonicalization" chapter of the doxygen
documentation.
There are also cases where the propagated canonical type has been
"confirmed", meaning, it cannot be cancelled anymore.
Unfortunately, there seems to be (wrong) cases where confirmed
propagated canonical types are being cancelled. Oops. That leads to
having types that are missing canonical types down the road.
This is exhibited by the self-comparison check of the
avr-binutils-2.39-1.fc36.armv7hl.rpm package (more precisely when
analyzing the 'objdump' binary from that package). This was triggered
by the command:
This patch improves the book keeping around IR canonical type
propagation to avoid cancelling confirmed propagated canonical types.
It introduces a flag to the type_base::priv sub-object to track the
confirmation status of propagated canonical types. It then updates
the book-keeping routines to make them avoid cancelling confirmed
propagated canonical types.
* src/abg-ir-priv.h
(type_base::priv::propagated_canonical_type_confirmed_): Define
new data member.
(type_base::priv::priv): Initialize it.
(type_base::priv::{propagated_canonical_type_confirmed,
set_propagated_canonical_type_confirmed}): Define new member
functions.
(type_base::priv::clear_propagated_canonical_type): Do not clear
the propagated canonical type if it has been confirmed already.
(type_base::priv::confirm_ct_propagation_for_types_dependant_on): Rename
type_base::confirm_ct into this. Mark the confirmed propagated
types as being confirmed.
(type_base::priv::confirm_ct_propagation): This is now a new member
function that calls
type_base::confirm_ct_propagation_for_types_dependant_on and that
does the book-keeping that was being done in
return_comparison_result.
(type_base::priv::cancel_ct_propagation_for_types_dependant_on):
Renamed type_base::priv::cancel_ct_propagation in this.
(type_base::priv::cancel_ct_propagation): In this new one, call
type_base::priv::cancel_ct_propagation_for_types_dependant_on. Perform here
the book-keeping that was being done in return_comparison_result.
Also, do not cancel a confirmed propagated canonical type.
(type_base::priv::add_to_types_with_non_confirmed_propagated_ct):
Define new member function.
* src/abg-ir.cc (return_comparison_result): Consider only types
with non-confirmed propagated canonical types for the
non-confirmed type queue. Also, only sub-types can be considered
non-confirmed. Split out some of the book-keeping into
type_base::priv::{confirm_ct_propagation, cancel_ct_propagation}
and call these instead. Confirm the propagated canonical types of
all types that remain after the comparison is fully done and is
successful.
(canonicalize): Assert the rule "The result of canonicalizing must
always been a confirmed canonical type".