[PATCH 08/12] libctf, ld: properly deduplicate function types

[Nick Alcock]

Some type kinds in CTF (functions, arrays, pointers, slices, and
cvr-quals) are intrinsically nameless: the ctt_name field in the CTF
is always zero, and the libctf API provides no way to set a name.
But the compiler can and does sometimes set names for some of these
kinds: in particular, the name it sets on CTF_K_FUNCTION types is the
means it uses to force the name of the function into the string table
so that it can point at it from the function info section.

So null out the name at hashing time so that the deduplicator can
correctly detect that e.g. function types identical but for name should
be considered truly identical, since they will not have a name when the
deduplicator re-emits them into the output.

ld/ChangeLog
2020-10-23  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/ld-ctf/data-func-conflicted.d: Shrink the expected
	size of the type section now that function types are being
	deduplicated properly.

libctf/ChangeLog
2020-10-23  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-dedup.c (ctf_dedup_rhash_type): Null out the names of nameless
	type kinds, just in case the input has named them.
---
 ld/testsuite/ld-ctf/data-func-conflicted.d |  2 +-
 libctf/ctf-dedup.c                         | 26 +++++++++++++++++-----
 2 files changed, 22 insertions(+), 6 deletions(-)

diff --git a/ld/testsuite/ld-ctf/data-func-conflicted.d b/ld/testsuite/ld-ctf/data-func-conflicted.d
index 626bfef052e..1fa8bb2fe5f 100644
--- a/ld/testsuite/ld-ctf/data-func-conflicted.d
+++ b/ld/testsuite/ld-ctf/data-func-conflicted.d
@@ -16,7 +16,7 @@ Contents of CTF section \.ctf:
     Data object section:	.* \(0xc bytes\)
     Function info section:	.* \(0x40 bytes\)
     Object index section:	.* \(0xc bytes\)
-    Type section:	.* \(0x228 bytes\)
+    Type section:	.* \(0xe8 bytes\)
     String section:	.*
 #...
   Data objects:
diff --git a/libctf/ctf-dedup.c b/libctf/ctf-dedup.c
index 3e95af09cac..b0be5a7044f 100644
--- a/libctf/ctf-dedup.c
+++ b/libctf/ctf-dedup.c
@@ -638,6 +638,27 @@ ctf_dedup_rhash_type (ctf_dict_t *fp, ctf_dict_t *input, ctf_dict_t **inputs,
 	&& ctf_dedup_record_origin (fp, input_num, decorated, type_id) < 0)
       return NULL;				/* errno is set for us.  */
 
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+  ctf_dprintf ("%lu: hashing thing with ID %i/%lx (kind %i): %s.\n",
+	       depth, input_num, type, kind, name ? name : "");
+#endif
+
+  /* Some type kinds don't have names: the API provides no way to set the name,
+     so the type the deduplicator outputs will be nameless even if the input
+     somehow has a name, and the name should not be mixed into the hash.  */
+
+  switch (kind)
+    {
+    case CTF_K_POINTER:
+    case CTF_K_ARRAY:
+    case CTF_K_FUNCTION:
+    case CTF_K_VOLATILE:
+    case CTF_K_CONST:
+    case CTF_K_RESTRICT:
+    case CTF_K_SLICE:
+      name = NULL;
+    }
+
   /* Mix in invariant stuff, transforming the type kind if needed.  Note that
      the vlen is *not* hashed in: the actual variable-length info is hashed in
      instead, piecewise.  The vlen is not part of the type, only the
@@ -647,11 +668,6 @@ ctf_dedup_rhash_type (ctf_dict_t *fp, ctf_dict_t *input, ctf_dict_t **inputs,
      *other types in the same TU* with the same name: so two types can easily
      have distinct nonroot flags, yet be exactly the same type.*/
 
-#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
-  ctf_dprintf ("%lu: hashing thing with ID %i/%lx (kind %i): %s.\n",
-	       depth, input_num, type, kind, name ? name : "");
-#endif
-
   ctf_sha1_init (&hash);
   if (name)
     ctf_dedup_sha1_add (&hash, name, strlen (name) + 1, "name", depth);
-- 
2.29.0.249.g249b51256f