[PATCH v4 05/14] BFD: Correct archive format determination for non-default link emulations

[Maciej W. Rozycki]

From: Alan Modra <amodra@gmail.com>

Fix an issue with archive format determination coming from commit 
b228303dd14e ("PR binutils/13278: --plugin doesn't work on archive"), 
<https://inbox.sourceware.org/binutils/20111011064345.GM25970@bubble.grove.modra.org/>, 
causing the default link emulation to be used despite archive members 
being in another object format and consequently triggering segmentation 
faults in `cris-aout' LD with the default (`crisaout') emulation where 
archive members are in the ELF format.

Address the issue by always checking the format of the first member of 
an archive even if no default target has been specified, except when the 
target being checked against is be the plugin target.  Verified by hand 
and the `powerpc64le-linux-gnu' target to cause no issue with the recipe 
in PR binutils/13278 and removing failures from segmentation faults with 
the new tests:

cris-aout  -FAIL: CRIS regular archive link (implicit emulation)
cris-aout  -FAIL: CRIS thin archive link (implicit emulation)

Co-Authored-By: Maciej W. Rozycki <macro@redhat.com>
---
Changes from v3 (11/18), 
<https://inbox.sourceware.org/binutils/alpine.DEB.2.21.2511192246460.57987@angie.orcam.me.uk/>:

- Retain the check for a defaulted target and instead also enable the
  first archive member format check for linker input.

- Request that no plugin be used for the first archive member format 
  check.

New change in v3.
---
 bfd/archive.c                    |    5 ++++-
 ld/testsuite/ld-cris/archive.exp |   21 +++++++++++++++++++++
 2 files changed, 25 insertions(+), 1 deletion(-)

binutils-bfd-archive-format-check-fix.diff
Index: binutils-gdb/bfd/archive.c
===================================================================
--- binutils-gdb.orig/bfd/archive.c
+++ binutils-gdb/bfd/archive.c
@@ -928,7 +928,8 @@ bfd_generic_archive_p (bfd *abfd)
       return NULL;
     }
 
-  if (abfd->target_defaulted && bfd_has_map (abfd))
+  if ((abfd->target_defaulted || abfd->is_linker_input)
+      && bfd_has_map (abfd))
     {
       bfd *first;
       unsigned int save;
@@ -951,6 +952,8 @@ bfd_generic_archive_p (bfd *abfd)
       if (first != NULL)
 	{
 	  first->target_defaulted = false;
+	  if (abfd->is_linker_input)
+	    first->plugin_format = bfd_plugin_no;
 	  if (!bfd_check_format (first, bfd_object)
 	      || first->xvec != abfd->xvec)
 	    bfd_set_error (bfd_error_wrong_object_format);
Index: binutils-gdb/ld/testsuite/ld-cris/archive.exp
===================================================================
--- binutils-gdb.orig/ld/testsuite/ld-cris/archive.exp
+++ binutils-gdb/ld/testsuite/ld-cris/archive.exp
@@ -68,6 +68,14 @@ run_ld_link_tests [list \
 	 {nm "" ../ld-archive/nx.nd}} \
 	"abcx" \
     ] \
+    [list "CRIS regular archive link (implicit emulation)" \
+	"-e ff" "tmpdir/ab.a --verbose --print-map" \
+	"$asemul" {../ld-archive/abc.s} \
+	{{ld ../ld-archive/abc.vd} \
+	 {nm "" ../ld-archive/abc.nd} \
+	 {nm "" ../ld-archive/nx.nd}} \
+	"abci" \
+    ] \
 ]
 if { ![istarget cris-*-*aout*] } {
     setup_xfail "binutils/33485" "*-*-*"
@@ -82,3 +90,16 @@ run_ld_link_tests [list \
 	"abtcx" \
     ] \
 ]
+if { ![istarget cris-*-*aout*] } {
+    setup_xfail "binutils/33485" "*-*-*"
+}
+run_ld_link_tests [list \
+    [list "CRIS thin archive link (implicit emulation)" \
+	"-e ff" "tmpdir/abt.a --verbose --print-map" \
+	"$asemul" {../ld-archive/abc.s} \
+	{{ld ../ld-archive/abtc.vd} \
+	 {nm "" ../ld-archive/abc.nd} \
+	 {nm "" ../ld-archive/nx.nd}} \
+	"abtci" \
+    ] \
+]