Bug 27297 - libctf.a malformed, build fails on x86_64-apple-darwin18.7.0
Summary: libctf.a malformed, build fails on x86_64-apple-darwin18.7.0
Status: RESOLVED FIXED
Alias: None
Product: binutils
Classification: Unclassified
Component: libctf (show other bugs)
Version: 2.36
: P2 normal
Target Milestone: ---
Assignee: Nick Alcock
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-01-31 07:08 UTC by Stephen Casner
Modified: 2021-06-11 14:25 UTC (History)
5 users (show)

See Also:
Host: x86_64-apple-darwin18.7.0
Target: pdp11-aout, x86_64-apple-darwin18.7.0, others?
Build:
Last reconfirmed:


Attachments
2.36 build log (22.00 KB, text/plain)
2021-01-31 07:08 UTC, Stephen Casner
Details
tarball of possibly relevant configs, Makefiles (973.59 KB, application/octet-stream)
2021-01-31 07:25 UTC, Stephen Casner
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stephen Casner 2021-01-31 07:08:35 UTC
Created attachment 13182 [details]
2.36 build log

I first encountered this problem when trying to build a pdp11-aout cross on x86_64-apple-darwin18.7.0, but the problem is there for the native build as well.  I have not tested to see if other hosts are affected (this is all I have).  This build is using the macOS system tools (clang, etc.).

This problem does NOT occur with 2.35.2.  It DOES occur with 2.36 and HEAD.

The problem arises here:

/bin/sh ./libtool  --tag=CC   --mode=link gcc -std=gnu99 -Wall -W -Wall -Wno-narrowing -Wwrite-strings -Wmissing-format-attribute -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -pedantic -Wno-long-long  -I/Users/casner/src/gnu/binutils-gdb/libctf/../zlib -g -O2     -version-info 0:0:0  -export-symbols-regex ctf_.* -Wl,-no_pie  -o libctf.la -rpath /usr/local/lib libctf_la-ctf-archive.lo libctf_la-ctf-dump.lo libctf_la-ctf-create.lo libctf_la-ctf-decl.lo libctf_la-ctf-error.lo libctf_la-ctf-hash.lo libctf_la-ctf-labels.lo libctf_la-ctf-dedup.lo libctf_la-ctf-link.lo libctf_la-ctf-lookup.lo libctf_la-ctf-open.lo libctf_la-ctf-sha1.lo libctf_la-ctf-string.lo libctf_la-ctf-subr.lo libctf_la-ctf-types.lo libctf_la-ctf-util.lo  libctf_la-ctf-open-bfd.lo ../bfd/libbfd.la -L/Users/casner/src/gnu/binutils-gdb/native/libctf/../libiberty -liberty ./../intl/libintl.a -liconv -L./../zlib -lz 

*** Warning: Linking the shared library libctf.la against the
*** static library ./../intl/libintl.a is not portable!
libtool: link: ar  rc .libs/libctf.a ./../intl/libintl.a  libctf_la-ctf-archive.o libctf_la-ctf-dump.o libctf_la-ctf-create.o libctf_la-ctf-decl.o libctf_la-ctf-error.o libctf_la-ctf-hash.o libctf_la-ctf-labels.o libctf_la-ctf-dedup.o libctf_la-ctf-link.o libctf_la-ctf-lookup.o libctf_la-ctf-open.o libctf_la-ctf-sha1.o libctf_la-ctf-string.o libctf_la-ctf-subr.o libctf_la-ctf-types.o libctf_la-ctf-util.o libctf_la-ctf-open-bfd.o

This causes libctf.a to contain libintl.a as its first component file:

auge48> ar -t libctf/.libs/libctf.a
__.SYMDEF SORTED
libintl.a
libctf_la-ctf-archive.o
libctf_la-ctf-dump.o
libctf_la-ctf-create.o
libctf_la-ctf-decl.o
libctf_la-ctf-error.o
libctf_la-ctf-hash.o
libctf_la-ctf-labels.o
libctf_la-ctf-dedup.o
libctf_la-ctf-link.o
libctf_la-ctf-lookup.o
libctf_la-ctf-open.o
libctf_la-ctf-sha1.o
libctf_la-ctf-string.o
libctf_la-ctf-subr.o
libctf_la-ctf-types.o
libctf_la-ctf-util.o
libctf_la-ctf-open-bfd.o

That causes the link of objdump to fail, ending the build:

libtool: link: gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -I/Users/casner/src/gnu/binutils-gdb/binutils/../zlib -g -O2 -Wl,-no_pie -o objdump objdump.o dwarf.o prdbg.o rddbg.o debug.o stabs.o rdcoff.o bucomm.o version.o filemode.o elfcomm.o od-macho.o  ../opcodes/.libs/libopcodes.a ../libctf/.libs/libctf.a -L/Users/casner/src/gnu/binutils-gdb/native/zlib -L/Users/casner/src/gnu/binutils-gdb/native/libctf/../libiberty /Users/casner/src/gnu/binutils-gdb/native/bfd/.libs/libbfd.a -liberty ../bfd/.libs/libbfd.a -ldl -lz ../libiberty/libiberty.a ./../intl/libintl.a -liconv
ld: warning: ignoring file ../libctf/.libs/libctf.a, building for macOS-x86_64 but attempting to link with file built for macOS-x86_64
Undefined symbols for architecture x86_64:
  "_ctf_archive_iter", referenced from:
      _dump_bfd in objdump.o
  "_ctf_bfdopen_ctfsect", referenced from:
      _dump_bfd in objdump.o
  "_ctf_close", referenced from:
      _dump_bfd in objdump.o
  "_ctf_dict_close", referenced from:
      _dump_bfd in objdump.o
  "_ctf_dict_open", referenced from:
      _dump_bfd in objdump.o
  "_ctf_dump", referenced from:
      _dump_ctf_archive_member in objdump.o
  "_ctf_errmsg", referenced from:
      _dump_bfd in objdump.o
      _dump_ctf_errs in objdump.o
      _dump_ctf_archive_member in objdump.o
  "_ctf_errno", referenced from:
      _dump_ctf_archive_member in objdump.o
  "_ctf_errwarning_next", referenced from:
      _dump_ctf_errs in objdump.o
  "_ctf_import", referenced from:
      _dump_ctf_archive_member in objdump.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Comment 1 Stephen Casner 2021-01-31 07:25:40 UTC
Created attachment 13183 [details]
tarball of possibly relevant configs, Makefiles

Added tarball of these files.  Let me know if any others would be helpful.

auge12> tar czf relevant.tgz intl/Makefile intl/config.* libctf/Makefile libctf/config.* libctf/libctf*.la libctf/.libs/libctf.a
auge13> v relevant.tgz 
-rw-r--r--+ 1 casner  staff  999896 Jan 30 23:19 relevant.tgz
Comment 2 Stephen Casner 2021-01-31 23:26:29 UTC
git bisect finds this is the commit where the problem began:

commit 1038406a8f6609ad0a449746da70393b0835f699
Author: Nick Alcock <nick.alcock@oracle.com>
Date:   Tue Jan 5 13:25:56 2021 +0000

    libctf: rip out BFD_DEPENDENCIES / BFD_LIBADD
    
    This complex morass inherited from libopcodes, which endeavours to
    implement the effect of specifying ../bfd/libbfd.la in _LIBADD without
    actually doing so, appears to be working around a libtool bug which as
    far as I can see is no longer present (i.e., the install directory no
    longer appears in -L arguments in libtool link-mode invocations, so
    there is no danger of picking up old libbfds or other dependent
    libraries).
    
    Replaced with a simple reference to libbfd.la in the appropriate place.
    Also adjusted things a little more so that libctf.la and libctf-nobfd.la
    are self-contained, even when linking statically.  This opens up the
    possibility of running libtool to link against libctf from inside the
    (upcoming) testsuite.
    
    libctf/ChangeLog
    2021-01-05  Nick Alcock  <nick.alcock@oracle.com>
    
            * configure.ac (BFD_LIBADD): Remove.
            (BFD_DEPENDENCIES): Likewise. Remove associated cases.
            (SHARED_LIBADD): Rename to...
            (CTF_LIBADD): ... this.  Stick in a suitable libiberty even when
            linking statically.
            * Makefile.am (libctf_nobfd_la_LIBADD): Adjust accordingly.
            libctf uses libintl.
            (libctf_la_LIBADD): Reference libbfd.la directly, not via
            BFD_LIBADD.
            (libctf_la_DEPENDENCIES): Remove.
            * Makefile.in: Regenerate.
            * configure: Likewise.
Comment 3 Nick Alcock 2021-02-02 14:04:31 UTC
Replicated. Working on a fix.
Comment 4 Stephen Casner 2021-02-11 05:25:06 UTC
Bug 27360 appears to be related, but I'm not sure if they can be called duplicates.
Comment 5 Nick Alcock 2021-03-08 17:19:50 UTC
Fixed (some time back, but couldn't update the bug) in master in a series in which the most important commits are these:

(not sure if it is worthy of going back into 2.36. The commits it relies on are quite big and invasive.)

commit 95148614026da7353721411dd020d024667e3482
Author: Nick Alcock <nick.alcock@oracle.com>
Date:   Wed Feb 3 18:42:06 2021 +0000

    bfd, opcodes, libctf: support --with-included-gettext
    
    Right now, these libraries hardwire -L../intl -lintl on a few fixed
    platforms, which works fine on those platforms but on other platforms
    leads to shared libraries that lack libintl_* symbols when configured
    --with-included-gettext, and/or static libraries that contain libintl as
    *another* static library.  If we instead use the LIBINTL variable
    defined in ../intl/config.intl, this gives us the right thing on all
    three classes of platform (gettext in libc, gettext in system libintl,
    gettext in ../intl/libintl.a)..  This also means we can rip out some
    Darwin-specific machinery from configure.ac and also simplify the Cygwin
    side.
    
    This also means that the libctf testsuite (and other places that include
    libbfd, libopcodes or libctf) don't need to grow libintl dependencies
    just on account of those libraries (though they still need such
    dependencies if they themselves use gettext machinery).
    
    bfd/ChangeLog
    2021-02-03  Nick Alcock  <nick.alcock@oracle.com>
    
            * configure.ac (SHARED_LIBADD): Remove explicit -lintl population in
            favour of LIBINTL.
            * configure: Regenerated.
    
    libctf/ChangeLog
    2021-02-02  Nick Alcock  <nick.alcock@oracle.com>
    
            * configure.ac (CTF_LIBADD): Remove explicit -lintl population in
            favour of LIBINTL.
            * Makefile.am (libctf_nobfd_la_LIBADD): No longer explicitly
            include $(LIBINTL).
            (check-DEJAGNU): Pass down to tests as well.
            * configure: Regenerated.
            * Makefile.in: Likewise.
    
    opcodes/ChangeLog
    2021-02-04  Nick Alcock  <nick.alcock@oracle.com>
    
            * configure.ac (SHARED_LIBADD): Remove explicit -lintl population in
            favour of LIBINTL.
            * configure: Regenerated.




commit aee224d6434c08a1404a4357cf0a664a4c2f02eb
Author: Nick Alcock <nick.alcock@oracle.com>
Date:   Thu Feb 4 16:58:35 2021 +0000

    intl: turn LIBINTL into -L / -l form
    
    This variable currently refers directly, not to a .la file, but to an .a
    file.  This produces wrong results when building into a library on some
    platforms: so convert it to the general form "-L${top_builddir}../intl
    -lintl ..." ... so that both libtool and non-libtool builds will always
    do the right thing for both static and shared links.
    
    intl/ChangeLog
    2021-02-04  Nick Alcock  <nick.alcock@oracle.com>
    
            * configure.ac (LIBINTL): Transform into -L/-lintl form.
            * configure: Regenerate.

commit 53d4244ec0ac70438d75abf3326cb3392bb9c828
Author: Nick Alcock <nick.alcock@oracle.com>
Date:   Tue Feb 2 15:39:26 2021 +0000

    intl: always picify
    
    libintl is included in several shared libraries (at least
    libinproctrace.so and libctf.so): unconditionally picify with code
    borrowed from libiberty configure.  (It's not performance-critical, so
    don't bother making separate PIC and non-PIC libraries like libiberty
    does.)
    
    intl/ChangeLog
    2021-02-02  Nick Alcock  <nick.alcock@oracle.com>
    
            * aclocal.m4: include picflag.m4.
            * configure.ac (PICFLAG): Add and substitute.
            * Makefile.in (PICFLAG): New.
            (COMPILE): Use it.
            * configure: Regenerate.
Comment 6 Yichao Yu 2021-03-09 16:21:36 UTC
Does any of the patches also fixed https://sourceware.org/bugzilla/show_bug.cgi?id=27360 ?
Comment 7 Nick Alcock 2021-03-09 18:07:43 UTC
Quite possibly, or at least I don't see that failure with the reported replication mechanism on binutils master: I'll update that bug too.
Comment 8 Nick Alcock 2021-06-11 14:25:47 UTC
Now I know the cause of that bug: no, these are unrelated (and a separate fix is needed for that bug).