Bug 27967

Summary: Build failure on solaris-11 ld: fatal: option --version-script requires option -z gnu-version-script-compat to be specified
Product: binutils Reporter: Sergei Trofimovich <slyich>
Component: binutilsAssignee: Nick Alcock <nick.alcock>
Status: RESOLVED FIXED    
Severity: normal CC: nick.alcock, nickc
Priority: P2    
Version: 2.34   
Target Milestone: ---   
Host: sparcv9-sun-solaris2.11 Target:
Build: Last reconfirmed: 2021-06-22 00:00:00
Attachments: build.log

Description Sergei Trofimovich 2021-06-08 07:51:47 UTC
Created attachment 13486 [details]
build.log

It's a forward of https://bugs.gentoo.org/791007 bug found by Michael Kosarev where on solaris-11 binutils building fails as:

"""
/export/home/ruse39/gentoo/tmp/bin/bash ./libtool  --tag=CC   --mode=link gcc -m64 -std=gnu99 -Wall -W -Wall -Wno-narrowing -Wwrite-strings -Wmissing-format-attribute -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -pedantic -Wno-long-long   -O2 -pipe -version-info 0:0:0  -Wl,--version-script='/export/home/ruse39/gentoo/tmp/var/tmp/portage/sys-devel/binutils-2.34-r1/work/binutils-2.34/libctf/libctf.ver' -L/export/home/ruse39/gentoo/tmp/usr/lib -R/export/home/ruse39/gentoo/tmp/usr/lib -R/export/home/ruse39/gentoo/tmp/tmp -o libctf.la -rpath /export/home/ruse39/gentoo/tmp/usr/lib/binutils/sparcv9-sun-solaris2.11/2.34 ctf-archive.lo ctf-dump.lo ctf-create.lo ctf-decl.lo ctf-error.lo ctf-hash.lo ctf-labels.lo ctf-link.lo ctf-lookup.lo ctf-open.lo ctf-string.lo ctf-subr.lo ctf-types.lo ctf-util.lo  ctf-open-bfd.lo -Wl,/export/home/ruse39/gentoo/tmp/var/tmp/portage/sys-devel/binutils-2.34-r1/work/build/libctf/../bfd/.libs/libbfd.so -L/export/home/ruse39/gentoo/tmp/var/tmp/portage/sys-devel/binutils-2.34-r1/work/build/libctf/../libiberty/pic -liberty -lz

libtool: link: gcc -m64 -shared -Wl,-z -Wl,text -Wl,-h -Wl,libctf.so.0 -o .libs/libctf.so.0.0.0  .libs/ctf-archive.o .libs/ctf-dump.o .libs/ctf-create.o .libs/ctf-decl.o .libs/ctf-error.o .libs/ctf-hash.o .libs/ctf-labels.o .libs/ctf-link.o .libs/ctf-lookup.o .libs/ctf-open.o .libs/ctf-string.o .libs/ctf-subr.o .libs/ctf-types.o .libs/ctf-util.o .libs/ctf-open-bfd.o   -R/export/home/ruse39/gentoo/tmp/usr/lib -R/export/home/ruse39/gentoo/tmp/tmp -L/export/home/ruse39/gentoo/tmp/usr/lib -L/export/home/ruse39/gentoo/tmp/var/tmp/portage/sys-devel/binutils-2.34-r1/work/build/libctf/../libiberty/pic -liberty -lz  -m64 -Wl,--version-script=/export/home/ruse39/gentoo/tmp/var/tmp/portage/sys-devel/binutils-2.34-r1/work/binutils-2.34/libctf/libctf.ver -Wl,/export/home/ruse39/gentoo/tmp/var/tmp/portage/sys-devel/binutils-2.34-r1/work/build/libctf/../bfd/.libs/libbfd.so

ld: fatal: option --version-script requires option -z gnu-version-script-compat to be specified
"""

Looks like solaris' ld needs extra `–z gnu-version-script-compat` option set to make `–version-script` work:

https://docs.oracle.com/cd/E88353_01/html/E37839/ld-1.html says we need an extra  option 
"""
–z gnu-version-script=mapfile
–z gnu-version-script-compat
–version-script mapfile

    Provides partial support for the GNU version script style of mapfile. Version scripts are based on the original Solaris version 1 symbol definition syntax, with some extensions. ld supports the most common such extension, the use of wildcard characters in the specified symbol names. Other GNU-specific extensions may not be supported. ld will issue an appropriate error if an unsupported extension is encountered.

    For convenience in building software developed with GNU version scripts, the native GNU –version-script option is accepted as an alias for –z gnu-version-script. Due to the partial nature of the support for GNU version scripts, the use of –version-script must be explicitly enabled by specifying –z gnu-version-script-compat.
"""

libctf/configure.ac has too simplistic detection of --version-script and probably will need to be extended:

"""
# Use a version script, if possible, or an -export-symbols-regex otherwise.
VERSION_FLAGS='-export-symbols-regex ctf_.*'
if $LD --help 2>&1 | grep -- --version-script >/dev/null; then
    VERSION_FLAGS="-Wl,--version-script='$srcdir/libctf.ver'"
fi
AC_SUBST(VERSION_FLAGS)
"""

Attached full build.log. binutils was configured as:

$ ./configure --enable-plugins --disable-nls --with-system-zlib --build=sparcv9-sun-solaris2.11 --prefix=/export/home/ruse39/gentoo/tmp/usr --host=sparcv9-sun-solaris2.11 --target=sparcv9-sun-solaris2.11 --datadir=/export/home/ruse39/gentoo/tmp/usr/share/binutils-data/sparcv9-sun-solaris2.11/2.34 --datarootdir=/export/home/ruse39/gentoo/tmp/usr/share/binutils-data/sparcv9-sun-solaris2.11/2.34 --infodir=/export/home/ruse39/gentoo/tmp/usr/share/binutils-data/sparcv9-sun-solaris2.11/2.34/info --mandir=/export/home/ruse39/gentoo/tmp/usr/share/binutils-data/sparcv9-sun-solaris2.11/2.34/man --bindir=/export/home/ruse39/gentoo/tmp/usr/sparcv9-sun-solaris2.11/binutils-bin/2.34 --libdir=/export/home/ruse39/gentoo/tmp/usr/lib/binutils/sparcv9-sun-solaris2.11/2.34 --libexecdir=/export/home/ruse39/gentoo/tmp/usr/lib/binutils/sparcv9-sun-solaris2.11/2.34 --includedir=/export/home/ruse39/gentoo/tmp/usr/lib/binutils/sparcv9-sun-solaris2.11/2.34/include --enable-obsolete --enable-shared --enable-threads --enable-relro --enable-install-libiberty --disable-werror --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion=Gentoo 2.34 p4 --disable-static --disable-gdb --disable-libdecnumber --disable-readline --disable-sim --without-stage1-ldflags --with-extra-soversion-suffix=gentoo-sys-devel-binutils-st
Comment 1 Nick Clifton 2021-06-17 13:54:13 UTC
(In reply to Sergei Trofimovich from comment #0)
Hi Sergei,

  I am not exactly sure what the error is here.  The linker's message seems pretty explicit:

> ld: fatal: option --version-script requires option -z
> gnu-version-script-compat to be specified

So either this is a libtool problem - in that it is not adding the necessary option to the gcc command line that is then used to invoke the linker, or it is a gcc problem, if you think that it should automagically add the option itself.

Possibly you are suggesting that the linker should not issue the error message, but just silently enable the option itself ?

Cheers
  Nick
Comment 2 Sergei Trofimovich 2021-06-18 08:02:48 UTC
(In reply to Nick Clifton from comment #1)
> (In reply to Sergei Trofimovich from comment #0)
> Hi Sergei,
> 
>   I am not exactly sure what the error is here.  The linker's message seems
> pretty explicit:
> 
> > ld: fatal: option --version-script requires option -z
> > gnu-version-script-compat to be specified
> 
> So either this is a libtool problem - in that it is not adding the necessary
> option to the gcc command line that is then used to invoke the linker, or it
> is a gcc problem, if you think that it should automagically add the option
> itself.
> 
> Possibly you are suggesting that the linker should not issue the error
> message, but just silently enable the option itself ?

I think the simplest solution for binutils would be somehow not to use --version-script against a solaris linker.

libctf/configure.ac has too simplistic detection of --version-script and probably will need to be extended:

"""
# Use a version script, if possible, or an -export-symbols-regex otherwise.
VERSION_FLAGS='-export-symbols-regex ctf_.*'
if $LD --help 2>&1 | grep -- --version-script >/dev/null; then
    VERSION_FLAGS="-Wl,--version-script='$srcdir/libctf.ver'"
fi
AC_SUBST(VERSION_FLAGS)
"""

Note1: flag support is detected only by grepping --help output (not by trying to use the flag for real). That's probably why we get a mismatch of a versioning feature support.
Note2: we don't probe libtool here and don't use binutils ld.
Comment 3 Nick Alcock 2021-06-22 09:39:24 UTC
Hm. --gnu-version-script and -z gnu-version-script appear to be recent enough that no machines in the compile farm support it, which makes it hard to replicate this. Nonetheless, I agree that doing a full link test should handle this case better. We could also check for --gnu-version-script, but without knowing how much of the GNU syntax is supported and with no way to test it myself this feels a little risky.

Myself, I hit bigger trouble trying to build on Solaris 11:

/bin/sh ./libtool  --tag=CC   --mode=link gcc -std=gnu99 -Wall -W -Wall -Wno-narrowing -Wwrite-strings -Wmissing-format-attribute -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -pedantic -Wno-long-long  -I../../libctf/../zlib -g -O2   -version-info 0:0:0  -export-symbols-regex ctf_.*  -o libctf.la -rpath /usr/local/lib libctf_la-ctf-archive.lo libctf_la-ctf-dump.lo libctf_la-ctf-create.lo libctf_la-ctf-decl.lo libctf_la-ctf-error.lo libctf_la-ctf-hash.lo libctf_la-ctf-labels.lo libctf_la-ctf-dedup.lo libctf_la-ctf-link.lo libctf_la-ctf-lookup.lo libctf_la-ctf-open.lo libctf_la-ctf-serialize.lo libctf_la-ctf-sha1.lo libctf_la-ctf-string.lo libctf_la-ctf-subr.lo libctf_la-ctf-types.lo libctf_la-ctf-util.lo libctf_la-ctf-qsort_r.lo libctf_la-ctf-open-bfd.lo ../bfd/libbfd.la -L/export/home/nix/binutils-gdb/foo/libctf/../libiberty/pic -liberty -lintl -L./../zlib -lz
libtool: link: nm  .libs/libctf_la-ctf-archive.o .libs/libctf_la-ctf-dump.o .libs/libctf_la-ctf-create.o .libs/libctf_la-ctf-decl.o .libs/libctf_la-ctf-error.o .libs/libctf_la-ctf-hash.o .libs/libctf_la-ctf-labels.o .libs/libctf_la-ctf-dedup.o .libs/libctf_la-ctf-link.o .libs/libctf_la-ctf-lookup.o .libs/libctf_la-ctf-open.o .libs/libctf_la-ctf-serialize.o .libs/libctf_la-ctf-sha1.o .libs/libctf_la-ctf-string.o .libs/libctf_la-ctf-subr.o .libs/libctf_la-ctf-types.o .libs/libctf_la-ctf-util.o .libs/libctf_la-ctf-qsort_r.o .libs/libctf_la-ctf-open-bfd.o   |  | /opt/csw/bin/gsed 's/.* //' | sort | uniq > .libs/libctf.exp


A pair of pipes with nothing between them is going to work really well! This is because symbol_pipe is unset in libtool, shown by

checking for sparc-sun-solaris2.11-ranlib... (cached) ranlib
checking command to parse nm output from gcc object... failed

This is because nm of /dev/null on Solaris 11.3 at least is emitting complaints about /dev/null not being a regular file rather than trying to *do* anything, so NM never gets set to nm -p and parsing fails: even if it succeeded, the list of symbol-type codes in libtool.m4 is out of date for Solaris 11, which breaks the test too. This means that builds on a stock Solaris 11 box seem likely to fail in any case.

Both these bugs can be fixed by suitably editing the top-level libtool.m4: I'll submit this to GCC (as needed for toplevel stuff) as well as binutils. (I'd submit it to libtool itself, but that seems to be dead.)

With that hacked around... I still can't reproduce your problem because I don't yet have access to a recent-enough Solaris box (though this will change in a couple of days): but I'll work on doing a proper link-time test anyway, since it's clear that what's there is inadequate.

More shortly.
Comment 4 Nick Alcock 2021-06-25 13:30:26 UTC
FYI, I think I have full support for symbol versioning on Solaris 11.4 working now. Could you try the users/nalcock/sol11.4 branch and see if it builds for you? (It's master, not 2.34, but I can backport if needed and if it works for you.)

Solaris 11.N for earlier N also had a broken build for different reasons and is similarly fixed.
Comment 5 Nick Alcock 2021-06-25 16:14:43 UTC
Mail sent out to binutils@ for review.
Comment 6 Sourceware Commits 2021-09-27 19:33:52 UTC
The master branch has been updated by Nick Alcock <nix@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=bc4b1401129c755eb78d434ae88605478f4299f1

commit bc4b1401129c755eb78d434ae88605478f4299f1
Author: Nick Alcock <nick.alcock@oracle.com>
Date:   Mon Sep 27 20:31:21 2021 +0100

    libtool.m4: augment symcode for Solaris 11
    
    This reports common symbols like GNU nm, via a type code of 'C'.
    
    ChangeLog
    2021-09-27  Nick Alcock  <nick.alcock@oracle.com>
    
            PR libctf/27967
            * libtool.m4 (lt_cv_sys_global_symbol_pipe): Augment symcode for
            Solaris 11.
Comment 7 Sourceware Commits 2021-09-27 19:33:57 UTC
The master branch has been updated by Nick Alcock <nix@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=bef9ef8ca0f941d743c77cc55b5fe7985990b2a7

commit bef9ef8ca0f941d743c77cc55b5fe7985990b2a7
Author: Nick Alcock <nick.alcock@oracle.com>
Date:   Mon Sep 27 20:31:21 2021 +0100

    libtool.m4: fix nm BSD flag detection
    
    Libtool needs to get BSD-format (or MS-format) output out of the system
    nm, so that it can scan generated object files for symbol names for
    -export-symbols-regex support.  Some nms need specific flags to turn on
    BSD-formatted output, so libtool checks for this in its AC_PATH_NM.
    Unfortunately the code to do this has a pair of interlocking flaws:
    
     - it runs the test by doing an nm of /dev/null.  Some platforms
       reasonably refuse to do an nm on a device file, but before now this
       has only been worked around by assuming that the error message has a
       specific textual form emitted by Tru64 nm, and that getting this
       error means this is Tru64 nm and that nm -B would work to produce
       BSD-format output, even though the test never actually got anything
       but an error message out of nm -B.  This is fixable by nm'ing *nm
       itself* (since we necessarily have a path to it).
    
     - the test is entirely skipped if NM is set in the environment, on the
       grounds that the user has overridden the test: but the user cannot
       reasonably be expected to know that libtool wants not only nm but
       also flags forcing BSD-format output.  Worse yet, one such "user" is
       the top-level Cygnus configure script, which neither tests for
       nor specifies any BSD-format flags.  So platforms needing BSD-format
       flags always fail to set them when run in a Cygnus tree, breaking
       -export-symbols-regex on such platforms.  Libtool also needs to
       augment $LD on some platforms, but this is done unconditionally,
       augmenting whatever the user specified: the nm check should do the
       same.
    
       One wrinkle: if the user has overridden $NM, a path might have been
       provided: so we use the user-specified path if there was one, and
       otherwise do the path search as usual.  (If the nm specified doesn't
       work, this might lead to a few extra pointless path searches -- but
       the test is going to fail anyway, so that's not a problem.)
    
    (Tested with NM unset, and set to nm, /usr/bin/nm, my-nm where my-nm is a
    symlink to /usr/bin/nm on the PATH, and /not-on-the-path/my-nm where
    *that* is a symlink to /usr/bin/nm.)
    
    ChangeLog
    2021-09-27  Nick Alcock  <nick.alcock@oracle.com>
    
            PR libctf/27967
            * libtool.m4 (LT_PATH_NM): Try BSDization flags with a user-provided
            NM, if there is one.  Run nm on itself, not on /dev/null, to avoid
            errors from nms that refuse to work on non-regular files.  Remove
            other workarounds for this problem.  Strip out blank lines from the
            nm output.
Comment 8 Sourceware Commits 2021-09-27 19:34:03 UTC
The master branch has been updated by Nick Alcock <nix@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=ea9c2009115d7e00732f5ad316c10a171fc66a53

commit ea9c2009115d7e00732f5ad316c10a171fc66a53
Author: Nick Alcock <nick.alcock@oracle.com>
Date:   Mon Sep 27 20:31:21 2021 +0100

    libctf: try several possibilities for linker versioning flags
    
    Checking for linker versioning by just grepping ld --help output for
    mentions of --version-script is inadequate now that Solaris 11.4
    implements a --version-script with different semantics.  Try linking a
    test program with a small wildcard-using version script with each
    supported set of flags in turn, to make sure that linker versioning is
    not only advertised but actually works.
    
    The Solaris "GNU-compatible" linker versioning is not quite
    GNU-compatible enough, but we can work around the differences by
    generating a new version script that removes the comments from the
    original (Solaris ld requires #-style comments), and making another
    version script for libctf-nonbfd in particular which doesn't mention any
    of the symbols that appear in libctf.la, to avoid Solaris ld introducing
    corresponding new NOTYPE symbols to match the version script.
    
    libctf/ChangeLog
    2021-09-27  Nick Alcock  <nick.alcock@oracle.com>
    
            PR libctf/27967
            * configure.ac (VERSION_FLAGS): Replace with...
            (ac_cv_libctf_version_script): ... this multiple test.
            (VERSION_FLAGS_NOBFD): Substitute this too.
            * Makefile.am (libctf_nobfd_la_LDFLAGS): Use it.  Split out...
            (libctf_ldflags_nover): ... non-versioning flags here.
            (libctf_la_LDFLAGS): Use it.
            * libctf.ver: Give every symbol not in libctf-nobfd a comment on
            the same line noting as much.
Comment 9 Nick Alcock 2023-07-05 16:14:43 UTC
This is now in a released binutils, so I think we can probably close it.