Bug 23350 - multiple prevailing defs for unused variable in lto mode
Summary: multiple prevailing defs for unused variable in lto mode
Status: RESOLVED INVALID
Alias: None
Product: binutils
Classification: Unclassified
Component: ld (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-06-28 12:50 UTC by Martin Liska
Modified: 2020-03-06 00:50 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed: 2018-07-02 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Liska 2018-06-28 12:50:06 UTC
Consider:

$ cat a.i
int wrl;

int main()
{
}

$ cat b.i
int wrl;
int wrl2;

$ cc -flto -O2 -c a.i
$ cc -flto -O2 -c b.i
$ ar rusc libpes.a b.o

$ cc a.o libpes.a libpes.a
lto1: fatal error: multiple prevailing defs for ‘wrl2’
compilation terminated.
lto-wrapper: fatal error: cc returned 1 exit status
compilation terminated.
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status

Where .res file looks as follows:
$ cat libpes.res
3
a.o 2
190 c40a787b82fa11ab PREVAILING_DEF main
194 c40a787b82fa11ab PREVAILING_DEF_IRONLY wrl
libpes.a@0x96 2
188 8e139599392dae30 PREVAILING_DEF_IRONLY wrl2
190 8e139599392dae30 RESOLVED_IR wrl
libpes.a@0x96 2
188 8e139599392dae30 PREVAILING_DEF_IRONLY wrl2
190 8e139599392dae30 RESOLVED_IR wrl

While gold is fine, having following resolution file:

$ cat libpes.res
1
a.o 2
190 c40a787b82fa11ab PREVAILING_DEF main
194 c40a787b82fa11ab PREVAILING_DEF_IRONLY wrl
Comment 1 H.J. Lu 2018-07-02 22:54:27 UTC
Which version of ld are you using?  Master branch works for me:

hjl@gnu-cfl-1 pr23350]$ make clean
rm -f *.o *.so
[hjl@gnu-cfl-1 pr23350]$ cat x.c
int wrl;

int main()
{
}
[hjl@gnu-cfl-1 pr23350]$ cat y.c
int wrl;
int wrl2;
[hjl@gnu-cfl-1 pr23350]$ make
/usr/gcc-7.2.1-x32/bin/gcc -flto -O2   -c -o x.o x.c
/usr/gcc-7.2.1-x32/bin/gcc -flto -O2   -c -o y.o y.c
ar --plugin `/usr/gcc-7.2.1-x32/bin/gcc -print-prog-name=liblto_plugin.so` -rusc liby.a y.o
/usr/gcc-7.2.1-x32/bin/gcc -o x x.o liby.a liby.a
[hjl@gnu-cfl-1 pr23350]$
Comment 2 Martin Liska 2018-07-03 08:59:42 UTC
(In reply to H.J. Lu from comment #1)
> Which version of ld are you using?  Master branch works for me:
> 
> hjl@gnu-cfl-1 pr23350]$ make clean
> rm -f *.o *.so
> [hjl@gnu-cfl-1 pr23350]$ cat x.c
> int wrl;
> 
> int main()
> {
> }
> [hjl@gnu-cfl-1 pr23350]$ cat y.c
> int wrl;
> int wrl2;
> [hjl@gnu-cfl-1 pr23350]$ make
> /usr/gcc-7.2.1-x32/bin/gcc -flto -O2   -c -o x.o x.c
> /usr/gcc-7.2.1-x32/bin/gcc -flto -O2   -c -o y.o y.c
> ar --plugin `/usr/gcc-7.2.1-x32/bin/gcc -print-prog-name=liblto_plugin.so`
> -rusc liby.a y.o
> /usr/gcc-7.2.1-x32/bin/gcc -o x x.o liby.a liby.a
> [hjl@gnu-cfl-1 pr23350]$

I've just updated to
GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.30.0.20180320-5

and the issue is gone. Thus closing as invalid.
Comment 3 Martin Liska 2018-07-09 14:20:38 UTC
So still present, shows here:

$ cat main.i
int wrl;

$ cat lib.i
int wrl;
void a() {}

$ gcc -c -flto main.i && gcc -c -flto lib.i && ar rusc lib.a lib.o
$ gcc main.o lib.a lib.a
lto1: fatal error: multiple prevailing defs for ‘a’
compilation terminated.
lto-wrapper: fatal error: gcc returned 1 exit status
compilation terminated.
/usr/lib64/gcc/x86_64-suse-linux/8/../../../../x86_64-suse-linux/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status

For:

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib64/gcc/x86_64-suse-linux/8/lto-wrapper
OFFLOAD_TARGET_NAMES=hsa:nvptx-none
Target: x86_64-suse-linux
Configured with: ../configure --prefix=/usr --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,ada,go --enable-offload-targets=hsa,nvptx-none=/usr/nvptx-none, --without-cuda-driver --enable-checking=release --disable-werror --with-gxx-include-dir=/usr/include/c++/8 --enable-ssp --disable-libssp --disable-libvtv --disable-cet --disable-libcc1 --enable-plugin --with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux' --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch --enable-version-specific-runtime-libs --with-gcc-major-version-only --enable-linker-build-id --enable-linux-futex --enable-gnu-indirect-function --program-suffix=-8 --without-system-libunwind --enable-multilib --with-arch-32=x86-64 --with-tune=generic --build=x86_64-suse-linux --host=x86_64-suse-linux
Thread model: posix
gcc version 8.1.1 20180614 [gcc-8-branch revision 261584] (SUSE Linux) 

$ ld --version
GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.30.0.20180320-5
Copyright (C) 2018 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
Comment 4 zenith432 2018-07-09 19:26:20 UTC
What difference does it make?  This is undefined behavior.  It doesn't link -fno-lto either.

gcc -c main.i && gcc -c lib.i && ar rusc lib.a lib.o
gcc main.o lib.a lib.a

/usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/crt1.o: in function `_start':
(.text+0x20): undefined reference to `main'
collect2: error: ld returned 1 exit status

Find an example where the -fno-lto link succeeds.

ld is loading the object files in the libraries because it's searching for main.
If I add a main() to lib.i it links successfully with -flto - the 2nd load of lib.a sets the duplicate symbol resolutions to PREEMPTED_IR.  The error with -flto is only if main is not found - which is also an error in -fno-lto.
Comment 5 H.J. Lu 2018-07-09 21:02:35 UTC
(In reply to Martin Liska from comment #3)
> So still present, shows here:
> 
> $ cat main.i
> int wrl;
> 
> $ cat lib.i
> int wrl;
> void a() {}
> 
> $ gcc -c -flto main.i && gcc -c -flto lib.i && ar rusc lib.a lib.o
> $ gcc main.o lib.a lib.a
> lto1: fatal error: multiple prevailing defs for ‘a’
> compilation terminated.
> lto-wrapper: fatal error: gcc returned 1 exit status
> compilation terminated.
> /usr/lib64/gcc/x86_64-suse-linux/8/../../../../x86_64-suse-linux/bin/ld:
> error: lto-wrapper failed
> collect2: error: ld returned 1 exit status
> 

Linker checks lib.a to see if there is a non-COMMON definition for
wrl.  But linker never includes lib.o in the final link.  What makes
lto1 believe that lib.o is needed?
Comment 6 Martin Liska 2018-07-10 07:25:02 UTC
Sorry for not precise reproducer. I was reducing that from an existing package that normally succeeds w/o LTO.

So what about this:

$ cat main.i
int wrl;

int main ()
{
  return 0;
}

$ cat lib.i
int wrl;
void a() {}

$ gcc -c main.i && gcc -c lib.i && ar rusc lib.a lib.o && gcc main.o lib.a lib.a

This works fine, however:

$ gcc -c -flto main.i && gcc -c -flto lib.i && ar rusc lib.a lib.o && gcc main.o lib.a lib.a -flto --save-temps
lto1: fatal error: multiple prevailing defs for ‘a’
compilation terminated.
lto-wrapper: fatal error: gcc returned 1 exit status
compilation terminated.
/usr/lib64/gcc/x86_64-suse-linux/8/../../../../x86_64-suse-linux/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status

$ cat lib.res
3
main.o 2
198 5dfd76773c77e89 PREVAILING_DEF main
202 5dfd76773c77e89 PREVAILING_DEF_IRONLY wrl
lib.a@0x92 2
198 e65cc6ac1d79da7a PREVAILING_DEF_IRONLY a
202 e65cc6ac1d79da7a RESOLVED_IR wrl
lib.a@0x92 2
198 e65cc6ac1d79da7a PREVAILING_DEF_IRONLY a
202 e65cc6ac1d79da7a RESOLVED_IR wrl

Hope it's fine test-case?
Comment 7 H.J. Lu 2018-07-10 19:31:24 UTC
get_symbols has

    if (!blhe)
        {
          /* The plugin is called to claim symbols in an archive element
             from plugin_object_p.  But those symbols aren't needed to
             create output.  They are defined and referenced only within
             IR.  */

But there is no way to tell LTO plugin to drop this symbol.
Comment 8 H.J. Lu 2018-07-11 15:53:03 UTC
This is a GCC bug:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86490
Comment 9 Dan Bonachea 2020-03-06 00:50:53 UTC
We recently hit what appears to be this same problem with our software.

I stumbled upon a workaround, which is to declare one of the tentative definitions as extern. Here's a demonstration applied to the code from comment #6:

$ gcc --version
gcc (GCC) 9.2.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ tail *.c
==> lib.c <==
int wrl;
extern int a() { return 0; }


==> main.c <==
#ifdef WORKAROUND
extern int wrl;
#else
int wrl;
#endif

int main () { return 0; }

$ gcc -c -flto main.c && gcc -c -flto lib.c && gcc-ar rusc lib.a lib.o && gcc -flto main.o lib.a lib.a             
lto1: fatal error: multiple prevailing defs for 'a'
compilation terminated.
lto-wrapper: fatal error: gcc returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status

$ gcc -DWORKAROUND -c -flto main.c && gcc -c -flto lib.c && gcc-ar rusc lib.a lib.o && gcc -flto main.o lib.a lib.a

In our case we are always providing a common-block definition from inside the library, so the workaround just meant declaring as extern in the library header file which is the declaration client objects see. Probably not applicable to all situations, but good enough to solve our problem.

Hope this helps..