Bug 12557 - gcc & Firefox fail to build with -flto and --as-needed LDFLAGS
Summary: gcc & Firefox fail to build with -flto and --as-needed LDFLAGS
Status: NEW
Alias: None
Product: binutils
Classification: Unclassified
Component: ld (show other bugs)
Version: 2.22
: P2 normal
Target Milestone: ---
Assignee: unassigned
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-03-09 19:45 UTC by Octoploid
Modified: 2011-09-02 13:58 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed: 2011-03-11 19:20:55


Attachments
mozconfig (521 bytes, text/plain)
2011-03-10 16:30 UTC, Octoploid
Details
Correct mozconfig (use this one) (550 bytes, text/plain)
2011-03-10 16:37 UTC, Octoploid
Details
output_bad (10.88 KB, text/plain)
2011-03-10 17:43 UTC, Octoploid
Details
map_bad (7.63 KB, text/plain)
2011-03-10 17:43 UTC, Octoploid
Details
output_good (9.35 KB, text/plain)
2011-03-10 17:43 UTC, Octoploid
Details
map_good (7.61 KB, text/plain)
2011-03-10 17:45 UTC, Octoploid
Details
output_good (9.21 KB, text/plain)
2011-03-11 07:27 UTC, Octoploid
Details
map_good (7.57 KB, text/plain)
2011-03-11 07:28 UTC, Octoploid
Details
gcc_verbose_good (1.97 KB, text/plain)
2011-03-11 18:22 UTC, Octoploid
Details
gcc_verbose_bad (3.45 KB, text/plain)
2011-03-11 18:23 UTC, Octoploid
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Octoploid 2011-03-09 19:45:33 UTC
Please see also:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45375#c53

Firefox 4 fails to build with the following flags:
CXXFLAGS="-march=native -fpermissive -flto=4 -fuse-linker-plugin -fwhole-program"
LDFLAGS="-Wl,-O1,--hash-style=gnu,--as-needed,--no-keep-memory"

c++ -o xpcshell -fno-rtti -fno-exceptions -Wall -Wpointer-arith
-Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor
-Wcast-align -Wno-invalid-offsetof -Wno-variadic-macros -Werror=return-type
-Wno-long-long -march=native -fpermissive -flto=4 -fuse-linker-plugin
-fwhole-program -fno-strict-aliasing -fshort-wchar -pthread -pipe -DNDEBUG
-DTRIMMED -O3  xpcshell.o   -lpthread
-Wl,-O1,--hash-style=gnu,--as-needed,--no-keep-memory 
-Wl,-rpath-link,/var/tmp/mozilla-central/moz-build-dir/dist/bin
-Wl,-rpath-link,/usr/lib  -L../../../../dist/bin -L../../../../dist/lib
../../../../dist/lib/libxpcomglue_s.a
-L/var/tmp/mozilla-central/moz-build-dir/dist/bin -lxpcom -lmozalloc -lxul 
-L/var/tmp/mozilla-central/moz-build-dir/dist/bin -lxpcom -lmozalloc -lxul  
-Wl,-R/usr/lib64 -L/usr/lib64 -lplds4 -lplc4 -lnspr4 -lpthread -ldl -ldl
../../../../dist/bin/libxul.so: undefined reference to `PR_smprintf_free'
../../../../dist/bin/libxul.so: undefined reference to `PR_SetEnv'
../../../../dist/bin/libxul.so: undefined reference to `PR_Now'
../../../../dist/bin/libxul.so: undefined reference to `PR_GetErrorText'
../../../../dist/bin/libxul.so: undefined reference to `PR_FindFunctionSymbol'
../../../../dist/bin/libxul.so: undefined reference to `PR_PushIOLayer'
../../../../dist/bin/libxul.so: undefined reference to `PR_ntohs'
../../../../dist/bin/libxul.so: undefined reference to `PR_FormatTimeUSEnglish'
../../../../dist/bin/libxul.so: undefined reference to `PR_MemMap'
../../../../dist/bin/libxul.so: undefined reference to `PR_LocalTimeParameters'
../../../../dist/bin/libxul.so: undefined reference to `PR_GetDefaultIOMethods'
../../../../dist/bin/libxul.so: undefined reference to `PR_ReadDir'
../../../../dist/bin/libxul.so: undefined reference to `PR_SetPollableEvent'
../../../../dist/bin/libxul.so: undefined reference to `PR_FindSymbol'
/usr/lib/libssl3.so: undefined reference to `PR_OpenAnonFileMap'
/usr/lib/libssl3.so: undefined reference to `PR_ExportFileMapAsString'
../../../../dist/bin/libxul.so: undefined reference to `PR_Delete'
../../../../dist/bin/libxul.so: undefined reference to `PR_AtomicSet'
/usr/lib/libnss3.so: undefined reference to `PR_NewRWLock'
../../../../dist/bin/libxul.so: undefined reference to `PR_SetNetAddr'
...

It builds fine without the "--as-needed" LDFLAG.
Comment 1 H.J. Lu 2011-03-09 23:12:56 UTC
Can you extra a small testcase?
Comment 2 Octoploid 2011-03-10 06:31:09 UTC
The problem comes from:
"-Wl,-R/usr/lib64 -L/usr/lib64 -lplds4 -lplc4 -lnspr4 -lpthread -ldl -ldl"
If I remove "-lnspr4" from above, ld links without any problem.

When I remove the whole system library part:
"-Wl,-R/usr/lib64 -L/usr/lib64 -lplds4 -lplc4 -lnspr4 -lpthread -ldl -ldl" 
ld still links fine.

So it seems that ld gets irritated when you explicitly specify system 
libraries that it already figured out itself in the -flto/--as-needed case.
Comment 3 H.J. Lu 2011-03-10 15:59:53 UTC
(In reply to comment #2)
> The problem comes from:
> "-Wl,-R/usr/lib64 -L/usr/lib64 -lplds4 -lplc4 -lnspr4 -lpthread -ldl -ldl"
> If I remove "-lnspr4" from above, ld links without any problem.
> 

You have 2 very different libnspr4.soes.  Please make sure that
linker sees the correct one first.
Comment 4 Octoploid 2011-03-10 16:30:57 UTC
Created attachment 5285 [details]
mozconfig

No, there's only one libnspr4.so on my system.
# cd /
/ # find . -name "libnspr4*"
./usr/lib/libnspr4.so
./usr/lib/libnspr4.so.8
/ #

And I build Firefox with --with-system-nspr ...
I've attached my .mozconfig in case someone would like
to reproduce the issue. (Just run "make -f client.mk"
in the Firefox top directory)
Comment 5 Octoploid 2011-03-10 16:37:22 UTC
Created attachment 5286 [details]
Correct mozconfig (use this one)
Comment 6 H.J. Lu 2011-03-10 17:20:13 UTC
Please provide the output of -Wl,--verbose=2,-Map,map as well as map
file.  I need both working/non-working outputs.
Comment 7 Octoploid 2011-03-10 17:43:09 UTC
Created attachment 5287 [details]
output_bad
Comment 8 Octoploid 2011-03-10 17:43:32 UTC
Created attachment 5288 [details]
map_bad
Comment 9 Octoploid 2011-03-10 17:43:59 UTC
Created attachment 5289 [details]
output_good
Comment 10 Octoploid 2011-03-10 17:45:27 UTC
Created attachment 5290 [details]
map_good

Everything is attached. 

Good > without --as-needed
Bad > with --as-needed
Comment 11 H.J. Lu 2011-03-10 18:18:05 UTC
I will take a look.
Comment 12 Octoploid 2011-03-10 18:28:19 UTC
Thanks. Unfortunately nothing obvious stands out in vimdiff.
Comment 13 Octoploid 2011-03-11 07:27:50 UTC
Created attachment 5292 [details]
output_good

This may be better for comparison:
removed:
"-Wl,-R/usr/lib64 -L/usr/lib64 -lplds4 -lplc4 -lnspr4 -lpthread -ldl -ldl"
Comment 14 Octoploid 2011-03-11 07:28:28 UTC
Created attachment 5293 [details]
map_good
Comment 15 Octoploid 2011-03-11 07:34:09 UTC
Now two lines stands out in "vimdiff verbose_good verbose_bad":

libnspr4.so needed by ../../../../dist/bin/libxul.so
found libnspr4.so at /usr/lib/libnspr4.so

These two lines are missing in verbose_bad...
Comment 16 H.J. Lu 2011-03-11 17:57:01 UTC
Please show the output of "g++ -v ...".
Comment 17 Octoploid 2011-03-11 18:22:34 UTC
Created attachment 5294 [details]
gcc_verbose_good
Comment 18 Octoploid 2011-03-11 18:23:03 UTC
Created attachment 5295 [details]
gcc_verbose_bad
Comment 19 H.J. Lu 2011-03-11 18:36:35 UTC
[hjl@gnu-6 pr12557]$ cat x.c
int x = 20;
[hjl@gnu-6 pr12557]$ cat foo.c
#include <stdio.h>

extern int x;

void
foo ()
{
  printf ("%d\n", x);
}
[hjl@gnu-6 pr12557]$ cat main.c
extern void foo ();

int
main ()
{
  foo ();
  return 0;
}
[hjl@gnu-6 pr12557]$ make
g++  -shared -fPIC -o libx.so x.c
g++  -shared -fPIC -o libfoo.so foo.c libx.so
g++  -Wl,--as-needed -o x main.o -L. -lfoo -lx -Wl,-R,.
main.o: In function `main':
main.c:(.text+0xa): undefined reference to `foo'
collect2: ld returned 1 exit status
make: *** [x] Error 1
[hjl@gnu-6 pr12557]$
Comment 20 H.J. Lu 2011-03-11 18:54:47 UTC
(In reply to comment #19)
> [hjl@gnu-6 pr12557]$ cat x.c

This test is wrong.
Comment 21 H.J. Lu 2011-03-11 19:02:11 UTC
Exactly which linker are you using?
Comment 22 Octoploid 2011-03-11 19:02:50 UTC
GNU ld (Linux/GNU Binutils) 2.21.51.0.7.20110306
Comment 23 H.J. Lu 2011-03-11 19:20:55 UTC
I really need a small testcase.
Comment 24 Octoploid 2011-03-11 19:24:33 UTC
>I really need a small testcase.

Understood. I will take a deeper look.
Comment 25 Octoploid 2011-03-11 21:02:12 UTC
Even gcc fails to build with the same combination (-flto and Wl,--as-needed):

...
/var/tmp/gcc_build_dir/./prev-gcc/g++ -B/var/tmp/gcc_build_dir/./prev-gcc/ -B/usr/x86_64-pc-linux-gnu/bin/ -nostdinc++ -B/var/tmp/gcc_build_dir/prev-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs -I/var/tmp/gcc_build_dir/prev-x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu -I/var/tmp/gcc_build_dir/prev-x86_64-pc-linux-gnu/libstdc++-v3/include -I/var/tmp/gcc/libstdc++-v3/libsupc++ -L/var/tmp/gcc_build_dir/prev-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs  -march=native -O2 -pipe -flto=jobserver -frandom-seed=1 -fprofile-use -DIN_GCC   -W -Wall -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings   -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc  gcov.o intl.o version.o errors.o ../libcpp/libcpp.a   ../libiberty/libiberty.a ../libdecnumber/libdecnumber.a  -o gcov
/lib64/libc.so.6: undefined reference to `_dl_argv@GLIBC_PRIVATE'
/lib64/libc.so.6: undefined reference to `_rtld_global_ro@GLIBC_PRIVATE'
/lib64/libc.so.6: undefined reference to `__tls_get_addr@GLIBC_2.3'
/lib64/libc.so.6: undefined reference to `_rtld_global@GLIBC_PRIVATE'
/lib64/libc.so.6: undefined reference to `__libc_enable_secure@GLIBC_PRIVATE'
collect2: ld returned 1 exit status
make[3]: *** [gcov] Error 1
make[3]: *** Waiting for unfinished jobs....
rm gfortran.pod cpp.pod gcov.pod gcc.pod fsf-funding.pod gfdl.pod
make[3]: Leaving directory `/var/tmp/gcc_build_dir/gcc'
make[2]: *** [all-stagefeedback-gcc] Error 2
make[2]: Leaving directory `/var/tmp/gcc_build_dir'
make[1]: *** [stagefeedback-bubble] Error 2
make[1]: Leaving directory `/var/tmp/gcc_build_dir'
make: *** [profiledbootstrap] Error 2

Configured with:
../gcc/configure --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/4.6.0 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.6.0/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.6.0 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.6.0/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.6.0/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.6.0/include/g++-v4 --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --disable-altivec --disable-fixed-point --without-ppl --without-cloog --enable-lto --enable-nls --without-included-gettext --with-system-zlib --disable-werror --enable-build-with-cxx --enable-initfini-array --with-gold --enable-secureplt --enable-libmudflap --disable-libssp --enable-libgomp --enable-cld --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/4.6.0/python --enable-checking=release --disable-libgcj --enable-languages=c,c++,fortran --enable-shared --enable-threads=posix --enable-__cxa_atexit --disable-multilib --enable-clocale=gnu --with-build-config=bootstrap-lto

And build with:
make -j4 BOOT_CFLAGS="-march=native -O2 -pipe" STAGE1_CFLAGS="-march=native -O2 -pipe" CFLAGS_FOR_TARGET="-march=native -O2 -pipe" LDFLAGS="-Wl,-O1,--hash-style=gnu,--as-needed" profiledbootstrap
Comment 26 Octoploid 2011-03-11 22:12:02 UTC
Hmm, the LDFLAGS are simply ignored by gcc.
And gcc builds fine when I configure it with:
--with-boot-ldflags=-Wl,-O1,--hash-style=gnu,--as-needed
Comment 27 H.J. Lu 2011-03-14 15:53:24 UTC
I have no problems with gcc bootstrap using

.../configure --enable-clocale=gnu --with-system-zlib --with-demangler-in-ld   --prefix=/usr/gcc-4.7.0 --with-local-prefix=/usr/local --enable-gnu-indirect-function --with-boot-ldflags=-Wl,-O1,--hash-style=gnu,--as-needed --with-build-config=bootstrap-lto --enable-cloog-backend=isl --with-ppl-include=/opt/gnu/include --with-ppl-lib=/opt/gnu/lib64 --with-cloog-include=/opt/gnu/include --with-cloog-lib=/opt/gnu/lib64 --with-fpmath=sse
Comment 28 Octoploid 2011-03-14 16:39:13 UTC
(In reply to comment #27)
> I have no problems with gcc bootstrap using
> 
> .../configure --enable-clocale=gnu --with-system-zlib --with-demangler-in-ld  
> --prefix=/usr/gcc-4.7.0 --with-local-prefix=/usr/local
> --enable-gnu-indirect-function
> --with-boot-ldflags=-Wl,-O1,--hash-style=gnu,--as-needed
> --with-build-config=bootstrap-lto --enable-cloog-backend=isl
> --with-ppl-include=/opt/gnu/include --with-ppl-lib=/opt/gnu/lib64
> --with-cloog-include=/opt/gnu/include --with-cloog-lib=/opt/gnu/lib64
> --with-fpmath=sse

Yes, the problem only occurs when you configure gcc without "--with-boot-ldflags=-Wl,-O1,--hash-style=gnu,--as-needed".

I've tested this a little further:

/var/tmp/gcc_build_dir/./prev-gcc/g++ -B/lib -B/var/tmp/gcc_build_dir/./prev-gcc/ -B/usr/x86_64-pc-linux-gnu/bin/ -nostdinc++ -B/var/tmp/gcc_build_dir/prev-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs -L/var/tmp/gcc_build_dir/prev-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs -pipe -flto -frandom-seed=1 -DIN_GCC -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc gcov.o intl.o version.o errors.o ../libiberty/libiberty.a -o gcov
/lib64/libc.so.6: undefined reference to `_dl_argv@GLIBC_PRIVATE'
/lib64/libc.so.6: undefined reference to `_rtld_global_ro@GLIBC_PRIVATE'
/lib64/libc.so.6: undefined reference to `__tls_get_addr@GLIBC_2.3'
/lib64/libc.so.6: undefined reference to `_rtld_global@GLIBC_PRIVATE'
/lib64/libc.so.6: undefined reference to `__libc_enable_secure@GLIBC_PRIVATE'
collect2: ld returned 1 exit status

Without -static-libstdc++ it compiles fine.
Without -flto it also compiles fine.
When I use xgcc instead of g++ there aren't any problems.
When I use gold as linker there aren't any problems either.
Comment 29 H.J. Lu 2011-03-15 22:27:47 UTC
It is fixed by commit d81cd6bcc0c848feb3785987de8a16196092e943 on
hjl/lto-mixed branch at

http://git.kernel.org/?p=devel/binutils/hjl/x86.git;a=summary
Comment 30 Octoploid 2011-03-16 08:06:47 UTC
Thanks H.J.. Both issues are fixed now.
Comment 31 Richard Biener 2011-09-02 10:36:33 UTC
Simple testcase

double x;
int main()
{
  return x*x*x*x;
}

> gcc-4.6 -o t t.o -O -ffast-math -flto
> gcc-4.6 -o t t.o -flto
/tmp/ccIqK5i6.ltrans0.ltrans.o: In function `main':
ccIqK5i6.ltrans0.o:(.text+0x19): undefined reference to `pow'
collect2: ld returned 1 exit status
> gcc-4.6 -o t t.o -flto -lm
> gcc-4.6 -o t t.o -flto -Wl,--as-needed -lm

which works for me with the 2.21.1 release.
Comment 32 H.J. Lu 2011-09-02 13:58:28 UTC
(In reply to comment #31)
> Simple testcase
> 
> double x;
> int main()
> {
>   return x*x*x*x;
> }
> 
> > gcc-4.6 -o t t.o -O -ffast-math -flto
> > gcc-4.6 -o t t.o -flto
> /tmp/ccIqK5i6.ltrans0.ltrans.o: In function `main':
> ccIqK5i6.ltrans0.o:(.text+0x19): undefined reference to `pow'
> collect2: ld returned 1 exit status
> > gcc-4.6 -o t t.o -flto -lm
> > gcc-4.6 -o t t.o -flto -Wl,--as-needed -lm
> 
> which works for me with the 2.21.1 release.

This looks like a GCC bug to me. If GCC turns 
x*x*x*x into pow, it should add -lm.