Bug 15648 - multiple definition of `__lll_lock_wait_private'
Summary: multiple definition of `__lll_lock_wait_private'
Status: NEW
Alias: None
Product: glibc
Classification: Unclassified
Component: nptl (show other bugs)
Version: unspecified
: P2 enhancement
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-06-19 10:15 UTC by Neil
Modified: 2014-06-13 15:03 UTC (History)
5 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:
fweimer: security-


Attachments
multiple definition of `__lll_lock_wait_private' example (19.35 KB, application/x-bzip2)
2013-06-20 02:47 UTC, Neil
Details
multiple definition of `__lll_lock_wait_private' example2 (419 bytes, text/x-csrc)
2013-06-20 06:10 UTC, Neil
Details
link error message (7.36 KB, text/plain)
2013-06-21 01:33 UTC, Neil
Details
Make __lll_lock_wait_private and __lll_unlock_wake_private weak in libc. (468 bytes, patch)
2013-06-21 13:13 UTC, Carlos O'Donell
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Neil 2013-06-19 10:15:08 UTC
Hello,

I build a program with gcc 4.6 and enable link time optimization.
when static linking, ld show error mesage 

/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/libpthread.a(lowlevellock.o): In function `__lll_lock_wait_private':
/build/buildd/eglibc-2.15/nptl/../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/lowlevellock.S:76: multiple definition of `__lll_lock_wait_private'
/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/libc.a(libc-lowlevellock.o):(.text+0x0): first defined here
/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/libpthread.a(lowlevellock.o): In function `__lll_unlock_wake_private':
/build/buildd/eglibc-2.15/nptl/../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/lowlevellock.S:332: multiple definition of `__lll_unlock_wake_private'
/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/libc.a(libc-lowlevellock.o):(.text+0x30): first defined here

I also search relative issue like
http://dev.drobilla.net/ticket/914

I see the glibc source files
libc/nptl/sysdeps/unix/sysv/linux/libc-lowlevellock.c:

#include <lowlevellock.c>

I am confused. Why nptl and libc share the same context?
Is it a potential bug?

I make the patch, and the problem is resolved.
Is this correct for arm?

--- a/libc/ports/sysdeps/unix/sysv/linux/arm/nptl/lowlevellock.c
+++ b/libc/ports/sysdeps/unix/sysv/linux/arm/nptl/lowlevellock.c
@@ -22,6 +22,8 @@
 #include <sys/time.h>
 
 void
+__lll_lock_wait_private (int *futex) __attribute__ ((__weak__));
+void
 __lll_lock_wait_private (int *futex)
 {
   do


Thanks in advance
Neil
Comment 1 Carlos O'Donell 2013-06-19 15:01:20 UTC
We need a complete reproducer for this issue please.

Include exactly the steps you took and the expected versus observed results.

This includes the exact gcc command you used to compile your application, and exactly what version of glibc you were using.

If you are using a distro glibc then you need to file this issue upstream with the distro so they can determine if any of their patches caused the problem.
Comment 2 Neil 2013-06-20 02:47:38 UTC
Created attachment 7085 [details]
multiple definition of `__lll_lock_wait_private'  example

OS version : Ubuntu 12.04
gcc version : gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5)


untar the tarball, and use make to reproduce the bug.

I expect there is no error message.
But I get error message when linking.

compiler options "-ftree-parallelize-loops=2 -flto -O1" are necessary.

gcc -ftree-parallelize-loops=2 -flto -O1 core_list_join.i -c -o core_list_join.o
gcc -ftree-parallelize-loops=2 -flto -O1 core_main.i -c -o core_main.o
gcc -ftree-parallelize-loops=2 -flto -O1 core_matrix.i -c -o core_matrix.o
gcc -ftree-parallelize-loops=2 -flto -O1 core_state.i -c -o core_state.o
gcc -ftree-parallelize-loops=2 -flto -O1 core_util.i -c -o core_util.o
gcc -ftree-parallelize-loops=2 -flto -O1 core_portme.i -c -o core_portme.o
gcc -ftree-parallelize-loops=2 -flto -O1 *.o -static -o coremark.exe
/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/libpthread.a(lowlevellock.o): In function `__lll_lock_wait_private':
/build/buildd/eglibc-2.15/nptl/../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/lowlevellock.S:76: multiple definition of `__lll_lock_wait_private'
/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/libc.a(libc-lowlevellock.o):(.text+0x0): first defined here
/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/libpthread.a(lowlevellock.o): In function `__lll_unlock_wake_private':
/build/buildd/eglibc-2.15/nptl/../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/lowlevellock.S:332: multiple definition of `__lll_unlock_wake_private'
/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/libc.a(libc-lowlevellock.o):(.text+0x30): first defined here
collect2: ld returned 1 exit status
make: *** [all] Error 1
Comment 3 Neil 2013-06-20 02:54:08 UTC
I also try linaro gcc 4.8 toolchain

http://releases.linaro.org/13.05/components/toolchain/binaries/gcc-linaro-arm-linux-gnueabihf-4.8-2013.05_linux.tar.bz2

the same situation

/work/linaro_tools/gcc-linaro-arm-linux-gnueabihf-4.8-2013.04-20130417_linux/bin/arm-linux-gnueabihf-gcc -ftree-parallelize-loops=2 -flto -O1 core_list_join.i -c -o core_list_join.o
/work/linaro_tools/gcc-linaro-arm-linux-gnueabihf-4.8-2013.04-20130417_linux/bin/arm-linux-gnueabihf-gcc -ftree-parallelize-loops=2 -flto -O1 core_main.i -c -o core_main.o
/work/linaro_tools/gcc-linaro-arm-linux-gnueabihf-4.8-2013.04-20130417_linux/bin/arm-linux-gnueabihf-gcc -ftree-parallelize-loops=2 -flto -O1 core_matrix.i -c -o core_matrix.o
/work/linaro_tools/gcc-linaro-arm-linux-gnueabihf-4.8-2013.04-20130417_linux/bin/arm-linux-gnueabihf-gcc -ftree-parallelize-loops=2 -flto -O1 core_state.i -c -o core_state.o
/work/linaro_tools/gcc-linaro-arm-linux-gnueabihf-4.8-2013.04-20130417_linux/bin/arm-linux-gnueabihf-gcc -ftree-parallelize-loops=2 -flto -O1 core_util.i -c -o core_util.o
/work/linaro_tools/gcc-linaro-arm-linux-gnueabihf-4.8-2013.04-20130417_linux/bin/arm-linux-gnueabihf-gcc -ftree-parallelize-loops=2 -flto -O1 core_portme.i -c -o core_portme.o
/work/linaro_tools/gcc-linaro-arm-linux-gnueabihf-4.8-2013.04-20130417_linux/bin/arm-linux-gnueabihf-gcc -ftree-parallelize-loops=2 -flto -O1 *.o -static -o coremark.exe
/work/linaro_tools/gcc-linaro-arm-linux-gnueabihf-4.8-2013.04-20130417_linux/bin/../arm-linux-gnueabihf/libc/usr/lib/arm-linux-gnueabihf/libpthread.a(lowlevellock.o): In function `__lll_lock_wait_private':
lowlevellock.c:(.text+0x0): multiple definition of `__lll_lock_wait_private'
/home/neilh/work/linaro_tools/gcc-linaro-arm-linux-gnueabihf-4.8-2013.04-20130417_linux/bin/../arm-linux-gnueabihf/libc/usr/lib/arm-linux-gnueabihf/libc.a(libc-lowlevellock.o):(.text+0x0): first defined here
collect2: error: ld returned 1 exit status
make: *** [all] Error 1

Hope these information are enough to reproduce the issue.

Regards,
Neil
Comment 4 Carlos O'Donell 2013-06-20 03:32:01 UTC
This doesn't work at all for me for entirely different reasons.

I expect that you haven't actually tried to use -flto.

glibc master is currently unbuildable with -flto because of:
http://sourceware.org/bugzilla/show_bug.cgi?id=15648

You should always see:
checking for .preinit_array/.init_array/.fini_array support... no
configure: error: Need linker with .init_array/.fini_array support.

This is because -flto throws out the .init_array support.

A potential fix is:
diff --git a/configure.in b/configure.in
index 8b11081..415e777 100644
--- a/configure.in
+++ b/configure.in
@@ -1383,7 +1383,7 @@ AC_CACHE_CHECK(for .preinit_array/.init_array/.fini_array support,
               libc_cv_initfini_array, [dnl
 LIBC_TRY_LINK_STATIC([
 int foo (void) { return 1; }
-int (*fp) (void) __attribute__ ((section (".init_array"))) = foo;
+int (*fp) (void) __attribute__ ((section (".init_array"), used)) = foo;
 ],
   [if $READELF -S conftest | fgrep INIT_ARRAY > /dev/null; then
     libc_cv_initfini_array=yes
~~~

However, given that nobody has built with -flto it's going to require considerable work to make it happen.

I'm retitling and marking as enhancement.
Comment 5 Carlos O'Donell 2013-06-20 03:34:51 UTC
(In reply to Carlos O'Donell from comment #4)
> glibc master is currently unbuildable with -flto because of:
> http://sourceware.org/bugzilla/show_bug.cgi?id=15648

Should have been:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51255
Comment 6 Neil 2013-06-20 06:10:19 UTC
Created attachment 7086 [details]
multiple definition of `__lll_lock_wait_private' example2

I don't want to build glibc with -flto option.
I build application with -flto.

The problem is like building thread program which static link with -lc -lpthread

For example:
gcc thread_program.c -lc -lpthread -static
/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/libpthread.a(lowlevellock.o): In function `__lll_lock_wait_private':
/build/buildd/eglibc-2.15/nptl/../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/lowlevellock.S:76: multiple definition of `__lll_lock_wait_private'
/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/libc.a(libc-lowlevellock.o):(.text+0x0): first defined here
/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/libpthread.a(lowlevellock.o): In function `__lll_unlock_wake_private':
/build/buildd/eglibc-2.15/nptl/../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/lowlevellock.S:332: multiple definition of `__lll_unlock_wake_private'
/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/libc.a(libc-lowlevellock.o):(.text+0x30): first defined here
collect2: ld returned 1 exit status

But, I don't know why -flto causes the problem for my first example. :(
Comment 7 Andreas Schwab 2013-06-20 08:15:17 UTC
This has nothing to do with building glibc with -flto.  There are simply two defintions of __lll_lock_wait_private and __lll_unlock_wait_private, one in libpthread.a(lowlevellock.o) and another one in libc.a(libc-lowlevellock.o).  One of them needs to be elided from the static library.
Comment 8 Neil 2013-06-20 09:24:28 UTC
Yes, I see the libc.a and libpthread.a

nm /usr/lib/i386-linux-gnu/libpthread.a
lowlevellock.o:
         U __have_futex_clock_realtime
00000030 T __lll_lock_wait
00000000 T __lll_lock_wait_private
00000060 T __lll_timedlock_wait
000001c0 T __lll_timedwait_tid
00000190 T __lll_unlock_wake
00000160 T __lll_unlock_wake_private
         U _dl_sysinfo

nm /usr/lib/i386-linux-gnu/libc.a
libc-lowlevellock.o:
00000000 T __lll_lock_wait_private
00000030 T __lll_unlock_wake_private
         U _dl_sysinfo

Why not to make __lll_lock_wait_private and __lll_unlock_wake_private in a individual file?

If ld include libc-lowlevellock.o first, it always cause error when includeing lowlevellock.o. Right?
Comment 9 Carlos O'Donell 2013-06-20 12:55:52 UTC
(In reply to Neil from comment #6)
> Created attachment 7086 [details]
> multiple definition of `__lll_lock_wait_private' example2
> 
> I don't want to build glibc with -flto option.
> I build application with -flto.

Thanks, I misunderstood your problem.
Comment 10 Carlos O'Donell 2013-06-20 12:57:25 UTC
(In reply to Carlos O'Donell from comment #9)
> (In reply to Neil from comment #6)
> > Created attachment 7086 [details]
> > multiple definition of `__lll_lock_wait_private' example2
> > 
> > I don't want to build glibc with -flto option.
> > I build application with -flto.
> 
> Thanks, I misunderstood your problem.

Filed:
Bug 15658 - Support building glibc with -flto.
http://sourceware.org/bugzilla/show_bug.cgi?id=15658
Comment 11 Jakub Jelinek 2013-06-20 12:58:47 UTC
(In reply to Neil from comment #8)
> If ld include libc-lowlevellock.o first, it always cause error when
> includeing lowlevellock.o. Right?

But why does that happen?
You link -lc before -lpthread?  That is never going to work properly when linking the libraries statically.
Comment 12 Carlos O'Donell 2013-06-20 13:51:42 UTC
(In reply to Jakub Jelinek from comment #11)
> (In reply to Neil from comment #8)
> > If ld include libc-lowlevellock.o first, it always cause error when
> > includeing lowlevellock.o. Right?
> 
> But why does that happen?
> You link -lc before -lpthread?  That is never going to work properly when
> linking the libraries statically.

A `gcc -v...` would help show what's going on here.

Neil, can you give us the verbose output of the compile?
Comment 13 Neil 2013-06-21 01:22:52 UTC
(In reply to Jakub Jelinek from comment #11)
> (In reply to Neil from comment #8)
> > If ld include libc-lowlevellock.o first, it always cause error when
> > includeing lowlevellock.o. Right?
> 
> But why does that happen?
> You link -lc before -lpthread?  That is never going to work properly when
> linking the libraries statically.

Exactly, statically link -c before -pthread never works well.

I mean I encounter the similar problem like that but I don't link -c before -pthread.

I use gcc4.6 and the option "-ftree-parallelize-loops=2 -flto -O1" to compile program.
Because of option "-ftree-parallelize-loops=2", gcc will auto apply -lpthread to linker.
Then the error message show up.

In this case, I'm not sure this is gcc issue or glibc issue.
But I modify glibc source, the issue can be resolved.
Comment 14 Neil 2013-06-21 01:33:43 UTC
Created attachment 7088 [details]
link error message

I add -v and -Wl,--verbose.
There are more information.
Please see the attachment file.

Following is linking information.

gcc -ftree-parallelize-loops=2 -flto -O1 -save-temps *.o -static -o coremark.exe -v -Wl,--verbose
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/i686-linux-gnu/4.6/lto-wrapper
Target: i686-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.6.3-1ubuntu5' --with-bugurl=file:///usr/share/doc/gcc-4.6/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.6 --enable-shared --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.6 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --enable-objc-gc --enable-targets=all --disable-werror --with-arch-32=i686 --with-tune=generic --enable-checking=release --build=i686-linux-gnu --host=i686-linux-gnu --target=i686-linux-gnu
Thread model: posix
gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5)
COMPILER_PATH=/usr/lib/gcc/i686-linux-gnu/4.6/:/usr/lib/gcc/i686-linux-gnu/4.6/:/usr/lib/gcc/i686-linux-gnu/:/usr/lib/gcc/i686-linux-gnu/4.6/:/usr/lib/gcc/i686-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/i686-linux-gnu/4.6/:/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/:/usr/lib/gcc/i686-linux-gnu/4.6/../../../../lib/:/lib/i386-linux-gnu/:/lib/../lib/:/usr/lib/i386-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/i686-linux-gnu/4.6/../../../:/lib/:/usr/lib/
Reading specs from /usr/lib/gcc/i686-linux-gnu/4.6/libgomp.spec
COLLECT_GCC_OPTIONS='-ftree-parallelize-loops=2' '-flto' '-O1' '-save-temps' '-static' '-o' 'coremark.exe' '-v' '-mtune=generic' '-march=i686' '-pthread'
 /usr/lib/gcc/i686-linux-gnu/4.6/collect2 -plugin /usr/lib/gcc/i686-linux-gnu/4.6/liblto_plugin.so -plugin-opt=/usr/lib/gcc/i686-linux-gnu/4.6/lto-wrapper -plugin-opt=-fresolution=--verbose.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_eh -plugin-opt=-pass-through=-lpthread -plugin-opt=-pass-through=-lc -flto --sysroot=/ --build-id --no-add-needed --as-needed -m elf_i386 --hash-style=gnu -static -z relro -o coremark.exe /usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/crt1.o /usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/crti.o /usr/lib/gcc/i686-linux-gnu/4.6/crtbeginT.o -L/usr/lib/gcc/i686-linux-gnu/4.6 -L/usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu -L/usr/lib/gcc/i686-linux-gnu/4.6/../../../../lib -L/lib/i386-linux-gnu -L/lib/../lib -L/usr/lib/i386-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/i686-linux-gnu/4.6/../../.. core_list_join.o core_main.o coremark.exe.ltrans0.o core_matrix.o core_portme.o core_state.o core_util.o --verbose -lgomp -lrt --start-group -lgcc -lgcc_eh -lpthread -lc --end-group /usr/lib/gcc/i686-linux-gnu/4.6/crtend.o /usr/lib/gcc/i686-linux-gnu/4.6/../../../i386-linux-gnu/crtn.o

Hope this will help you.
Thanks,
Neil
Comment 15 Andreas Schwab 2013-06-21 07:42:48 UTC
The problem is that even though libpthread.a(lowlevellock.o) would resolve the reference to __lll_lock_wait_private and/or __lll_unlock_wake_private the first reference is from libc.a, and libpthread.a will only be rescanned after libc.a cannot resolve any more references.
Comment 16 Jakub Jelinek 2013-06-21 07:52:18 UTC
Static linking of -lpthread never worked well, there are tons of issues with that, which is why in Fedora/RHEL we are just linking all the content of libpthread.a using ld -r into libpthread.o and libpthread.a contains just that single object.  Then this bug doesn't exist.  There are many assumptions (unwritten dependencies) in between libpthread object files, with dynamic linking you simply have all of them together or nothing, but with libpthread, unless you do that ld -r hack, you don't.  Not to mention that many programs (e.g. gcc gthr.h) check using weakref for availability of some pthread_* symbol(s) and if non-NULL, assume all of libpthread is available, rather than random parts from it.

Why exactly are you linking statically?  http://www.akkadia.org/drepper/no_static_linking.html
Comment 17 Carlos O'Donell 2013-06-21 13:13:15 UTC
Created attachment 7089 [details]
Make __lll_lock_wait_private and __lll_unlock_wake_private weak in libc.

Could you test the following patch to see if it fixes the issue for you?

http://sourceware.org/glibc/wiki/Testing/Builds
Comment 18 Carlos O'Donell 2013-06-21 13:21:05 UTC
(In reply to Jakub Jelinek from comment #16)
> Static linking of -lpthread never worked well, there are tons of issues with
> that, which is why in Fedora/RHEL we are just linking all the content of
> libpthread.a using ld -r into libpthread.o and libpthread.a contains just
> that single object.  Then this bug doesn't exist.  There are many
> assumptions (unwritten dependencies) in between libpthread object files,
> with dynamic linking you simply have all of them together or nothing, but
> with libpthread, unless you do that ld -r hack, you don't.  Not to mention
> that many programs (e.g. gcc gthr.h) check using weakref for availability of
> some pthread_* symbol(s) and if non-NULL, assume all of libpthread is
> available, rather than random parts from it.

Then we need to file a bug to fix glibc to use ld -r or gather community consensus that static linking is no longer supported. Current community consensus is that we will continue to support static linking where possible.

> Why exactly are you linking statically? 
> http://www.akkadia.org/drepper/no_static_linking.html

Good question, but it doesn't answer the more fundamental question of glibc support for static linking.
Comment 19 Neil 2013-06-21 14:42:29 UTC
(In reply to Carlos O'Donell from comment #17)
> Created attachment 7089 [details]
> Make __lll_lock_wait_private and __lll_unlock_wake_private weak in libc.
> 
> Could you test the following patch to see if it fixes the issue for you?
> 
> http://sourceware.org/glibc/wiki/Testing/Builds

That's awesome!

This patch fixes the issue for me.

Is this patch OK for the trunk?
Comment 20 Ondrej Bilka 2013-10-02 17:05:25 UTC
Carlos, is your patch ready to libc-alpha?
Comment 21 Carlos O'Donell 2013-10-03 01:00:46 UTC
(In reply to Ondrej Bilka from comment #20)
> Carlos, is your patch ready to libc-alpha?

No. This issue needs to be driven to consensus over the solution to use for libpthread, either we link with `ld -r' and force all of libpthread into the application during a static link or we do it piece-meal dependency by dependency trying to keep the link minimal.

Previous comments from Roland and myself was that we should try to fix these kinds of issues one at a time, but it's looking liked `ld -r' is going to be the only real solution given the number of applications and core tools that use a weakref and a check to test for conditional library inclusion.

This has to go to libca-alpha for discussion, but I don't have time to champion this right now. If you are interested I encourage you to get consensus and implement a solution, I'd be happy to review.