Bug 15444 - Use of -g for large project results in: relocation truncated to fit: R_X86_64_32 against `.debug_info'
Summary: Use of -g for large project results in: relocation truncated to fit: R_X86_64...
Status: RESOLVED WONTFIX
Alias: None
Product: binutils
Classification: Unclassified
Component: ld (show other bugs)
Version: 2.23
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-05-07 22:08 UTC by Peeter Joot
Modified: 2013-05-22 08:44 UTC (History)
3 users (show)

See Also:
Host:
Target: x86_64-suse-linux
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Peeter Joot 2013-05-07 22:08:10 UTC
Attempting to link a large shared lib where too much code has been compiled with -g results in:

/usr/lib64/gcc/x86_64-suse-linux/4.1.2/../../../../lib64/crtn.o:(.debug_aranges+0x6):     relocation truncated to fit: R_X86_64_32 against `.debug_info'

Adding -Wl,-verbose to the link flags, shows that all the .o's and .a's in the link command line are processed without error, and that it's only (presumably) near the end when this crtn.o is processed do things go badly:

attempt to open /usr/lib64/gcc/x86_64-suse-linux/4.1.2//libc.so failed
attempt to open /usr/lib64/gcc/x86_64-suse-linux/4.1.2//libc.a failed
attempt to open /usr/lib64/gcc/x86_64-suse-linux/4.1.2/../../../../lib64/libc.so succeeded
opened script file /usr/lib64/gcc/x86_64-suse-linux/4.1.2/../../../../lib64/libc.so
opened script file /usr/lib64/gcc/x86_64-suse-linux/4.1.2/../../../../lib64/libc.so
attempt to open /lib64/libc.so.6 succeeded
/lib64/libc.so.6
attempt to open /usr/lib64/libc_nonshared.a succeeded
attempt to open /lib64/ld-linux-x86-64.so.2 succeeded
/lib64/ld-linux-x86-64.so.2
attempt to open /usr/lib64/gcc/x86_64-suse-linux/4.1.2/crtendS.o succeeded
/usr/lib64/gcc/x86_64-suse-linux/4.1.2/crtendS.o
attempt to open /usr/lib64/gcc/x86_64-suse-linux/4.1.2/../../../../lib64/crtn.o succeeded
/usr/lib64/gcc/x86_64-suse-linux/4.1.2/../../../../lib64/crtn.o
/usr/lib64/gcc/x86_64-suse-linux/4.1.2/../../../../lib64/crtn.o:(.debug_aranges+0x6): relocation truncated to fit: R_X86_64_32 against `.debug_info'

This error occurs with both the default sles10 linker (GNU ld version 2.16.91.0.5 20051219), and with the newest version: GNU ld (GNU Binutils) 2.23.2)

I'm unsure if this is a
1) an ld bug, or
2) a bug in the crtn.o file (this is from the sles10 version of gcc, version: gcc (GCC) 4.1.2 20070115, which is old), or
3) a limitation (if so, is it one imposed by the c runtime crtn.o file or the linker?)
Comment 1 Peeter Joot 2013-05-07 23:35:12 UTC
I've ruled out the gcc version, and see the same relocation truncated error when the gcc 4.8 version of libgcc/crtbegin.o/crtend.o/... files.

However, the reason for this not making a difference appears to be this crtn.o file isn't part of the compiler, but is part of the C runtime library:

$ rpm -qf /usr/lib64/crtn.o
glibc-devel-2.4-31.81.11

... so it doesn't make any difference whether I use the default system compiler crt* files or the ones from gcc 4.8 (We are actually using the intel icpc compiler to invoke the linker, but it uses the system libgcc and crtbegin/end.o's by default).
Comment 2 H.J. Lu 2013-05-08 17:47:21 UTC
Your debug info sections are too big.  You need to
reduce sizes of debug info sections in a single
shared library.
Comment 3 Peeter Joot 2013-05-08 17:58:57 UTC
Is there a mechanism to do this?
Comment 4 H.J. Lu 2013-05-08 18:35:33 UTC
You can avoid -g or break one big shared library into 2.
Comment 5 Paul Pluzhnikov 2013-05-08 18:47:13 UTC
(In reply to comment #4)
> You can avoid -g or break one big shared library into 2.

Neither of these approaches is particularly appealing (the tail is wagging the dog here).

Peeter may be interested in the fission work:
http://gcc.gnu.org/ml/gcc/2011-10/msg00326.html
http://gcc.gnu.org/wiki/DebugFission
Comment 6 Peeter Joot 2013-05-08 19:00:01 UTC
Looks like the fission work is a plan to take --build-id or debug-link separation of the binary and debug info to the next logical step, separating it into a different file at link time.  I was looking for such a mechanism when I asked the following:

http://stackoverflow.com/questions/16408893/gnu-ld-gdb-separate-debug-files-how-to-produce-the-debug-file-when-theres-too

What's the state of this work?
Comment 7 Paul Pluzhnikov 2013-05-08 19:07:41 UTC
(In reply to comment #6)

> What's the state of this work?

It is committed in GCC-4.8, trunk binutils/gold, and GDB 7.6.

We are starting to use it at Google, but it is very fresh, and we are discovering some bugs in both gold and GDB.

If you can wait another 3 months, fission will likely become more stable.

Or join the party as an early adopter :-)
Comment 8 Peeter Joot 2013-05-08 22:19:26 UTC
Note that as well as having the (very interesting) fission support, the gold linker is able to link the same large shared lib that the bfd linker bombs attempting (also allowing the debug symbols to be stripped out into a separate file even without the fission support using the --build-id or --add-gnu-debuglink mechanisms for external debug info).
Comment 9 H.J. Lu 2013-05-09 00:02:05 UTC
Do you have a testcase? Which version of ICC are you using?
Comment 10 Peeter Joot 2013-05-09 00:52:43 UTC
I'm using: 

icpc (ICC) 13.0.0 20130327

I could potentially bundle up all the required .a's and other auxillary libraries required to reproduce this... but it would be huge (my objcopy --only-keep-debug output alone is about 6Gb).
Comment 11 Alan Modra 2013-05-09 01:31:52 UTC
Peeter, the answer to your original question is (3).  This is a limitation of the debug format variant that was used when compiling crtn.o.  DWARF allows different encodings for addresses and offsets.  In your case you have a 32-bit offset being used in .debug_aranges to access something in .debug_info, which apparently is too large for a 32-bit offset.

Stripping the debug info out of crtn.o might work for you.
Comment 12 Peeter Joot 2013-05-09 02:42:10 UTC
Hi Alan,

Interesting, however, if that is the case, what's the reason that I can work around this by switching to the gold linker.
Comment 13 Alan Modra 2013-05-09 03:02:55 UTC
Perhaps gold is better at removing duplicate debug info, so the resulting .debug_info is smaller?  Or perhaps gold places debug sections in a different order, resulting in the relative offset from the .debug_aranges field to the .debug_info location being smaller?  Whatever the reason, I'd wager you will run into trouble even with gold if you make your shared library large enough.

Another possibility is that gold is simply not reporting an error on relocation overflow..
Comment 14 Alan Modra 2013-05-09 04:23:53 UTC
Yes, gold is "succeeding" simply because it doesn't report the overflow.  :-(

    case elfcpp::R_X86_64_32:
      // FIXME: we need to verify that value + addend fits into 32 bits:
Comment 15 Peeter Joot 2013-05-09 14:32:34 UTC
So, if this is a limitation, shall I close the bug report, or rename it, leaving it open to fix the gold FIXME pointed out above?
Comment 16 H.J. Lu 2013-05-09 15:23:51 UTC
(In reply to comment #10)
> I'm using: 
> 
> icpc (ICC) 13.0.0 20130327
> 

It could be an ICC bug which generates huge debug info.  Does
it happen with GCC?
Comment 17 Peeter Joot 2013-05-10 20:59:48 UTC
Looks like ICC is definitely triggering this (we are getting close to the limit with g++ too).

With g++:

$ objcopy --only-keep-debug libdb2e.so.1 libdb2e.so.1.debug
$ objcopy --strip-debug libdb2e.so.1 libdb2e.so.1.nodebug
$ ls -l libdb2e.so.1.debug libdb2e.so.1.nodebug

-rwxr-xr-x 1 peeterj pdxdb2 2336638415 2013-05-10 16:17 libdb2e.so.1.debug
-rwxr-xr-x 1 peeterj pdxdb2  390515219 2013-05-10 16:18 libdb2e.so.1.nodebug

With ICC:

-rw-r--r-- 1 peeterj pdxdb2 6308565713 2013-05-10 16:11 libdb2e.so.1.debug
-rw-r--r-- 1 peeterj pdxdb2  501988212 2013-05-10 16:17 libdb2e.so.1.nodebug

The intel shared lib is 1.28x bigger, but it is generating 2.7x times more debug code with -g (for a grand total of 5.9 Gb of -g info).
Comment 18 H.J. Lu 2013-05-10 21:13:25 UTC
(In reply to comment #17)
> Looks like ICC is definitely triggering this (we are getting close to the limit
> with g++ too).
> 
> The intel shared lib is 1.28x bigger, but it is generating 2.7x times more
> debug code with -g (for a grand total of 5.9 Gb of -g info).

Please report this to your Intel support representative.  They
may be able to fix ICC.
Comment 19 Peeter Joot 2013-05-10 21:19:37 UTC
Already done;)

As for this defect.  Shall we keep it to fix the gold linker so that it also fails with this relocation error instead of silently ignoring it?  (or close this, and open a separate report for that issue).
Comment 20 H.J. Lu 2013-05-10 21:26:22 UTC
(In reply to comment #19)
> 
> As for this defect.  Shall we keep it to fix the gold linker so that it also
> fails with this relocation error instead of silently ignoring it?  (or close
> this, and open a separate report for that issue).

Please open a separate gold bug.
Comment 21 Peeter Joot 2013-05-11 01:34:39 UTC
http://sourceware.org/bugzilla/show_bug.cgi?id=15460 is opened.
Comment 22 Alan Modra 2013-05-22 08:44:11 UTC
Can't be fixed without parsing all debug info and re-emitting.