Summary: | Document how to use LTO | ||
---|---|---|---|
Product: | binutils | Reporter: | dilyan.palauzov <dilyan.palauzov> |
Component: | binutils | Assignee: | Not yet assigned to anyone <unassigned> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | markus, nickc |
Priority: | P2 | ||
Version: | unspecified | ||
Target Milestone: | --- | ||
Host: | Target: | ||
Build: | Last reconfirmed: | 2016-07-13 00:00:00 |
Description
dilyan.palauzov@aegee.org
2016-07-09 13:23:14 UTC
(In reply to dilyan.palauzov@aegee.org from comment #0) > Please document what the user has to do, in order to use LTO with ld, gold, > nm, ar, ranlib in regards to the -?-plugin option. In particular, > > * when is 'compiler -print-prog-name=...' called to find the plugin, > * if it is not called by nm, ar, ranlib, how to the latter find the linker > plugin > * how can different plugins coexist (for gcc and clang, and for different > versions of gcc) Yes. At least it works for gcc and clang. Different versions of gcc are compatible right now, so you can just use the latest plugin and it should work with older gcc versions, too. (This could change in the future.) > * how is the directory bfd-plugins supposed to be used Just add symlinks to the plugins, e.g.: markus@x4 ~ % ls -al /usr/x86_64-pc-linux-gnu/binutils-bin/lib/bfd-plugins/ total 8 drwxr-xr-x 1 root root 66 Jun 3 13:17 . drwxr-xr-x 1 root root 22 Jun 27 2015 .. lrwxrwxrwx 1 root root 65 Apr 27 13:32 liblto_plugin.so.0.0.0 -> /usr/libexec/gcc/x86_64-pc-linux-gnu/6.1.1/liblto_plugin.so.0.0.0 lrwxrwxrwx 1 root root 28 Jun 3 13:17 LLVMgold.so -> /usr/local/lib64/LLVMgold.so > * in which binutils versions have the aforementioned behaviours changed plugins coexistence was added in 2014. Forgot to mention that binutils automatically uses the first plugin that claims the object in question. The bfd-plugins directory is not documented. Doing "gcc -c t.c" "strace nm --plugin=liblto_plugin.so.0.0.0 t.o" does not show that the bfd-plugins directory is checked, nor does "strace ar csr --plugin liblto_plugin.so.0.0.o libt.a t.o" check it. I use binutils 2.26.51.20160709 with ./configure --prefix=/usr/local. I guess the bfd-plugins directory has to be somewhere under /usr/local (/usr/local/lib/bfd-plugins or /usr/local/x86_64-pc-linux-gnu/lib/bfd-plugins), but strace does not show anything under local, except execve("/usr/local/{ar,nm}") Why is here: open("/lib/x86_64-linux-gnu/tls/x86_64/liblto_plugin.so.0.0.o", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/lib/x86_64-linux-gnu/tls/liblto_plugin.so.0.0.o", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/lib/x86_64-linux-gnu/x86_64/liblto_plugin.so.0.0.o", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/lib/x86_64-linux-gnu/liblto_plugin.so.0.0.o", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib/x86_64-linux-gnu/tls/x86_64/liblto_plugin.so.0.0.o", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib/x86_64-linux-gnu/tls/liblto_plugin.so.0.0.o", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib/x86_64-linux-gnu/x86_64/liblto_plugin.so.0.0.o", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib/x86_64-linux-gnu/liblto_plugin.so.0.0.o", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) used "x86_64-linux-gnu" instead of "x86_64-pc-linux-gnu" There is no need to use an explicit --plugin option. On my system: % strace ar cr a.a a.o ... open("/usr/x86_64-pc-linux-gnu/binutils-bin/git/../git/../lib/bfd-plugins", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 6 fstat(6, {st_mode=S_IFDIR|0755, st_size=66, ...}) = 0 getdents(6, /* 4 entries */, 32768) = 128 stat("/usr/x86_64-pc-linux-gnu/binutils-bin/git/../git/../lib/bfd-plugins/.", {st_mode=S_IFDIR|0755, st_size=66, ...}) = 0 stat("/usr/x86_64-pc-linux-gnu/binutils-bin/git/../git/../lib/bfd-plugins/..", {st_mode=S_IFDIR|0755, st_size=22, ...}) = 0 stat("/usr/x86_64-pc-linux-gnu/binutils-bin/git/../git/../lib/bfd-plugins/liblto_plugin.so.0.0.0", {st_mode=S_IFREG|0755, st_size=83976, ...}) = 0 open("/usr/x86_64-pc-linux-gnu/binutils-bin/git/../git/../lib/bfd-plugins/liblto_plugin.so.0.0.0", O_RDONLY|O_CLOEXEC) = 7 And plaese configure binutils with: --enable-plugins I had --enable-plugins. "strace ar csr ... &|grep bfd" prints openat(AT_FDCWD, "../bin/../lib/bfd-plugins", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = -1 ENOENT (No such file or directory) hence if it works, depends on the current directory. In bfd/plugin.c:load_plugin() I have on line 337 BINDIR=/usr/local/bin then plugin_dir is evaluated to /usr/local/bin/../lib/bfd-plugins and p (result of make_relative_prefix()) is ${HOME}/binutils/binutils/../bin/../lib/bfd-plugins. Afterwards plugin_dir is discarded and the plugins are searched in the relative dir, stored in p. How are the plugins supposed to be always found in the relative directories? The part with bfd/plugin.c was a false alarm. The problem was, that I had the change below, which was supposed to fix these warnings, when I compile binutils with -flto (cf. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70611): /git/binutils-gdb/libiberty/make-relative-prefix.c: In function 'make_relative_prefix_1.constprop': /git/binutils-gdb/libiberty/make-relative-prefix.c:228:1: error: stack usage might be unbounded [-Werror=stack-usage=] make_relative_prefix_1 (const char *progname, const char *bin_prefix, ^ lto1: all warnings being treated as errors Nevertheless, there is no documentation to state, that the linker plugin shall be installed in bfd-plugins/. index fe639d1..dba3429 100644 --- a/libiberty/make-relative-prefix.c +++ b/libiberty/make-relative-prefix.c @@ -256,7 +256,7 @@ make_relative_prefix_1 (const char *progname, const char *bin_prefix, #ifdef HAVE_HOST_EXECUTABLE_SUFFIX len += strlen (HOST_EXECUTABLE_SUFFIX); #endif - nstore = (char *) alloca (len); + nstore = (char *) malloc (len); startp = endp = temp; while (1) @@ -304,6 +304,7 @@ make_relative_prefix_1 (const char *progname, const char *bin_prefix, else endp++; } + free(nstore); } } Hi Dilyan, Given the information that you have accumulated in this PR, perhaps you would like to have a go at writing some documentation ? I am happy to take any text that you provide and add it to the linker and binutils manuals. Cheers Nick When the bfd-plugins directory looks like: me@home:/usr/local/lib/bfd-plugins# ls -l total 4 lrwxrwxrwx 1 root staff 14 Jul 21 15:11 LLVMgold.so -> ../LLVMgold.so lrwxrwxrwx 1 root staff 71 Jul 10 17:56 liblto_plugin.so.0.0.0 -> /usr/local/libexec/gcc/x86_64-pc-linux-gnu/6.1.1/liblto_plugin.so.0.0.0* How does "nm t.o" decide, if it shall open liblto_plugin or LLVMgold to proceed the .o file? Why does ld proceed in a different way? (In reply to dilyan.palauzov@aegee.org from comment #9) > When the bfd-plugins directory looks like: > > me@home:/usr/local/lib/bfd-plugins# ls -l > total 4 > lrwxrwxrwx 1 root staff 14 Jul 21 15:11 LLVMgold.so -> ../LLVMgold.so > lrwxrwxrwx 1 root staff 71 Jul 10 17:56 liblto_plugin.so.0.0.0 -> > /usr/local/libexec/gcc/x86_64-pc-linux-gnu/6.1.1/liblto_plugin.so.0.0.0* > > > How does "nm t.o" decide, if it shall open liblto_plugin or LLVMgold to > proceed the .o file? It tries them in alphabetic order. The first plugin that claims the object in question gets used. diff --git a/binutils/doc/binutils.texi b/binutils/doc/binutils.texi index 0a2c4c6ab4..d7b3657aa0 100644 --- a/binutils/doc/binutils.texi +++ b/binutils/doc/binutils.texi @@ -534,9 +534,17 @@ default for @sc{gnu} @command{ar}. @command{ar} does not support any of the oth which is the default for AIX @command{ar}. The optional command line switch @option{--plugin} @var{name} causes -@command{ar} to load the plugin called @var{name} which adds support -for more file formats. This option is only available if the toolchain -has been built with plugin support enabled. +@command{ar} to load the plugin called @var{name} which adds support for more +file formats, including object files with link-time optimization information. +This option is only available if the toolchain has been built with plugin +support enabled. If @option{--plugin} is not provided, @command{ar} iterates +over the files in $@{libdir@}/bfd-plugins in alphabetic order and the first +plugin that claims the object in question is used. Please note, that the +implicit search path is not identical to the one used by @command{ld} +@option{-plugin}. To make @command{ar} consider implicitly the linker plugin, +copy it to $@{libdir@}/bfd-plugins. For GCC the plugin is +@file{liblto_plugin.so.0.0.0}, for Clang - @file{LLVMgold.so}. The GCC plugin +is backwards compatible, so it is sufficient to have just the newest one. The optional command line switch @option{--target} @var{bfdname} specifies that the archive members are in an object code format @@ -1009,7 +1017,14 @@ Display only defined symbols for each object file. @cindex load plugin Load the plugin called @var{name} to add support for extra target types. This option is only available if the toolchain has been built -with plugin support enabled. +with plugin support enabled. If @option{--plugin} is not provided, +@command{nm} iterates over the files in $@{libdir@}/bfd-plugins in alphabetical +order and the first plugin that claims the object is used. Please note, that +the plugin path is not identical to the one used by @command{ld} +@option{-plugin}. To make @command{nm} consider implicitly the linker plugin, +copy it to $@{libdir@}/bfd-plugins. For GCC the plugin is @file{liblto_plugin.so.0.0.0}, for Clang - @file{LLVMgold.so}. The +GCC plugin is backwards compatible, so it is sufficient to have just the newest +one. @item --size-sort Sort symbols by size. For ELF objects symbol sizes are read from the diff --git a/ld/ld.texinfo b/ld/ld.texinfo index 29c2131a2b..ce2137c209 100644 --- a/ld/ld.texinfo +++ b/ld/ld.texinfo @@ -828,6 +828,16 @@ the linker may make more use of this option. Also currently there is no difference in the linker's behaviour for different non-zero values of this option. Again this may change with future releases. +@kindex -plugin @var{linker-plugin} +@item -plugin @var{linker-plugin} +Involve a plugin in the linking process. This parameter is automatically added +by the complier, when using link time optimization, and contains the absolute +file name the the plugin, where the installation process has put it. Note, that +the path is different from the place, where +@command{ar}/@command{nm}/@command{ranlib} search implicitly for the linker +plugin. All gcc linker plugins are backward compatible, it is sufficient to +have just the newest one. + @kindex --push-state @cindex push state governing input file handling @item --push-state The master branch has been updated by Nick Clifton <nickc@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=387dd77738619d7e898f063bbeb1b8b6faf6cad5 commit 387dd77738619d7e898f063bbeb1b8b6faf6cad5 Author: Dilyan Palauzov <dilyan.palauzov@aegee.org> Date: Fri Jan 27 13:20:24 2017 +0000 Update description of the -plugin option used by the linker, ar and nm. PR 20343 ld * ld.texinfo (Options): Extend documentation of the --plugin option. Include a description of where the plugins should be located. binutils* doc/binutils.texi (ar): Extend documentation of the --plugin option. Include a description of where the plugins should be located. (nm): Likewise. Hi Dilyan, Thanks very much for taking the time to write the documentation patch, and I apologise for taking my time to review it. The patch seems reasonable to me and so I have checked it in, although I hope that you wont mind that I have made a few small changes to your deathless prose... Cheers Nick |