Bug 32153

Summary: [2.43 regression]: ld: Failed to link llvm bitcode module: Expected at most one ThinLTO module per bitcode file
Product: binutils Reporter: Andrei Horodniceanu <a.horodniceanu>
Component: ldAssignee: Not yet assigned to anyone <unassigned>
Status: RESOLVED FIXED    
Severity: normal CC: hjl.tools, sam
Priority: P2    
Version: 2.43.1   
Target Milestone: 2.44   
See Also: https://sourceware.org/bugzilla/show_bug.cgi?id=32083
Host: Target:
Build: Last reconfirmed:

Description Andrei Horodniceanu 2024-09-08 17:23:26 UTC
Since commit a6f8fe0a9e9cbe871652e46ba7c22d5e9fb86208 (https://sourceware.org/bugzilla/show_bug.cgi?id=32083) the following started failing:
---------
$ cat a.c
int main () {}
$ clang -c -flto=thin a.c
$ /usr/bin/gcc-14 -Wl,-v a.o -o a -Wl,-plugin,/usr/lib/llvm/18/lib64/LLVMgold.so
collect2 version 14.2.1 20240905 040b979a08843806ea930a8d839dd94301323ebe
/usr/libexec/gcc/x86_64-pc-linux-gnu/ld -plugin /usr/libexec/gcc/x86_64-pc-linux-gnu/14/liblto_plugin.so -plugin-opt=/usr/libexec/gcc/x86_64-pc-linux-gnu/14/lto-wrapper -plugin-opt=-fresolution=/tmp/ccsKV00t.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie -z now -o a /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../lib64/Scrt1.o /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/14/crtbeginS.o -L/usr/lib/gcc/x86_64-pc-linux-gnu/14 -L/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../.. -v a.o -plugin /usr/lib/llvm/18/lib64/LLVMgold.so -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib/gcc/x86_64-pc-linux-gnu/14/crtendS.o /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../lib64/crtn.o
GNU ld (GNU Binutils) 2.43.50.20240815
/tmp/binutils-broken/ld/.libs/ld-new: error: Failed to link module a.o: Expected at most one ThinLTO module per bitcode file
collect2: error: ld returned 1 exit status
---------

The command has been extracted from the ldc (LLVM D compiler) link invocation:
--------
$ cat a.d
void main () {}
$ CC=gcc-14 ldc2 -flto=thin a.d -v
<snip>
/usr/bin/gcc-14 a.o /usr/lib/ldc2/1.39/lib64/ldc_rt.dso.o -o a -Wl,-plugin,/usr/lib/llvm/18/lib64/LLVMgold.so -Wl,-plugin-opt=thinlto -Wl,-plugin-opt=mcpu=x86-64 -Wl,-plugin-opt=O0 -Wl,-plugin-opt=-function-sections -Wl,-plugin-opt=-data-sections -L/usr/lib/ldc2/1.39/lib64 -L/usr/lib/clang/18/lib/linux -lphobos2-ldc-shared -ldruntime-ldc-shared -Wl,-rpath,/usr/lib/ldc2/1.39/lib64 -Wl,--gc-sections -lrt -ldl -lpthread -lm -m64
/tmp/binutils-broken/ld/.libs/ld-new: error: Failed to link module a.o: Expected at most one ThinLTO module per bitcode file
collect2: error: ld returned 1 exit status
Error: /usr/bin/gcc-14 failed with status: 1
--------

To reproduce you need a newish version of gcc, one that contains https://gcc.gnu.org/pipermail/gcc-patches/2023-May/617277.html. The specific version of gcc that I had:
--------
$ gcc-14 --version
gcc-14 (Gentoo Hardened 14.3.9999 p, commit d676863b64b58bf73a7f26eea70f81aeba517b13) 14.2.1 20240905 040b979a08843806ea930a8d839dd94301323ebe
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
--------

The commit that introduced the regression mentions an accompanying gcc fix: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=116361 but even with it applied the issue persists.

H.J., since you're the author of the commit, do you think that your fix is the issue or is there some other problem?
Comment 1 H.J. Lu 2024-09-08 23:43:06 UTC
This is very similar to

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=116361

LLVM could implement LDPT_REGISTER_CLAIM_FILE_HOOK_V2:

https://gcc.gnu.org/pipermail/gcc-patches/2023-May/617277.html

/* Callback used by a linker to check if the plugin can claim FILE.
   Writes the result in CAN_BE_CLAIMED.  If KNOWN_USED != 0, the object
   is known by the linker to be included in link output, or an older API
   version is in use that does not provide that information.  Otherwise,
   the linker is only determining whether this is a plugin object and
   only the symbol table is needed by the linker.  In this case, the
   object should not be included in link output and this function will
   be called by the linker again with KNOWN_USED != 0 after the linker
   decides the object should be included in link output. */

This change:

diff --git a/bfd/plugin.c b/bfd/plugin.c
index f6c6fdbee69..b481c35115f 100644
--- a/bfd/plugin.c
+++ b/bfd/plugin.c
@@ -597,7 +597,7 @@ static bfd_cleanup
 bfd_plugin_object_p (bfd *abfd)
 {
   if (ld_plugin_object_p)
-    return ld_plugin_object_p (abfd, false);
+    return ld_plugin_object_p (abfd, true);
 
   if (abfd->plugin_format == bfd_plugin_unknown && !load_plugin (abfd))
     return NULL;

works for this test.  I don't think that it will work for all cases.
Comment 2 Sourceware Commits 2024-09-10 02:16:40 UTC
The master branch has been updated by H.J. Lu <hjl@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=c839a44c39161c9932d58c28c2949ab3bb94ea6f

commit c839a44c39161c9932d58c28c2949ab3bb94ea6f
Author: H.J. Lu <hjl.tools@gmail.com>
Date:   Sun Sep 8 17:27:00 2024 -0700

    bfd: Pass true to ld_plugin_object_p
    
    Since linker calls bfd_plugin_object_p, which calls ld_plugin_object_p,
    only for command-line input objects, pass true to ld_plugin_object_p so
    that the same input IR file won't be included twice if the new LTO hook,
    LDPT_REGISTER_CLAIM_FILE_HOOK_V2 isn't used.
    
            PR ld/32153
            * plugin.c (bfd_plugin_object_p): Pass true to ld_plugin_object_p.
    
    Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Comment 3 Sourceware Commits 2024-09-18 20:23:55 UTC
The binutils-2_43-branch branch has been updated by H.J. Lu <hjl@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=f307db3d8b755e7f6f6efc3157aad592a6d9ed80

commit f307db3d8b755e7f6f6efc3157aad592a6d9ed80
Author: H.J. Lu <hjl.tools@gmail.com>
Date:   Sun Sep 8 17:27:00 2024 -0700

    bfd: Pass true to ld_plugin_object_p
    
    Since linker calls bfd_plugin_object_p, which calls ld_plugin_object_p,
    only for command-line input objects, pass true to ld_plugin_object_p so
    that the same input IR file won't be included twice if the new LTO hook,
    LDPT_REGISTER_CLAIM_FILE_HOOK_V2 isn't used.
    
            PR ld/32153
            * plugin.c (bfd_plugin_object_p): Pass true to ld_plugin_object_p.
    
    Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
    (cherry picked from commit c839a44c39161c9932d58c28c2949ab3bb94ea6f)
Comment 4 H.J. Lu 2024-09-18 20:25:34 UTC
Fixed for binutils 2.44 and 2.43 branch.