Attempting to build a glibc POWER10 multilib, fails with the following error /glibc_prestine/new_build/linkobj/libc_pic.a(acct.os): in function `acct': (.text+0x64): call to `__syscall_error' lacks nop, can't restore toc; (toc save/adjust stub) /usr/bin/ld: final link failed: bad value collect2: error: ld returned 1 exit status make[1]: *** [Makerules:656: /glibc_prestine/new_build/linkobj/libc.so] Error 1 make[1]: Leaving directory '/glibc_prestine/glibc' make: *** [Makefile:9: all] Error 2 How to reproduce : $ ./configure 'CFLAGS=-O2 -mcpu=power9' --prefix=/usr --with-headers=/usr/include --build=ppc64le-linux --with-cpu=power10 $ make
Disassembly of section .text: 0000000000000000 <__GI_acct>: # if SYSCALL_ULONG_ARG_1 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS, SYSCALL_ULONG_ARG_1, SYSCALL_ULONG_ARG_2) # else T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) 0: 00 00 4c 3c addis r2,r12,0 0: R_PPC64_REL16_HA .TOC. 4: 00 00 42 38 addi r2,r2,0 4: R_PPC64_REL16_LO .TOC.+0x4 8: 33 00 00 38 li r0,51 c: d1 ff 21 f8 stdu r1,-48(r1) 10: 20 00 e1 fb std r31,32(r1) 14: 98 8f ed eb ld r31,-28776(r13) 18: 10 00 ff 77 andis. r31,r31,16 1c: 1c 00 82 41 beq 38 <__GI_acct+0x38> 20: a6 02 28 7d mflr r9 24: 40 00 21 f9 std r9,64(r1) 28: 01 00 00 44 scv 0 2c: 40 00 21 e9 ld r9,64(r1) 30: a6 03 28 7d mtlr r9 34: 08 00 00 48 b 3c <__GI_acct+0x3c> 38: 02 00 00 44 sc # endif ret 3c: 00 00 bf 2e cmpdi cr5,r31,0 40: 20 00 e1 eb ld r31,32(r1) 44: 30 00 21 38 addi r1,r1,48 48: 18 00 96 41 beq cr5,60 <__GI_acct+0x60> 4c: 01 f0 20 39 li r9,-4095 50: 40 48 23 7c cmpld r3,r9 54: 20 00 e0 4d bltlr+ 58: d0 00 63 7c neg r3,r3 5c: 08 00 00 48 b 64 <__GI_acct+0x64> 60: 20 00 e3 4c bnslr+ 64: 00 00 00 48 b 64 <__GI_acct+0x64> 64: R_PPC64_REL24 __syscall_error <-- this is causing the issue ... 70: 40 20 0c 00 .long 0xc2040 74: 68 00 00 00 .long 0x68 78: 04 00 61 63 ori r1,r27,4 7c: 63 74 00 00 .long 0x7463
If glibc is configured without 'CFLAGS=-O2 -mcpu=power9', i.e. $ ./configure --prefix=/usr --with-headers=/usr/include --build=ppc64le-linux --with-cpu=power10 $ make This successfully builds glibc. Comparing the pass case with the fail case of acct.os we get 5c: 08 00 00 48 b 64 <__GI_acct+0x64> 60: 20 00 e3 4c bnslr+ 64: 00 00 00 48 b 64 <__GI_acct+0x64> - 64: R_PPC64_REL24_NOTOC __syscall_error + 64: R_PPC64_REL24 __syscall_error ... 70: 40 20 0c 00 .long 0xc2040 74: 68 00 00 00 .long 0x68 The build goes through because R_PPC64_REL24_NOTOC is generated.
The above code comes from sysdeps/powerpc/powerpc64/sysdep.h: b JUMPTARGET (NOTOC (__syscall_error)) The NOTOC macro defined in the same file is defined as: #ifdef USE_PPC64_NOTOC # define NOTOC(l) l@notoc #else # define NOTOC(l) l #endif ...and USE_PPC64_NOTOC is a configure time macro from sysdeps/powerpc/powerpc64/configure.ac: AC_CACHE_CHECK([if the compiler supports @notoc], libc_cv_ppc64_notoc, [dnl cat > conftest.c <<EOF int bar (void); int foo (void) { return bar () + 1; } EOF libc_cv_ppc64_notoc=no if AC_TRY_COMMAND([${CC-cc} $libc_cv_cc_submachine $CFLAGS $CPPFLAGS -S -o conftest.s conftest.c]) \ && AC_TRY_COMMAND([grep -q -E 'bar@notoc' conftest.s]) then libc_cv_ppc64_notoc=yes fi rm -rf conftest.*]) The "bug" is that $libc_cv_cc_submachine (ie, -mcpu=power10 from the --with-cpu=power10 configure option) is overridden by the -mcpu=power9 from the CLFAGS. The following GLIBC patch fixes the error for me: @@ -35,7 +35,7 @@ int bar (void); int foo (void) { return bar () + 1; } EOF libc_cv_ppc64_notoc=no - if AC_TRY_COMMAND([${CC-cc} $libc_cv_cc_submachine $CFLAGS $CPPFLAGS -S -o conftest.s conftest.c]) \ + if AC_TRY_COMMAND([${CC-cc} $CFLAGS $CPPFLAGS $libc_cv_cc_submachine -S -o conftest.s conftest.c]) \ && AC_TRY_COMMAND([grep -q -E 'bar@notoc' conftest.s]) then libc_cv_ppc64_notoc=yes
Alan, I CC'd you not for your help with the GLIBC fix, but to ask why does the linker even give an error here? This is a tail-call (b __syscall_error, not bl __syscall_error), so the callee will never return to the instruction after the direct branch, so we shouldn't need to place a toc-restore nop here, correct?
As the linker error says, this is a call that is going via a toc save or toc adjust stub. ie. the stub will be modifying r2. Since this is a tail call there is no way to restore the callers r2. I believe the linker error is correct. As far as I'm aware there is only one case where the linker error is wrong, and that is when calling a function that never returns.
I believe this is INVALID. Passing -mcpu=power9 in CFLAGS and configuring using --with-cpu=power10 looks like a user error to me.