lld status with powerpc64

Fangrui Song maskray@google.com
Mon Nov 8 22:38:18 GMT 2021


I agree with Adhemerval's point that "The main problem is this imposes
an extra burden for the linker" is unfortunate. (Magic behavior of
--power10-stubs=auto).

On 2021-11-08, Nemanja Ivanovic wrote:
>Just to chime in here...
> 
>Adhemerval,
>just a couple of notes about what you are referring to as an "optimization".
>The reason that LLD emits P10 PC-Relative prefixed instructions in stubs is
>because of an implicit assumption that the *_NOTOC relocation is restricted for
>P10 PC-Relative builds as it was never emitted by any compiler prior. However,
>I don't think it is accurate to refer to no P10 stubs with *_NOTOC as an
>"optimization". The two linkers have slightly different logic for deciding when
>it is OK to use P10 stubs. GLIBC is developed using the GNU toolchain so
>naturally relies on behaviour of GNU tools. In this case, the GNU linkers do
>not emit P10 stubs so GLIBC decided to use the @notoc. However, in a slightly
>different situation, the GNU linkers would generate P10 stubs as well. I
>imagine if GLIBC development added code that causes that to happen, that code
>would never be upstreamed as it causes broken builds.
> 
>Ultimately, what is needed is a clear indication in each relocation that using
>P10 stubs is OK for that call. Relying on presence of P10 relocations elsewhere
>in the compilation unit is problematic for at least two reasons (less efficient
>for the linker to look over all relocations twice, can lead to incorrect
>choices). This is why the proposal is to have different relocations for P10 and
>pre-P10 code. My preference would be to make this clear in the textual
>relocations as well (rather than the assembler emitting two different
>relocations for @notoc depending on -mpower10 vs -m<something_older>).

Thanks for investigating the long-term solution.

Nowadays among popular architectures, all of AArch64/RISC-V/x86 have
open places discussing ABI and toolchain implementations. It'd be nice
if ppc64 had a similar place so that we could detect problems earlier.
If more eyes had stared at the @notoc usage, we probably could have
avoided some friction in glibc and ld.lld...

---

> Current course and speed is that @notoc will imply pcrel stubs on P10
> and later, but will not do so on P9 and earlier.  The relocation
> currently associated with @notoc actually came with ELFv2 on P8 and,
> although no compilers ever generated @notoc, assembly routines using it
> prior to P10 are a valid case and should not be punished.  This can be
> handled by generating a different reloc for @notoc in the two cases.

In the context of glibc's @notoc assembly usage, I don't see why
suppresing @notoc for pre-POWER10 builds  would be a punishment:)

The branch targets are non-preemptible symbols.
In the absence of a long branch, @notoc to toc code needs a thunk (in
both GNU ld and ld.lld) while toc to toc doesn't.

In the case of a long branch, @notoc with PC-relative instructions is
slightly more efficient because it uses PC-relative instead of TOC's loading an address from .branch_lt
but the very minor improvement doesn't seem to justify the complexity :)

I say "minor" because I have checked the glibc build on a POWER9 machine
(GCC 8.3.0, binutils 2.31.1) and an x86-64 using cross compilatioin (GCC
10.2.1, binutils 2.37).
On the POWER9 machine, I don't find any thunk for __syscall_error and a
few other symbols using the glibc NOTOC macro.
On x86-64, I saw a long branch call site but I am not sure it can ever
be hot :)

For
https://sourceware.org/pipermail/libc-alpha/2021-November/132767.html ,
I'd like to support it. It will address the immediate problem linking
glibc with LLD 13.0.0 since glibc 2.35 is not too faraway (scheduled on
2022-02-01).

But the patch seems to complicate things.
I think checking whether with the supplied CFLAGS, CC generates @notoc
is better than checking assembler support.

>Nemanja Ivanovic
>LLVM PPC Backend Development
>IBM Toronto Lab
>Email: nemanjai@ca.ibm.com
>Phone: 905-413-3388
> 
>
>    ----- Original message -----
>    From: "Bill Schmidt" <wschmidt@linux.ibm.com>
>    To: "Adhemerval Zanella" <adhemerval.zanella@linaro.org>, "Fangrui Song"
>    <maskray@google.com>, "Tulio Magno Quites Machado Filho"
>    <tuliom@linux.ibm.com>, "Alan Modra" <amodra@gmail.com>, "Nemanja Ivanovic"
>    <nemanjai@ca.ibm.com>
>    Cc: libc-alpha@sourceware.org
>    Subject: Re: lld status with powerpc64
>    Date: Mon, Nov 8, 2021 9:12 AM
>     
>    On 11/8/21 8:11 AM, Adhemerval Zanella wrote:
>    >
>    > On 08/11/2021 10:59, Bill Schmidt wrote:
>    >> On 11/8/21 7:54 AM, Adhemerval Zanella wrote:
>    >>> On 08/11/2021 10:26, Bill Schmidt wrote:
>    >>>> On 11/8/21 5:37 AM, Adhemerval Zanella wrote:
>    >>>>> On 07/11/2021 11:24, Bill Schmidt wrote:
>    >>>>>> Please coordinate with Alan Modra and Nemanja Ivanovic on this
>    topic.  There are ongoing discussions about bfd and lld linker support
>    around @notoc that will be resolved soon, and Tulio is on vacation, so I
>    don't want the community to make steps they'll have to undo later, or for
>    people to engage in duplicate work.
>    >>>>> For this specific issue I just sent a patch to fix it on glibc side
>    [1].
>    >>>>> However I think it would be good if lld also implements the ld.bfd
>    >>>>> optimization to fallback to older stub generation if no pcrel
>    relocation
>    >>>>> is found (although it is debatable that @notoc should implicit
>    generate
>    >>>>> older ISA code depending of the resulting objects being linked
>    against).
>    >>>>>
>    >>>>> [1] https://patchwork.sourceware.org/project/glibc/patch/
>    20211108113316.8867-1-adhemerval.zanella@linaro.org/
>    >>>> Hi Adhemerval,
>    >>>>
>    >>>> Current course and speed is that @notoc will imply pcrel stubs on P10
>    and later,
>    >>>> but will not do so on P9 and earlier.  The relocation currently
>    associated with
>    >>>> @notoc actually came with ELFv2 on P8 and, although no compilers ever
>    generated
>    >>>> @notoc, assembly routines using it prior to P10 are a valid case and
>    should not be
>    >>>> punished.  This can be handled by generating a different reloc for
>    @notoc in the
>    >>>> two cases.
>    >>>>
>    >>>> If this solution holds up, then changes to glibc should be
>    unnecessary.
>    >>> The main problem is this imposes an extra burden for the linker, where
>    it
>    >>> need to implement the ld.bfd optimization to not generate the power10
>    stubs
>    >>> if no pcrel is found.  And it seems that lld does not yet support this
>    >>> yes and I guess it has not been an issue because @notoc in assembly
>    >>> routines should be rare.
>    >>>
>    >>> It also means that stubs generation are subject to a combination of
>    >>> relocation on different objects (@notoc on assembly does not necessary
>    >>> generate power10 stub with default linker option).  I think it should
>    >>> be ok, although I see this as really confusing since it took some time
>    >>> to figure out what ld.lfd was doing.
>    >> My only point is that the linker teams from both groups have been
>    huddling about
>    >> this, so I wanted you to be aware the issue goes a little deeper than it
>    appears
>    >> on the surface.  How glibc deals with this is up to glibc, of course. :)
>    >>
>    > Well, my patch had to disable notoc exactly because lld version does not
>    support
>    > such optimization (so default builds always use power10 instruction).  It
>    means
>    > that lld powerpc64le support is either incomplete or @notoc handling need
>    to be
>    > clarified.
>    Precisely.
>
> 
>


More information about the Libc-alpha mailing list