global pointer gets overwritten with dlopen(3) on RISC-V

[Palmer Dabbelt]

On Fri, 12 May 2023 14:09:08 PDT (-0700), maskray@google.com wrote:
> On 2023-05-12, Palmer Dabbelt wrote:
>>On Fri, 12 May 2023 13:11:43 PDT (-0700), fweimer@redhat.com wrote:
>>>* Fangrui Song:
>>>
>>>>>1. Make -mno-relax the default for ld(1) (on Linux?). We have no
>>>>>benchmarks whatsoever, but global variables aren't very popular in
>>>>>application code these days and the gp register allows access to a
>>>>>single memory page (4kB) only. No big deal really.
>>>>
>>>>I do agree that --no-relax-gp is a sensible default choice for GNU ld.
>>>>https://maskray.me/blog/2021-03-14-the-dark-side-of-riscv-linker-relaxation#global-pointer-relaxation
>>>>
>>>>Perhaps you can start a separate topic on binutils? :)
>>>>
>>>>According to a doc from SiFive about -static -mcpu=sifive-u74 builds,
>>>>https://docs.google.com/spreadsheets/d/14V7cPbyc80AcGHzsMaw9hYb232dzRbGCmTApnxj-SpU/edit#gid=0
>>>>global pointer relaxation saves at best 0.5% size (I guess that refers
>>>>to .text. If we count all allocable sections, the percentage is likely
>>>>even smaller.)
>>>
>>>For a mature toolchain, 0.5% in code size reduction would be *a lot*,
>>>so I wouldn't dismiss that.
>>
>>That's broadly speaking why it sticks around.  We've got a bunch of
>>headaches related to relaxation, GP or otherwise, but they improve
>>performance and nobody's figured out how to replace that yet.
>>
>>>Do we have a reproducer?  Is the issue actually about gp relaxation for
>>>the main executable?
>>
>>In general we don't reference GP from shared libraries as we don't
>>have a GP save/restore scheme.  There may be a bug floating around
>>here somewhere, in which case we should fix it, but the original post
>>sounds like it wasn't a supported use case.
>>
>>>Thanks,
>>>Florian
>
> Global pointer relaxation only applies to +-2KiB data relative to __global_pointer$ (= .sdata + 0x800).
> The area that potentially benefits global pointer relaxation is very small.
>
> 0.5% code size reduction (relative to .text?) is the best case. I
> suspect the program somehow has a lot global variable accesses and
> placing these variables in .sdata helps.
>
> I've got results from Yingwei Zheng at PLCT lab using many
> configurations. The saving is like 0.1%.
> https://docs.google.com/spreadsheets/d/1Gz0h-C4U0toa9qELFtRaEWT_CzauE5JD9xMsLR8RyK8/edit#gid=1721258109
>
> On the binutils side, we occasionally see patches to fix global pointer
> relaxation bugs, e.g. the patch just sent a few hours ago:
> https://sourceware.org/pipermail/binutils/2023-May/127413.html
>
> I do not know the embedded toolchain well, but for Linux desktop/server,
> disabling global pointer relaxation seems like a sensible choice. If we
> discover a better way to utilize GP (x3) in the future, disabling global
> pointer relaxation today will result in fewer compatibility issues.

This comes up all the time, you're just pushing for a backdoor ABI 
break.  I get the desire to remove GP, if we were to be able to redo 
things I'd also not have it, but it's in the ABI and we can't change 
the binaries that exist.

If you want a GP-free ABI then you should just go write one up.  Then 
it'll become a distro problem, and if it turns out that users also don't 
want in then the GP ABI will rot and we can eventually deprecate it.

> Haiku
> (https://github.com/riscv-non-isa/riscv-elf-psabi-doc/issues/298#issuecomment-1344724796), Android, and Fuchsia
> have mentioned that they don't use global pointer relaxation.