Bringing rseq back into glibc

[Noah Goldstein]

On Thu, Nov 18, 2021 at 4:17 AM Florian Weimer via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> I would like to bring back rseq for glibc 2.35.  I propose the following
> steps:
>
> 1. Enable rseq registration in glibc, for internal use only.  This time,
>    put the rseq area into struct pthread, not into a initial-exec TLS
>    symbol.  (This helps to avoid with initial-exec TLS bloat with dlopen
>    and simplifies initialization somewhat.)

Isn't THREAD_SELF also implemented in TLS? Or am I missing
something?

>
> 2. Add a tunable to disable rseq registration in glibc.  This way, if
>    there is already an rseq user, it can be made to work again by
>    setting the tunable.
>
> 3. Implement sched_getcpu on top of rseq.
>
> 4. Add public symbols __rseq_abi_offset, __rseq_abi_size (currently 32
>    or 0), __rseq_abi_flags (currently 0).  __rseq_abi_offset is the
>    offset to add to the thread pointer (see __builtin_thread_pointer) to
>    get to the rseq area.  They will be public ABI symbols.  These
>    variables are initialized before user code runs, and changing the
>    results in undefined behavior.
>
> Under this model, the rseq area offset is clearly constant across all
> threads.  (This was previously implied by using initial-exec TLS
> memory.)  rseq registration failure is indicated by __rseq_abi_size ==
> 0.  If the size is non-zero, rseq will be registered on all threads
> created by glibc, and all the time as far as user code is concernes.
> (This assumes that if rseq registration succeeds on the main thread, it
> will succeed on all other threads.  We will terminate the process if
> not.)  For example, if a JIT compiler sees __rseq_abi_size >= 32, in
> generated code, it can inline a version of sched_getcpu that
> materializes the thread pointer and loads the cpu_id field from the rseq
> area, without further checks.  Under the old TLS-based model, it was
> less clear that this was a valid optimization.
>
> Furthermore, I believe this approach will be more compatible with
> potential future kernel changes in this area.  If the kernel tells us
> some day through the auxiliary vector that we should register a 128-byte
> rseq area with 64-byte alignment, we can make that happen and change
> __rseq_abi_offset and __rseq_abi_size accordingly.
>
> Steps 1 to 3 are backportable to previous glibc version, especially to
> 2.34 with its integrated libpthread.
>
> Comments?  As I said, I'd like to bring these changes into glibc 2.35,
> hopefully in early December.
>
> Thanks,
> Florian
>