Proposal for simpler libc rseq registration ABI across libraries

[Carlos O'Donell]

On 3/22/19 4:27 PM, Mathieu Desnoyers wrote:
> Hi,
> 
> Here is a email thread dedicated to discuss a simpler ABI proposal for
> interaction between libc and early adopter libraries/applications with
> respect to registration of restartable sequences (rseq).
> 
> This is based on my current proposal
> "glibc: Perform rseq(2) registration at C startup and thread creation (v7)"
> https://lore.kernel.org/patchwork/patch/1041324/
> 
> We can move the __rseq_lib_abi TLS refcount and register_state to an external
> library (librseq).
> 
> glibc would expose a new global "int" variable symbol __rseq_handled acting as
> a boolean. It would be initially 0. glibc would set it to 1 in its C startup
> code when it effectively handles rseq registration. That symbol would _not_ be
> a TLS (it's global).
> 
> librseq would be a new library used by rseq early adopters. It would expose a
> rseq register/unregister API, which internally would:
> 
> - Check whether __rseq_handled is true. If so, it would do nothing, leaving
>    rseq registration to the libc.
> - If __rseq_handled is false, deal with many early adopters with TLS refcount
>    and register_state variables internal to librseq.so.
> 
> That should take care of minimizing those metrics:
> - libc ABI complexity and maintenance burden in the long term,
> - pain for rseq early adopters when upgrading to newer libc which supports rseq
>    registration,
> 
> Does that make sense ?

Yes. Is the presence of __rseq_handled enough (depends on how you handle linkage)?

I think it's a reasonable design to use a symbol as feature detection for a feature
that exposes no API.

The alternative would be to implement an rseq wrapper and use the presence of that
wrapper to determine if libc was doing registration. So we use a known and possibly
required wrapper to implement the feature without needing another symbol.

-- 
Cheers,
Carlos.