This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: [PATCH 1/5] glibc: Perform rseq(2) registration at C startup and thread creation (v10)
- From: Mathieu Desnoyers <mathieu dot desnoyers at efficios dot com>
- To: Florian Weimer <fweimer at redhat dot com>
- Cc: carlos <carlos at redhat dot com>, Joseph Myers <joseph at codesourcery dot com>, Szabolcs Nagy <szabolcs dot nagy at arm dot com>, libc-alpha <libc-alpha at sourceware dot org>, Thomas Gleixner <tglx at linutronix dot de>, Ben Maurer <bmaurer at fb dot com>, Peter Zijlstra <peterz at infradead dot org>, "Paul E. McKenney" <paulmck at linux dot vnet dot ibm dot com>, Boqun Feng <boqun dot feng at gmail dot com>, Will Deacon <will dot deacon at arm dot com>, Dave Watson <davejwatson at fb dot com>, Paul Turner <pjt at google dot com>, Rich Felker <dalias at libc dot org>, linux-kernel <linux-kernel at vger dot kernel dot org>, linux-api <linux-api at vger dot kernel dot org>
- Date: Fri, 31 May 2019 10:48:29 -0400 (EDT)
- Subject: Re: [PATCH 1/5] glibc: Perform rseq(2) registration at C startup and thread creation (v10)
- Dkim-filter: OpenDKIM Filter v2.10.3 mail.efficios.com 2F25D22C738
- References: <20190503184219.19266-1-mathieu.desnoyers@efficios.com> <20190503184219.19266-2-mathieu.desnoyers@efficios.com> <87h89gjgaf.fsf@oldenburg2.str.redhat.com> <1239705947.14878.1558985272873.JavaMail.zimbra@efficios.com> <140718133.18261.1559144710554.JavaMail.zimbra@efficios.com> <2022553041.20966.1559249801435.JavaMail.zimbra@efficios.com> <875zprm4jo.fsf@oldenburg2.str.redhat.com>
----- On May 31, 2019, at 4:06 AM, Florian Weimer fweimer@redhat.com wrote:
> * Mathieu Desnoyers:
>
>> I found that it's because touching a __thread variable from
>> ld-linux-x86-64.so.2 ends up setting the DF_STATIC_TLS flag
>> for that .so, which is really not expected.
>>
>> Even if I tweak the assert to make it more lenient there,
>> touching the __thread variable ends up triggering a SIGFPE.
>
> Sorry, I got distracted at this critical juncture. Yes, I forgot that
> there isn't TLS support in the dynamic loader today.
>
>> So rather than touching the TLS from ld-linux-x86-64.so.2,
>> I've rather experimented with moving the rseq initialization
>> for both SHARED and !SHARED cases to a library constructor
>> within libc.so.
>>
>> Are you aware of any downside to this approach ?
>
> The information whether the kernel supports rseq would not be available
> to IFUNC resolvers. And in some cases, ELF constructors for application
> libraries could run before the libc.so.6 constructor, so applications
> would see a transition from lack of kernel support to kernel support.
>
>> +static
>> +__attribute__ ((constructor))
>> +void __rseq_libc_init (void)
>> +{
>> + rseq_init ();
>> + /* Register rseq ABI to the kernel. */
>> + (void) rseq_register_current_thread ();
>> +}
>
> I think the call to rseq_init (and the __rseq_handled variable) should
> still be part of the dynamic loader. Otherwise there could be confusion
> about whether glibc handles the registration (due the constructor
> ordering issue).
Let's break this down into the various sub-issues involved:
1) How early do we need to setup rseq ? Should it be setup before:
- LD_PRELOAD .so constructors ?
- Without circular dependency,
- With circular dependency,
- audit libraries initialization ?
- IFUNC resolvers ?
- other callbacks ?
- memory allocator calls ?
We may end up in a situation where we need memory allocation to be setup
in order to initialize TLS before rseq can be registered for the main
thread. I suspect we will end up needing a fallbacks which always work
for the few cases that would try to use rseq too early in dl/libc startup.
2) Do we need to setup __rseq_handled and __rseq_abi at the same stage of
startup, or is it OK to setup __rseq_handled before __rseq_abi ?
3) Which shared object owns __rseq_handled and __rseq_abi ?
- libc.so ?
- ld-linux-*.so.2 ?
- Should both symbols be owned by the same .so ?
- What about the !SHARED case ? I think this would end up in libc.a in all cases.
4) Inability to touch a TLS variable (__rseq_abi) from ld-linux-*.so.2
- Should we extend the dynamic linker to allow such TLS variable to be
accessed ? If so, how much effort is required ?
- Can we find an alternative way to initialize rseq early during
dl init stages while still performing the TLS access from a function
implemented within libc.so ?
So far, I got rseq to be initialized before LD_PRELOADed library constructors
by doing the initialization in a constructor within libc.so. I don't particularly
like this approach, because the constructor order is not guaranteed.
One possible solution would be to somehow expose a rseq initialization function
symbol from libc.so, look it up from ld-linux-*.so.2, and invoke it after libc.so
has been loaded. It would end up being similar to a constructor, but with a
fixed invocation order.
I'm just not sure we have everything we need to do this in ld-linux-*.so.2
init stages.
Thoughts ?
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com