[PATCH v2 2/3] nptl: Add libc allocated shadow stack for new threads

[Adhemerval Zanella Netto]

On 16/09/25 11:20, Yury Khrustalev wrote:
> * Adhemerval
> 
> On Mon, Sep 08, 2025 at 08:18:07AM -0300, Adhemerval Zanella Netto wrote:
>>
>>> It is possible to switch between shadow stacks at runtime with no kernel
>>> intervention, if a userspace is doing that then there is no guarantee
>>> that the shadow stack in use at the time the thread exits is the same
>>> one that the thread was started with.  Potentially a thread will be
>>> started with a stack that was previously in use in another context.  You
>>> either end up with userspace having to deallocate stacks when they're
>>> finished with or having the kernel always allocate a stack and userspace
>>> pivot off that if it wants something else.
>>
>> But this is not the usual programming mode for pthread code, and in places
>> glibc does it (makecontext) it leaks the allocate shadow stack anyway.
> 
> This will need to be fixed at some point (we couldn't do it at the time
> because there isn't a good internal memory allocator for smaller chunks
> of data).
> 
>> I can't not say much what other runtime might do wrt shadow stack management,
>> or if users will mess with stack pointer along with pthread; but in either
>> cases it seems that the shadow stack management is out of scope of the C
>> runtime.
>>
>> So maybe if clone3 just allows to setup the expected shadow stack size instead
>> of requiring C runtime to manage the shadow stack itself would be a 
>> more straightforward interface to use along pthread.
> 
> The original shadow stack ABI was developed before clone3 was available
> and there was no way to communicate stack size (normal or shadow alike)
> to the kernel, that is why there is map_shadow_stack() system call.
> 
> The AArch64 implementation largely followed x86 decisions so API would
> be similar too [1]. So, in some sense providing the shadow stack token
> (rather than it's size) as part of clone3 ABI has to be this way for
> historical reasons.
> 
> Supporting shadow stack allocation on the userspace side allows for more
> flexibility and doesn't confine any runtime to the choices made by the
> kernel. For example, we would not be able to benefit from re-use of the
> shadow stack at all (akin reusing normal stacks for new threads) if
> allocation and deallocation would be done by the kernel.
> 
> Since we deal with shadow stack allocation in Glibc any way (in
> makecontext()), we could do the same in the pthread implementation,
> although I agree that the use case is a bit different.
> 
> [1]: https://lore.kernel.org/linux-arm-kernel/20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org/

But what is the point in adding composability (map_shadow_stack) on clone3, when the 
expected usage does not allow the reuse of the previously allocated shadow stack and 
forces the userland to manually manage and always deallocate the shadow stack when 
the thread finishes?

The runtime flexibility could make sense in a runtime where green-threads are 
first-order primitives, where it exchanges the stack of the underlying OS threads 
depending on the workload. However, for the C runtime (and I believe for many other 
runtimes as well), and more specifically, pthread, it would be much simpler if 
userland could simply specify the shadow stack size. Even for C++ coroutines or other 
runtimes built on top of C, I believe it will require some assistance from the C
runtime to properly manage the shadow stack.

So, imho, the current clone3 interface for shadow stack token only adds an extra burden 
to the C runtime, and since it is not yet in master, perhaps we can still refine it.