[PATCH v2 2/3] nptl: Add libc allocated shadow stack for new threads

[Mark Brown]

On Thu, Sep 18, 2025 at 10:02:49AM -0300, Adhemerval Zanella Netto wrote:
> On 17/09/25 19:13, Mark Brown wrote:

> > Remember that we have kernel allocated shadow stacks if one isn't
> > explictly provided, the size of which is a number we currently pull out
> > of thin air as far as userspace is concerned so we pick an extremely
> > high value.  We can add a prctl() or something to configure that number.

> If the idea is to either provide a pthread API to specify the shadow stack 
> size or add a way to automatically set it at runtime, it is still somewhat a 
> clunky api. It would require the thread to issue the prctl to change the 
> desired shadow stack size ratio and reset it after the clone call. It is
> doable nevertheless.

Yeah, I was thinking of that more for the fire and forget use case where
the size doesn't change much so you're not constantly updating it.
Though for the spawning a new thread case we could do something that
only affects the next clone() I suppose (we have a similar thing for the
vector extensions where you can set the VL for the next exec).

> >>> I also previously suggested that if we want to enable reuse of stacks
> >>> after a thread has exited we should support having the kernel write a
> >>> token to the current pointer of exiting threads (mirroring how the stack
> >>> pivot instructions work), I didn't get any feedback on that suggestion.
> >>> I would expect this to be gated behind a separate ABI (most likely a new
> >>> permission on PR_SHADOW_STACK_SET_STATUS) so readily discoverable if
> >>> added independently.

> >> Sorry if I missed it, if the kernel can provide it I think it would
> >> be useful.

> > OK, great.  I think that if we've got that then you'd not have any
> > concerns, you'd be able to reuse the stacks from threads that exit
> > cleanly?  For those that don't I imagine it'd be more trouble than it's
> > worth to work out where the token ended up so you'd be just as well
> > throwing the GCS away and starting over but that seems reasonable.

> Should it be a stick state after it sets?

Yes, I'd implement this as a mode like the existing ones so it'd stay
set and would be inherited by new threads.  Or just a non-optional
feature for that matter, we'd have to see how things go in review but it
feels safer to allow userspace to control if it gets this behaviour.

> I am not sure how to improve the kernel/userland iteration; both parts work at different
> paces and not always in the same direction. We also have different runtimes with different 
> requirements, and not always are they readily available for feedback unless someone actually
> starts to implement the proposed API.

Yes, I think I would expect a lot of the review for things like this to
come from the review of the userspace implementations.

> The last time I tried to come up with some ideas on how to improve this [2], the 
> impression I got was that the kernel was more willing to create a nolibc library 
> with syscall wrappers to avoid such a burden.

> [1] https://sourceware.org/pipermail/libc-alpha/2025-July/168475.html
> [2] https://lpc.events/event/2/sessions/33/

Do you have a timestamp in that video (or ideally some slides or
something)?  The chapter markers aren't terribly useful and the video is
two and a half hours long.  There is growing usage of nolibc but a lot
of it is more about testing interfaces that glibc is supposed to be
using without trampling over glibc than anything else.

> > There is no requirement to use this interface at all, if userspace
> > doesn't use this interface it will continue to see the existing
> > behaviour.

> Right, and I think adding a pthread extension to set up the shadow stack size would 
> be really error-prone and difficult to use. It would require callers to know the call 
> stack size and tune accordingly. I have seen reports where programs fail with 
> different stack sizes because either the C runtime has a different size than glibc, 
> or the user uses a smaller size than the required one. This issue tends to be difficult 
> to debug, but it may be less pronounced with the shadow stack. 

Whenever it's come up in discussion it's always seemed much more likely
that someone will run into trouble with the regular stack before the
shadow stack since the size of the shadow stack has fewer variable
elements, if you're going to pick the wrong number it'll probably show
up with the regular stack first.

> I think a prctl to adjust the factor might work better, and we can add a tunable so 
> users might adjust during process startup. It will not be per-thread, though.

Right, that's the sort of thing I was thinking of for a prctl.  We could
do absolute sizes as well but that doesn't feel like such a good fit.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20250918/02eaacdf/attachment.sig>