[PATCH v2 2/3] nptl: Add libc allocated shadow stack for new threads

[Adhemerval Zanella Netto]

On 17/09/25 19:13, Mark Brown wrote:
> On Wed, Sep 17, 2025 at 04:29:36PM -0300, Adhemerval Zanella Netto wrote:
>> On 17/09/25 14:37, Mark Brown wrote:
> 
>>> I think if we want to add an interface that's purely for configuring the
>>> stack size to use that should be separate to clone3() since that's also
>>> an issue for things using plain clone(), including workloads that can't
>>> switch to clone3() due to it not playing nicely with seccomp.
> 
>> It is not clear to me how can you accomplish it without also tying the
>> allocation to some handler the kernel would provide (which is what 
>> map_shadow_stack does essentially).  
> 
> Remember that we have kernel allocated shadow stacks if one isn't
> explictly provided, the size of which is a number we currently pull out
> of thin air as far as userspace is concerned so we pick an extremely
> high value.  We can add a prctl() or something to configure that number.

If the idea is to either provide a pthread API to specify the shadow stack 
size or add a way to automatically set it at runtime, it is still somewhat a 
clunky api. It would require the thread to issue the prctl to change the 
desired shadow stack size ratio and reset it after the clone call. It is
doable nevertheless.
> 
>>> I also previously suggested that if we want to enable reuse of stacks
>>> after a thread has exited we should support having the kernel write a
>>> token to the current pointer of exiting threads (mirroring how the stack
>>> pivot instructions work), I didn't get any feedback on that suggestion.
>>> I would expect this to be gated behind a separate ABI (most likely a new
>>> permission on PR_SHADOW_STACK_SET_STATUS) so readily discoverable if
>>> added independently.
> 
>> Sorry if I missed it, if the kernel can provide it I think it would
>> be useful.
> 
> OK, great.  I think that if we've got that then you'd not have any
> concerns, you'd be able to reuse the stacks from threads that exit
> cleanly?  For those that don't I imagine it'd be more trouble than it's
> worth to work out where the token ended up so you'd be just as well
> throwing the GCS away and starting over but that seems reasonable.

Should it be a stick state after it sets?

> 
>>>> So, imho, the current clone3 interface for shadow stack token only adds an extra burden 
>>>> to the C runtime, and since it is not yet in master, perhaps we can still refine it.
> 
>>> I have to say it is a little surprising to have this come up almost two
>>> years into the review.
> 
>> Perhaps because it was only recently discussed how it would be implemented in glibc?
>> (I do not follow lkml that clone, but maybe I should more).
> 
> There's been glibc people on copy for the kernel thread the whole time,
> I guess the wrong ones though, and I had thought that some earlier code
> had been posted though it's possible I got the wrong end of the stick
> there.

The first RFC from Yury was issued in July of this year [1], and I added some comments,
although only for the first part.

I am not sure how to improve the kernel/userland iteration; both parts work at different
paces and not always in the same direction. We also have different runtimes with different 
requirements, and not always are they readily available for feedback unless someone actually
starts to implement the proposed API.

The last time I tried to come up with some ideas on how to improve this [2], the 
impression I got was that the kernel was more willing to create a nolibc library 
with syscall wrappers to avoid such a burden.

[1] https://sourceware.org/pipermail/libc-alpha/2025-July/168475.html
[2] https://lpc.events/event/2/sessions/33/

> 
>> Previously, when I reviewed the initial GCS work, I had the impression that the shadow 
>> stack was supposed to be transparently managed by the kernel through the clone3 syscall 
>> and that map_shadow_stack was added to manage the corner cases of setjmp calls and the 
>> possible usage of green-threads.
> 
> That's the current case, yes.  All the clone() family of syscalls will
> transparently give you a shadow stack if needed with the shadow stack
> support that is in currently released kernels.  The existing ABI was
> designed prior to clone3() existing at all.
> 
>> What I am trying to say is that with the current ABI, it only adds an extra burden on 
>> userland to manage the shadow stacks for most runtimes. However, if the defined kernel 
>> ABI is used, it can work out this on glibc. 
> 
> There is no requirement to use this interface at all, if userspace
> doesn't use this interface it will continue to see the existing
> behaviour.

Right, and I think adding a pthread extension to set up the shadow stack size would 
be really error-prone and difficult to use. It would require callers to know the call 
stack size and tune accordingly. I have seen reports where programs fail with 
different stack sizes because either the C runtime has a different size than glibc, 
or the user uses a smaller size than the required one. This issue tends to be difficult 
to debug, but it may be less pronounced with the shadow stack. 

I think a prctl to adjust the factor might work better, and we can add a tunable so 
users might adjust during process startup. It will not be per-thread, though.