[PATCH 10/12] riscv/cfi: Adjust setjmp/longjmp for shadow stack to work
Deepak Gupta
debug@rivosinc.com
Mon Jun 23 23:52:49 GMT 2025
Hmm..., there were multiple issues which led to different decisions.
One of them being not allowing write capability to shadow stack, another
being userspace changes landing before kernel changes and that led to
some compat changes. I don't exactly remember the reason to use increment
ssp or search for token instead of directly using the mechanism for saving
then token and later restoring it.
I vaguely remember that qemu co-routines use jmpbufs and ucontext quite
creatively.
CCing HJ and Nagy, if they can recollect.
On Tue, Jun 24, 2025 at 04:02:24AM +0800, Jesse Huang wrote:
>After a little bit of thought, I ran into a problem.
>
>Saving a restore token on top of the shadow stack indicates the shadow
>stack is
>no longer active and usable, because any subsequent use would just overwrite
>the restore token, or, if we keep the token by decreasing the ssp by a
>slot, then it
>simply crashes on some normal return that pops the restore token.
Why would that no applicable to stack (I mean overwriting of stack) ?
All we are trying to do is balance both the stack.
In case of data stack, we record the pointer in buffer (jmpbuf or context)
In case of shadow stack, we store the token and then record the pointer.
During restore, we restore data stack and then validate shadow stack token
and then restore it.
>
>I checked the longjmp implementation of x86, they are still using unwinding
>for this
>case (where target and source share the same shadow stack). But we get back
>to the
>problem that we can't distinguish both without saving the base in ucontext.
>
>Another thing I don't understand is that x86 try to search for a restore
>token on the jmp_buf
>in longjmp, then unwind if it fails to find one, while setjmp should be the
>only possible producer
>of the jmp_buf and they don't save restore token in setjmp. So what could
>be the other sources
>of jmp_buf that holds a restore token, or are they trying to support some
>hand-crafted jmp_buf?
I seem to have forgotten some detail. But here is detailed discussion during
x86 patches. On x86 side there was one complication of user stuff patches
merged before kernel patches. But I don't remember if that was the reason
to perform unwind using `incssp` (if bases are same) or search for token if
they are discontigous.
Pasting a URL to relevant discussion
https://lore.kernel.org/all/CAMe9rOpZYwD=v0vcseBrjNvMy4J3Kgy2i8hCcBsU+1gNUcR9qA@mail.gmail.com/
Although it was more about alternate signal stack.
Added HJ and Nagy. Perhaps they can provide more context behind the reasons
of unwinding this way.
>
>Deepak Gupta <debug@rivosinc.com> 於 2025年6月19日 週四 上午4:18寫道：
>
>> Hi Jesse,
>>
>> Thanks for taking the lead.
>>
>> Can we do save and restore a bit differently using a token mechanism.
>>
>> On Wed, Jun 18, 2025 at 01:42:56AM -0700, Jesse Huang wrote:
>> >Since longjmp to a previous setjmp'ed state could change the stack
>> >frame and involves stack frame unwinding, shadow stacks is also required
>> >to be unwinded.
>> >
>> >The unwinding is implemented according to the zicfiss spec by increasing
>> >the ssp by a page size (4K) at most, to prevent from accidentally point
>> >to another legal shadow stack page after the adjustment.
>> >---
>> > sysdeps/riscv/__longjmp.S | 30 ++++++++++++++++++++++++++++++
>> > sysdeps/riscv/bits/setjmp.h | 4 ++++
>> > sysdeps/riscv/setjmp.S | 10 ++++++++++
>> > 3 files changed, 44 insertions(+)
>> >
>> >diff --git a/sysdeps/riscv/__longjmp.S b/sysdeps/riscv/__longjmp.S
>> >index 47ff3aa09e..3f1fc3148d 100644
>> >--- a/sysdeps/riscv/__longjmp.S
>> >+++ b/sysdeps/riscv/__longjmp.S
>> >@@ -51,6 +51,36 @@ ENTRY (__longjmp)
>> > FREG_L fs11,14*SZREG+11*SZFREG(a0)
>> > #endif
>> >
>> >+#ifdef __riscv_shadow_stack
>> >+ /* read ssp into t0 */
>> >+ ssrdp t0
>> >+ /* skip unwinding if ss is not enabled */
>> >+ beqz t0, .Lunwind_fin
>> >+# ifndef __riscv_float_abi_soft
>> >+ REG_L t1, 14*SZREG+12*SZFREG(a0)
>> >+# else
>> >+ REG_L t1, 14*SZREG(a0)
>> >+# endif
>> >+.Lunwind:
>> >+ /* should not be taken in normal condition */
>> >+ bleu t1, t0, .Lunwind_fin
>> >+ /* The unwinding algorithm came from the shadow_stack spec,
>> increase ssp
>> >+ by at most a page size to ensure always run into a guard page
>> >+ before accidentally point to another legal shadow stack page
>> */
>> >+ /* t0 = (t1 - t0 >= 4096) ? t0 + 4096 : t1 */
>> >+ lui a0, 1
>> >+ add t0, t0, a0
>> >+ bleu t0, t1, 1f
>> >+ mv t0, t1
>> >+1:
>> >+ csrw ssp, t0
>> >+ /* Test if the location pointed by ssp is legal */
>> >+ sspush x5
>> >+ sspopchk x5
>>
>> Above is useful in C++ exception unwinding where a lot of other stuff gets
>> unwinded too (registers, etc) as per static information in DWARF (I think
>> its
>> in dwarf)
>>
>> In case of setjmp/longjmp, we can use shadow stack token mechainsm.
>> We can do below instead to restore shadow stack pointer (see save token
>> in my comments in setjmp)
>>
>> ssrdp t0
>> beqz t0, .Lskip_ss_token_rstor
>>
>> /*
>> * non-zero t0 means shadow stack is enabled. thus ssamoswap/csrw
>> to
>> * ssp won't fault.
>> */
>>
>> REG_L t0, 14*SZREG(a0) /* get the saved ss pointer */
>> ssamoswap t1, x0, (t0) /* Get the saved token, and store 0 */
>> addi t0, t0, 8 /* increment ss pointer */
>> bneq t0, t1, fatal /* ss ptr and token must match */
>> csrw ssp, t0 /* we did validation and restoring ssp
>> correctly */
>>
>> .Lskip_ss_token_rstor:
>>
>>
>> fatal:
>> terminate or recovery action
>>
>> >+ j .Lunwind
>> >+.Lunwind_fin:
>> >+#endif
>> >+
>> > seqz a0, a1
>> > add a0, a0, a1 # a0 = (a1 == 0) ? 1 : a1
>> > ret
>> >diff --git a/sysdeps/riscv/bits/setjmp.h b/sysdeps/riscv/bits/setjmp.h
>> >index 5ebbec393a..5cbc12fa3d 100644
>> >--- a/sysdeps/riscv/bits/setjmp.h
>> >+++ b/sysdeps/riscv/bits/setjmp.h
>> >@@ -33,6 +33,10 @@ typedef struct __jmp_buf_internal_tag
>> > double __fpregs[12];
>> > #elif !defined __riscv_float_abi_soft
>> > # error unsupported FLEN
>> >+#endif
>> >+#ifdef __riscv_shadow_stack
>> >+ /* Shadow stack pointer. */
>> >+ long int __ssp;
>> > #endif
>> > } __jmp_buf[1];
>> >
>> >diff --git a/sysdeps/riscv/setjmp.S b/sysdeps/riscv/setjmp.S
>> >index df048cb544..f547bc0238 100644
>> >--- a/sysdeps/riscv/setjmp.S
>> >+++ b/sysdeps/riscv/setjmp.S
>> >@@ -61,6 +61,16 @@ ENTRY (__sigsetjmp)
>> > FREG_S fs11,14*SZREG+11*SZFREG(a0)
>> > #endif
>> >
>> >+#ifdef __riscv_shadow_stack
>> >+ /* read ssp into t0 */
>> >+ ssrdp t0
>>
>> It will be better if we do something like below to save shadow stack
>> pointer
>> as token on shadow stack itself.
>>
>> ssrdp t0
>> beqz t0, .Lskip_ss_token_prep
>>
>> mv t1, t0 /* record current ss pointer in t1 */
>> addi t0, t0, -8 /* decrement current ss pointer by 8 */
>> ssamoswap x0, t1, (t0) /* save away current ss pointer on ss
>> itself */
>>
>> .Lskip_ss_token_prep:
>> >+# ifndef __riscv_float_abi_soft
>> >+ REG_S t0, 14*SZREG+12*SZFREG(a0)
>> >+# else
>> >+ REG_S t0, 14*SZREG(a0)
>> >+# endif
>> >+#endif
>> >+
>> > #if !IS_IN (libc) && IS_IN (rtld)
>> > /* In ld.so we never save the signal mask. */
>> > li a0, 0
>> >--
>> >2.39.3
>> >
>>
More information about the Libc-alpha
mailing list