[PATCH 10/12] riscv/cfi: Adjust setjmp/longjmp for shadow stack to work
H.J. Lu
hjl.tools@gmail.com
Tue Jun 24 01:38:18 GMT 2025
On Tue, Jun 24, 2025 at 7:52 AM Deepak Gupta <debug@rivosinc.com> wrote:
>
> Hmm..., there were multiple issues which led to different decisions.
> One of them being not allowing write capability to shadow stack, another
> being userspace changes landing before kernel changes and that led to
> some compat changes. I don't exactly remember the reason to use increment
> ssp or search for token instead of directly using the mechanism for saving
> then token and later restoring it.
On x86, setjmp doesn't put a restore token on shadow stack since if longjmp
isn't called, the restore token will left on show stack which causes
shadow stack mismatch.
BTW, sysdeps/x86/tst-setjmp-cet.c is the shadow stack test.
> I vaguely remember that qemu co-routines use jmpbufs and ucontext quite
> creatively.
>
> CCing HJ and Nagy, if they can recollect.
>
> On Tue, Jun 24, 2025 at 04:02:24AM +0800, Jesse Huang wrote:
> >After a little bit of thought, I ran into a problem.
> >
> >Saving a restore token on top of the shadow stack indicates the shadow
> >stack is
> >no longer active and usable, because any subsequent use would just overwrite
> >the restore token, or, if we keep the token by decreasing the ssp by a
> >slot, then it
> >simply crashes on some normal return that pops the restore token.
>
> Why would that no applicable to stack (I mean overwriting of stack) ?
> All we are trying to do is balance both the stack.
> In case of data stack, we record the pointer in buffer (jmpbuf or context)
> In case of shadow stack, we store the token and then record the pointer.
>
> During restore, we restore data stack and then validate shadow stack token
> and then restore it.
>
> >
> >I checked the longjmp implementation of x86, they are still using unwinding
> >for this
> >case (where target and source share the same shadow stack). But we get back
> >to the
> >problem that we can't distinguish both without saving the base in ucontext.
> >
> >Another thing I don't understand is that x86 try to search for a restore
> >token on the jmp_buf
> >in longjmp, then unwind if it fails to find one, while setjmp should be the
> >only possible producer
> >of the jmp_buf and they don't save restore token in setjmp. So what could
> >be the other sources
> >of jmp_buf that holds a restore token, or are they trying to support some
> >hand-crafted jmp_buf?
>
> I seem to have forgotten some detail. But here is detailed discussion during
> x86 patches. On x86 side there was one complication of user stuff patches
> merged before kernel patches. But I don't remember if that was the reason
> to perform unwind using `incssp` (if bases are same) or search for token if
> they are discontigous.
>
> Pasting a URL to relevant discussion
> https://lore.kernel.org/all/CAMe9rOpZYwD=v0vcseBrjNvMy4J3Kgy2i8hCcBsU+1gNUcR9qA@mail.gmail.com/
> Although it was more about alternate signal stack.
>
> Added HJ and Nagy. Perhaps they can provide more context behind the reasons
> of unwinding this way.
>
> >
> >Deepak Gupta <debug@rivosinc.com> 於 2025年6月19日 週四 上午4:18寫道：
> >
> >> Hi Jesse,
> >>
> >> Thanks for taking the lead.
> >>
> >> Can we do save and restore a bit differently using a token mechanism.
> >>
> >> On Wed, Jun 18, 2025 at 01:42:56AM -0700, Jesse Huang wrote:
> >> >Since longjmp to a previous setjmp'ed state could change the stack
> >> >frame and involves stack frame unwinding, shadow stacks is also required
> >> >to be unwinded.
> >> >
> >> >The unwinding is implemented according to the zicfiss spec by increasing
> >> >the ssp by a page size (4K) at most, to prevent from accidentally point
> >> >to another legal shadow stack page after the adjustment.
> >> >---
> >> > sysdeps/riscv/__longjmp.S | 30 ++++++++++++++++++++++++++++++
> >> > sysdeps/riscv/bits/setjmp.h | 4 ++++
> >> > sysdeps/riscv/setjmp.S | 10 ++++++++++
> >> > 3 files changed, 44 insertions(+)
> >> >
> >> >diff --git a/sysdeps/riscv/__longjmp.S b/sysdeps/riscv/__longjmp.S
> >> >index 47ff3aa09e..3f1fc3148d 100644
> >> >--- a/sysdeps/riscv/__longjmp.S
> >> >+++ b/sysdeps/riscv/__longjmp.S
> >> >@@ -51,6 +51,36 @@ ENTRY (__longjmp)
> >> > FREG_L fs11,14*SZREG+11*SZFREG(a0)
> >> > #endif
> >> >
> >> >+#ifdef __riscv_shadow_stack
> >> >+ /* read ssp into t0 */
> >> >+ ssrdp t0
> >> >+ /* skip unwinding if ss is not enabled */
> >> >+ beqz t0, .Lunwind_fin
> >> >+# ifndef __riscv_float_abi_soft
> >> >+ REG_L t1, 14*SZREG+12*SZFREG(a0)
> >> >+# else
> >> >+ REG_L t1, 14*SZREG(a0)
> >> >+# endif
> >> >+.Lunwind:
> >> >+ /* should not be taken in normal condition */
> >> >+ bleu t1, t0, .Lunwind_fin
> >> >+ /* The unwinding algorithm came from the shadow_stack spec,
> >> increase ssp
> >> >+ by at most a page size to ensure always run into a guard page
> >> >+ before accidentally point to another legal shadow stack page
> >> */
> >> >+ /* t0 = (t1 - t0 >= 4096) ? t0 + 4096 : t1 */
> >> >+ lui a0, 1
> >> >+ add t0, t0, a0
> >> >+ bleu t0, t1, 1f
> >> >+ mv t0, t1
> >> >+1:
> >> >+ csrw ssp, t0
> >> >+ /* Test if the location pointed by ssp is legal */
> >> >+ sspush x5
> >> >+ sspopchk x5
> >>
> >> Above is useful in C++ exception unwinding where a lot of other stuff gets
> >> unwinded too (registers, etc) as per static information in DWARF (I think
> >> its
> >> in dwarf)
> >>
> >> In case of setjmp/longjmp, we can use shadow stack token mechainsm.
> >> We can do below instead to restore shadow stack pointer (see save token
> >> in my comments in setjmp)
> >>
> >> ssrdp t0
> >> beqz t0, .Lskip_ss_token_rstor
> >>
> >> /*
> >> * non-zero t0 means shadow stack is enabled. thus ssamoswap/csrw
> >> to
> >> * ssp won't fault.
> >> */
> >>
> >> REG_L t0, 14*SZREG(a0) /* get the saved ss pointer */
> >> ssamoswap t1, x0, (t0) /* Get the saved token, and store 0 */
> >> addi t0, t0, 8 /* increment ss pointer */
> >> bneq t0, t1, fatal /* ss ptr and token must match */
> >> csrw ssp, t0 /* we did validation and restoring ssp
> >> correctly */
> >>
> >> .Lskip_ss_token_rstor:
> >>
> >>
> >> fatal:
> >> terminate or recovery action
> >>
> >> >+ j .Lunwind
> >> >+.Lunwind_fin:
> >> >+#endif
> >> >+
> >> > seqz a0, a1
> >> > add a0, a0, a1 # a0 = (a1 == 0) ? 1 : a1
> >> > ret
> >> >diff --git a/sysdeps/riscv/bits/setjmp.h b/sysdeps/riscv/bits/setjmp.h
> >> >index 5ebbec393a..5cbc12fa3d 100644
> >> >--- a/sysdeps/riscv/bits/setjmp.h
> >> >+++ b/sysdeps/riscv/bits/setjmp.h
> >> >@@ -33,6 +33,10 @@ typedef struct __jmp_buf_internal_tag
> >> > double __fpregs[12];
> >> > #elif !defined __riscv_float_abi_soft
> >> > # error unsupported FLEN
> >> >+#endif
> >> >+#ifdef __riscv_shadow_stack
> >> >+ /* Shadow stack pointer. */
> >> >+ long int __ssp;
> >> > #endif
> >> > } __jmp_buf[1];
> >> >
> >> >diff --git a/sysdeps/riscv/setjmp.S b/sysdeps/riscv/setjmp.S
> >> >index df048cb544..f547bc0238 100644
> >> >--- a/sysdeps/riscv/setjmp.S
> >> >+++ b/sysdeps/riscv/setjmp.S
> >> >@@ -61,6 +61,16 @@ ENTRY (__sigsetjmp)
> >> > FREG_S fs11,14*SZREG+11*SZFREG(a0)
> >> > #endif
> >> >
> >> >+#ifdef __riscv_shadow_stack
> >> >+ /* read ssp into t0 */
> >> >+ ssrdp t0
> >>
> >> It will be better if we do something like below to save shadow stack
> >> pointer
> >> as token on shadow stack itself.
> >>
> >> ssrdp t0
> >> beqz t0, .Lskip_ss_token_prep
> >>
> >> mv t1, t0 /* record current ss pointer in t1 */
> >> addi t0, t0, -8 /* decrement current ss pointer by 8 */
> >> ssamoswap x0, t1, (t0) /* save away current ss pointer on ss
> >> itself */
> >>
> >> .Lskip_ss_token_prep:
> >> >+# ifndef __riscv_float_abi_soft
> >> >+ REG_S t0, 14*SZREG+12*SZFREG(a0)
> >> >+# else
> >> >+ REG_S t0, 14*SZREG(a0)
> >> >+# endif
> >> >+#endif
> >> >+
> >> > #if !IS_IN (libc) && IS_IN (rtld)
> >> > /* In ld.so we never save the signal mask. */
> >> > li a0, 0
> >> >--
> >> >2.39.3
> >> >
> >>
--
H.J.
More information about the Libc-alpha
mailing list