[PATCH 10/12] riscv/cfi: Adjust setjmp/longjmp for shadow stack to work
Deepak Gupta
debug@rivosinc.com
Tue Jun 24 06:54:07 GMT 2025
On Mon, Jun 23, 2025 at 6:38 PM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Tue, Jun 24, 2025 at 7:52 AM Deepak Gupta <debug@rivosinc.com> wrote:
> >
> > Hmm..., there were multiple issues which led to different decisions.
> > One of them being not allowing write capability to shadow stack, another
> > being userspace changes landing before kernel changes and that led to
> > some compat changes. I don't exactly remember the reason to use increment
> > ssp or search for token instead of directly using the mechanism for saving
> > then token and later restoring it.
>
> On x86, setjmp doesn't put a restore token on shadow stack since if longjmp
> isn't called, the restore token will left on show stack which causes
> shadow stack mismatch.
>
> BTW, sysdeps/x86/tst-setjmp-cet.c is the shadow stack test.
Aah that's right. Thanks HJ.
I think Jesse was also trying to say the same thing that on normal
return (without longjmp),
restore token will get consumed on some return and will fault. I
didn't get it first time. Sorry
about that.
The problem is not unique to x86. This puts RISC-V into same situation.
I don't see a way out except increment ssp to restore ssp. So your
first approach was
correct (again sorry about this. I had forgotten this detail)
To find shadow stack base, we can find some place in TLS on RISC-V?
>
>
> > I vaguely remember that qemu co-routines use jmpbufs and ucontext quite
> > creatively.
> >
> > CCing HJ and Nagy, if they can recollect.
> >
> > On Tue, Jun 24, 2025 at 04:02:24AM +0800, Jesse Huang wrote:
> > >After a little bit of thought, I ran into a problem.
> > >
> > >Saving a restore token on top of the shadow stack indicates the shadow
> > >stack is
> > >no longer active and usable, because any subsequent use would just overwrite
> > >the restore token, or, if we keep the token by decreasing the ssp by a
> > >slot, then it
> > >simply crashes on some normal return that pops the restore token.
> >
> > Why would that no applicable to stack (I mean overwriting of stack) ?
> > All we are trying to do is balance both the stack.
> > In case of data stack, we record the pointer in buffer (jmpbuf or context)
> > In case of shadow stack, we store the token and then record the pointer.
> >
> > During restore, we restore data stack and then validate shadow stack token
> > and then restore it.
> >
> > >
> > >I checked the longjmp implementation of x86, they are still using unwinding
> > >for this
> > >case (where target and source share the same shadow stack). But we get back
> > >to the
> > >problem that we can't distinguish both without saving the base in ucontext.
> > >
> > >Another thing I don't understand is that x86 try to search for a restore
> > >token on the jmp_buf
> > >in longjmp, then unwind if it fails to find one, while setjmp should be the
> > >only possible producer
> > >of the jmp_buf and they don't save restore token in setjmp. So what could
> > >be the other sources
> > >of jmp_buf that holds a restore token, or are they trying to support some
> > >hand-crafted jmp_buf?
> >
> > I seem to have forgotten some detail. But here is detailed discussion during
> > x86 patches. On x86 side there was one complication of user stuff patches
> > merged before kernel patches. But I don't remember if that was the reason
> > to perform unwind using `incssp` (if bases are same) or search for token if
> > they are discontigous.
> >
> > Pasting a URL to relevant discussion
> > https://lore.kernel.org/all/CAMe9rOpZYwD=v0vcseBrjNvMy4J3Kgy2i8hCcBsU+1gNUcR9qA@mail.gmail.com/
> > Although it was more about alternate signal stack.
> >
> > Added HJ and Nagy. Perhaps they can provide more context behind the reasons
> > of unwinding this way.
> >
> > >
> > >Deepak Gupta <debug@rivosinc.com> 於 2025年6月19日 週四 上午4:18寫道：
> > >
> > >> Hi Jesse,
> > >>
> > >> Thanks for taking the lead.
> > >>
> > >> Can we do save and restore a bit differently using a token mechanism.
> > >>
> > >> On Wed, Jun 18, 2025 at 01:42:56AM -0700, Jesse Huang wrote:
> > >> >Since longjmp to a previous setjmp'ed state could change the stack
> > >> >frame and involves stack frame unwinding, shadow stacks is also required
> > >> >to be unwinded.
> > >> >
> > >> >The unwinding is implemented according to the zicfiss spec by increasing
> > >> >the ssp by a page size (4K) at most, to prevent from accidentally point
> > >> >to another legal shadow stack page after the adjustment.
> > >> >---
> > >> > sysdeps/riscv/__longjmp.S | 30 ++++++++++++++++++++++++++++++
> > >> > sysdeps/riscv/bits/setjmp.h | 4 ++++
> > >> > sysdeps/riscv/setjmp.S | 10 ++++++++++
> > >> > 3 files changed, 44 insertions(+)
> > >> >
> > >> >diff --git a/sysdeps/riscv/__longjmp.S b/sysdeps/riscv/__longjmp.S
> > >> >index 47ff3aa09e..3f1fc3148d 100644
> > >> >--- a/sysdeps/riscv/__longjmp.S
> > >> >+++ b/sysdeps/riscv/__longjmp.S
> > >> >@@ -51,6 +51,36 @@ ENTRY (__longjmp)
> > >> > FREG_L fs11,14*SZREG+11*SZFREG(a0)
> > >> > #endif
> > >> >
> > >> >+#ifdef __riscv_shadow_stack
> > >> >+ /* read ssp into t0 */
> > >> >+ ssrdp t0
> > >> >+ /* skip unwinding if ss is not enabled */
> > >> >+ beqz t0, .Lunwind_fin
> > >> >+# ifndef __riscv_float_abi_soft
> > >> >+ REG_L t1, 14*SZREG+12*SZFREG(a0)
> > >> >+# else
> > >> >+ REG_L t1, 14*SZREG(a0)
> > >> >+# endif
> > >> >+.Lunwind:
> > >> >+ /* should not be taken in normal condition */
> > >> >+ bleu t1, t0, .Lunwind_fin
> > >> >+ /* The unwinding algorithm came from the shadow_stack spec,
> > >> increase ssp
> > >> >+ by at most a page size to ensure always run into a guard page
> > >> >+ before accidentally point to another legal shadow stack page
> > >> */
> > >> >+ /* t0 = (t1 - t0 >= 4096) ? t0 + 4096 : t1 */
> > >> >+ lui a0, 1
> > >> >+ add t0, t0, a0
> > >> >+ bleu t0, t1, 1f
> > >> >+ mv t0, t1
> > >> >+1:
> > >> >+ csrw ssp, t0
> > >> >+ /* Test if the location pointed by ssp is legal */
> > >> >+ sspush x5
> > >> >+ sspopchk x5
> > >>
> > >> Above is useful in C++ exception unwinding where a lot of other stuff gets
> > >> unwinded too (registers, etc) as per static information in DWARF (I think
> > >> its
> > >> in dwarf)
> > >>
> > >> In case of setjmp/longjmp, we can use shadow stack token mechainsm.
> > >> We can do below instead to restore shadow stack pointer (see save token
> > >> in my comments in setjmp)
> > >>
> > >> ssrdp t0
> > >> beqz t0, .Lskip_ss_token_rstor
> > >>
> > >> /*
> > >> * non-zero t0 means shadow stack is enabled. thus ssamoswap/csrw
> > >> to
> > >> * ssp won't fault.
> > >> */
> > >>
> > >> REG_L t0, 14*SZREG(a0) /* get the saved ss pointer */
> > >> ssamoswap t1, x0, (t0) /* Get the saved token, and store 0 */
> > >> addi t0, t0, 8 /* increment ss pointer */
> > >> bneq t0, t1, fatal /* ss ptr and token must match */
> > >> csrw ssp, t0 /* we did validation and restoring ssp
> > >> correctly */
> > >>
> > >> .Lskip_ss_token_rstor:
> > >>
> > >>
> > >> fatal:
> > >> terminate or recovery action
> > >>
> > >> >+ j .Lunwind
> > >> >+.Lunwind_fin:
> > >> >+#endif
> > >> >+
> > >> > seqz a0, a1
> > >> > add a0, a0, a1 # a0 = (a1 == 0) ? 1 : a1
> > >> > ret
> > >> >diff --git a/sysdeps/riscv/bits/setjmp.h b/sysdeps/riscv/bits/setjmp.h
> > >> >index 5ebbec393a..5cbc12fa3d 100644
> > >> >--- a/sysdeps/riscv/bits/setjmp.h
> > >> >+++ b/sysdeps/riscv/bits/setjmp.h
> > >> >@@ -33,6 +33,10 @@ typedef struct __jmp_buf_internal_tag
> > >> > double __fpregs[12];
> > >> > #elif !defined __riscv_float_abi_soft
> > >> > # error unsupported FLEN
> > >> >+#endif
> > >> >+#ifdef __riscv_shadow_stack
> > >> >+ /* Shadow stack pointer. */
> > >> >+ long int __ssp;
> > >> > #endif
> > >> > } __jmp_buf[1];
> > >> >
> > >> >diff --git a/sysdeps/riscv/setjmp.S b/sysdeps/riscv/setjmp.S
> > >> >index df048cb544..f547bc0238 100644
> > >> >--- a/sysdeps/riscv/setjmp.S
> > >> >+++ b/sysdeps/riscv/setjmp.S
> > >> >@@ -61,6 +61,16 @@ ENTRY (__sigsetjmp)
> > >> > FREG_S fs11,14*SZREG+11*SZFREG(a0)
> > >> > #endif
> > >> >
> > >> >+#ifdef __riscv_shadow_stack
> > >> >+ /* read ssp into t0 */
> > >> >+ ssrdp t0
> > >>
> > >> It will be better if we do something like below to save shadow stack
> > >> pointer
> > >> as token on shadow stack itself.
> > >>
> > >> ssrdp t0
> > >> beqz t0, .Lskip_ss_token_prep
> > >>
> > >> mv t1, t0 /* record current ss pointer in t1 */
> > >> addi t0, t0, -8 /* decrement current ss pointer by 8 */
> > >> ssamoswap x0, t1, (t0) /* save away current ss pointer on ss
> > >> itself */
> > >>
> > >> .Lskip_ss_token_prep:
> > >> >+# ifndef __riscv_float_abi_soft
> > >> >+ REG_S t0, 14*SZREG+12*SZFREG(a0)
> > >> >+# else
> > >> >+ REG_S t0, 14*SZREG(a0)
> > >> >+# endif
> > >> >+#endif
> > >> >+
> > >> > #if !IS_IN (libc) && IS_IN (rtld)
> > >> > /* In ld.so we never save the signal mask. */
> > >> > li a0, 0
> > >> >--
> > >> >2.39.3
> > >> >
> > >>
>
>
>
> --
> H.J.
More information about the Libc-alpha
mailing list