[PATCH 10/12] riscv/cfi: Adjust setjmp/longjmp for shadow stack to work
Jesse Huang
jesse.huang@sifive.com
Wed Jun 25 19:19:09 GMT 2025
On Tue, Jun 24, 2025 at 2:54 PM Deepak Gupta <debug@rivosinc.com> wrote:
> On Mon, Jun 23, 2025 at 6:38 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> >
> > On Tue, Jun 24, 2025 at 7:52 AM Deepak Gupta <debug@rivosinc.com> wrote:
> > >
> > > Hmm..., there were multiple issues which led to different decisions.
> > > One of them being not allowing write capability to shadow stack,
> another
> > > being userspace changes landing before kernel changes and that led to
> > > some compat changes. I don't exactly remember the reason to use
> increment
> > > ssp or search for token instead of directly using the mechanism for
> saving
> > > then token and later restoring it.
> >
> > On x86, setjmp doesn't put a restore token on shadow stack since if
> longjmp
> > isn't called, the restore token will left on show stack which causes
> > shadow stack mismatch.
> >
> > BTW, sysdeps/x86/tst-setjmp-cet.c is the shadow stack test.
>
> Aah that's right. Thanks HJ.
>
> I think Jesse was also trying to say the same thing that on normal
> return (without longjmp),
> restore token will get consumed on some return and will fault. I
> didn't get it first time. Sorry
> about that.
>
> The problem is not unique to x86. This puts RISC-V into same situation.
> I don't see a way out except increment ssp to restore ssp. So your
> first approach was
> correct (again sorry about this. I had forgotten this detail)
>
> To find shadow stack base, we can find some place in TLS on RISC-V?
>
TLS seems to be a feasible solution. I found x86 storing ssp, base and the
shadow
stack size in the ucontext, however I don't find the shadow stack size is
used
anywhere. I'd like to discuss whether we really need to store the size in
the ucontext?
Since once this gets merged it's nearly impossible to change, I want to
make sure
I'm not missing anything.
>
> >
> > > I vaguely remember that qemu co-routines use jmpbufs and ucontext quite
> > > creatively.
> > >
> > > CCing HJ and Nagy, if they can recollect.
> > >
> > > On Tue, Jun 24, 2025 at 04:02:24AM +0800, Jesse Huang wrote:
> > > >After a little bit of thought, I ran into a problem.
> > > >
> > > >Saving a restore token on top of the shadow stack indicates the shadow
> > > >stack is
> > > >no longer active and usable, because any subsequent use would just
> overwrite
> > > >the restore token, or, if we keep the token by decreasing the ssp by a
> > > >slot, then it
> > > >simply crashes on some normal return that pops the restore token.
> > >
> > > Why would that no applicable to stack (I mean overwriting of stack) ?
> > > All we are trying to do is balance both the stack.
> > > In case of data stack, we record the pointer in buffer (jmpbuf or
> context)
> > > In case of shadow stack, we store the token and then record the
> pointer.
> > >
> > > During restore, we restore data stack and then validate shadow stack
> token
> > > and then restore it.
> > >
> > > >
> > > >I checked the longjmp implementation of x86, they are still using
> unwinding
> > > >for this
> > > >case (where target and source share the same shadow stack). But we
> get back
> > > >to the
> > > >problem that we can't distinguish both without saving the base in
> ucontext.
> > > >
> > > >Another thing I don't understand is that x86 try to search for a
> restore
> > > >token on the jmp_buf
> > > >in longjmp, then unwind if it fails to find one, while setjmp should
> be the
> > > >only possible producer
> > > >of the jmp_buf and they don't save restore token in setjmp. So what
> could
> > > >be the other sources
> > > >of jmp_buf that holds a restore token, or are they trying to support
> some
> > > >hand-crafted jmp_buf?
> > >
> > > I seem to have forgotten some detail. But here is detailed discussion
> during
> > > x86 patches. On x86 side there was one complication of user stuff
> patches
> > > merged before kernel patches. But I don't remember if that was the
> reason
> > > to perform unwind using `incssp` (if bases are same) or search for
> token if
> > > they are discontigous.
> > >
> > > Pasting a URL to relevant discussion
> > >
> https://lore.kernel.org/all/CAMe9rOpZYwD=v0vcseBrjNvMy4J3Kgy2i8hCcBsU+1gNUcR9qA@mail.gmail.com/
> > > Although it was more about alternate signal stack.
> > >
> > > Added HJ and Nagy. Perhaps they can provide more context behind the
> reasons
> > > of unwinding this way.
> > >
> > > >
> > > >Deepak Gupta <debug@rivosinc.com> 於 2025年6月19日 週四 上午4:18寫道：
> > > >
> > > >> Hi Jesse,
> > > >>
> > > >> Thanks for taking the lead.
> > > >>
> > > >> Can we do save and restore a bit differently using a token
> mechanism.
> > > >>
> > > >> On Wed, Jun 18, 2025 at 01:42:56AM -0700, Jesse Huang wrote:
> > > >> >Since longjmp to a previous setjmp'ed state could change the stack
> > > >> >frame and involves stack frame unwinding, shadow stacks is also
> required
> > > >> >to be unwinded.
> > > >> >
> > > >> >The unwinding is implemented according to the zicfiss spec by
> increasing
> > > >> >the ssp by a page size (4K) at most, to prevent from accidentally
> point
> > > >> >to another legal shadow stack page after the adjustment.
> > > >> >---
> > > >> > sysdeps/riscv/__longjmp.S | 30 ++++++++++++++++++++++++++++++
> > > >> > sysdeps/riscv/bits/setjmp.h | 4 ++++
> > > >> > sysdeps/riscv/setjmp.S | 10 ++++++++++
> > > >> > 3 files changed, 44 insertions(+)
> > > >> >
> > > >> >diff --git a/sysdeps/riscv/__longjmp.S b/sysdeps/riscv/__longjmp.S
> > > >> >index 47ff3aa09e..3f1fc3148d 100644
> > > >> >--- a/sysdeps/riscv/__longjmp.S
> > > >> >+++ b/sysdeps/riscv/__longjmp.S
> > > >> >@@ -51,6 +51,36 @@ ENTRY (__longjmp)
> > > >> > FREG_L fs11,14*SZREG+11*SZFREG(a0)
> > > >> > #endif
> > > >> >
> > > >> >+#ifdef __riscv_shadow_stack
> > > >> >+ /* read ssp into t0 */
> > > >> >+ ssrdp t0
> > > >> >+ /* skip unwinding if ss is not enabled */
> > > >> >+ beqz t0, .Lunwind_fin
> > > >> >+# ifndef __riscv_float_abi_soft
> > > >> >+ REG_L t1, 14*SZREG+12*SZFREG(a0)
> > > >> >+# else
> > > >> >+ REG_L t1, 14*SZREG(a0)
> > > >> >+# endif
> > > >> >+.Lunwind:
> > > >> >+ /* should not be taken in normal condition */
> > > >> >+ bleu t1, t0, .Lunwind_fin
> > > >> >+ /* The unwinding algorithm came from the shadow_stack
> spec,
> > > >> increase ssp
> > > >> >+ by at most a page size to ensure always run into a
> guard page
> > > >> >+ before accidentally point to another legal shadow
> stack page
> > > >> */
> > > >> >+ /* t0 = (t1 - t0 >= 4096) ? t0 + 4096 : t1 */
> > > >> >+ lui a0, 1
> > > >> >+ add t0, t0, a0
> > > >> >+ bleu t0, t1, 1f
> > > >> >+ mv t0, t1
> > > >> >+1:
> > > >> >+ csrw ssp, t0
> > > >> >+ /* Test if the location pointed by ssp is legal */
> > > >> >+ sspush x5
> > > >> >+ sspopchk x5
> > > >>
> > > >> Above is useful in C++ exception unwinding where a lot of other
> stuff gets
> > > >> unwinded too (registers, etc) as per static information in DWARF (I
> think
> > > >> its
> > > >> in dwarf)
> > > >>
> > > >> In case of setjmp/longjmp, we can use shadow stack token mechainsm.
> > > >> We can do below instead to restore shadow stack pointer (see save
> token
> > > >> in my comments in setjmp)
> > > >>
> > > >> ssrdp t0
> > > >> beqz t0, .Lskip_ss_token_rstor
> > > >>
> > > >> /*
> > > >> * non-zero t0 means shadow stack is enabled. thus
> ssamoswap/csrw
> > > >> to
> > > >> * ssp won't fault.
> > > >> */
> > > >>
> > > >> REG_L t0, 14*SZREG(a0) /* get the saved ss pointer */
> > > >> ssamoswap t1, x0, (t0) /* Get the saved token, and store 0
> */
> > > >> addi t0, t0, 8 /* increment ss pointer */
> > > >> bneq t0, t1, fatal /* ss ptr and token must match */
> > > >> csrw ssp, t0 /* we did validation and restoring
> ssp
> > > >> correctly */
> > > >>
> > > >> .Lskip_ss_token_rstor:
> > > >>
> > > >>
> > > >> fatal:
> > > >> terminate or recovery action
> > > >>
> > > >> >+ j .Lunwind
> > > >> >+.Lunwind_fin:
> > > >> >+#endif
> > > >> >+
> > > >> > seqz a0, a1
> > > >> > add a0, a0, a1 # a0 = (a1 == 0) ? 1 : a1
> > > >> > ret
> > > >> >diff --git a/sysdeps/riscv/bits/setjmp.h
> b/sysdeps/riscv/bits/setjmp.h
> > > >> >index 5ebbec393a..5cbc12fa3d 100644
> > > >> >--- a/sysdeps/riscv/bits/setjmp.h
> > > >> >+++ b/sysdeps/riscv/bits/setjmp.h
> > > >> >@@ -33,6 +33,10 @@ typedef struct __jmp_buf_internal_tag
> > > >> > double __fpregs[12];
> > > >> > #elif !defined __riscv_float_abi_soft
> > > >> > # error unsupported FLEN
> > > >> >+#endif
> > > >> >+#ifdef __riscv_shadow_stack
> > > >> >+ /* Shadow stack pointer. */
> > > >> >+ long int __ssp;
> > > >> > #endif
> > > >> > } __jmp_buf[1];
> > > >> >
> > > >> >diff --git a/sysdeps/riscv/setjmp.S b/sysdeps/riscv/setjmp.S
> > > >> >index df048cb544..f547bc0238 100644
> > > >> >--- a/sysdeps/riscv/setjmp.S
> > > >> >+++ b/sysdeps/riscv/setjmp.S
> > > >> >@@ -61,6 +61,16 @@ ENTRY (__sigsetjmp)
> > > >> > FREG_S fs11,14*SZREG+11*SZFREG(a0)
> > > >> > #endif
> > > >> >
> > > >> >+#ifdef __riscv_shadow_stack
> > > >> >+ /* read ssp into t0 */
> > > >> >+ ssrdp t0
> > > >>
> > > >> It will be better if we do something like below to save shadow stack
> > > >> pointer
> > > >> as token on shadow stack itself.
> > > >>
> > > >> ssrdp t0
> > > >> beqz t0, .Lskip_ss_token_prep
> > > >>
> > > >> mv t1, t0 /* record current ss pointer in t1
> */
> > > >> addi t0, t0, -8 /* decrement current ss pointer by
> 8 */
> > > >> ssamoswap x0, t1, (t0) /* save away current ss pointer on
> ss
> > > >> itself */
> > > >>
> > > >> .Lskip_ss_token_prep:
> > > >> >+# ifndef __riscv_float_abi_soft
> > > >> >+ REG_S t0, 14*SZREG+12*SZFREG(a0)
> > > >> >+# else
> > > >> >+ REG_S t0, 14*SZREG(a0)
> > > >> >+# endif
> > > >> >+#endif
> > > >> >+
> > > >> > #if !IS_IN (libc) && IS_IN (rtld)
> > > >> > /* In ld.so we never save the signal mask. */
> > > >> > li a0, 0
> > > >> >--
> > > >> >2.39.3
> > > >> >
> > > >>
> >
> >
> >
> > --
> > H.J.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20250626/2260c471/attachment-0001.htm>
More information about the Libc-alpha
mailing list