glibc 2.32 rseq support incompatible with Firefox sandbox

[Szabolcs Nagy]

The 07/09/2020 17:03, Florian Weimer via Libc-alpha wrote:
> I tried to run Firefox on current the current glibc development branch
> which will become glibc 2.32 in early August.  It fails with SIGSYS
> during rseq registration:
> 
> Core was generated by `/home/test/firefox/firefox-bin -contentproc -childID 6 -isForBrowser -prefsLen'.
> Program terminated with signal SIGSYS, Bad system call.
> #0 rseq_register_current_thread ()
> at ../sysdeps/unix/sysv/linux/rseq-internal.h:38
> 38 if (INTERNAL_SYSCALL_ERROR_P (ret))
> [Current thread is 1 (Thread 0x7f545a45e640 (LWP 5932))]
> (gdb) l
> 33 if (__rseq_abi.cpu_id != RSEQ_CPU_ID_UNINITIALIZED)
> 34 __libc_fatal ("glibc fatal error: "
> 35 "rseq already initialized for this thread\n");
> 36 ret = INTERNAL_SYSCALL_CALL (rseq, &__rseq_abi, sizeof (struct rseq),
> 37 0, RSEQ_SIG);
> 38 if (INTERNAL_SYSCALL_ERROR_P (ret))
> 39 {
> 40 const char *msg = NULL;
> 41
> 42 switch (INTERNAL_SYSCALL_ERRNO (ret))
> 
> (gdb) bt
> #0 rseq_register_current_thread ()
> at ../sysdeps/unix/sysv/linux/rseq-internal.h:38
> #1 start_thread (arg=<optimized out>) at pthread_create.c:390
> #2 0x00007f546b86d283 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
> 
> This looks like the earlier dlopen problem because during the creation
> of new threads, all signals are blocked. This is required for
> correctness of the rseq implementation, so that applications cannot
> observe a thread state during which rseq is not registered.
> 
> I filed a Firefox bug here:
> 
>   <https://bugzilla.mozilla.org/show_bug.cgi?id=1651701>
> 
> How can we work together to fix this?

so SIGSYS is raised because rseq is not an allowed
syscall but treated with SECCOMP_RET_TRAP in the
sandbox.

the sandbox expects to handle the SIGSYS but if the
signal is blocked then the process is terminated.

this sandbox design is broken: blocking signals is
valid and new syscalls are added all the time (e.g.
all time related syscalls just got replaced on 32bit
systems).

the libc has a backward abi compatibility guarantee:
a binary that works with an old libc must work with
a new libc too. such sandboxes break this guarantee
creating problems where ppl keep their libc or linux
system downgraded so their binaries work.

new syscalls may be necessary to fix security bugs
so this potentially prevents deploying security
updates too.

if firefox wants to filter syscalls it must coordinate
with the libc and not poke at the linux abi behind
its back.