glibc 2.32 rseq support incompatible with Firefox sandbox

[Florian Weimer]

I tried to run Firefox on current the current glibc development branch
which will become glibc 2.32 in early August.  It fails with SIGSYS
during rseq registration:

Core was generated by `/home/test/firefox/firefox-bin -contentproc -childID 6 -isForBrowser -prefsLen'.
Program terminated with signal SIGSYS, Bad system call.
#0 rseq_register_current_thread ()
at ../sysdeps/unix/sysv/linux/rseq-internal.h:38
38 if (INTERNAL_SYSCALL_ERROR_P (ret))
[Current thread is 1 (Thread 0x7f545a45e640 (LWP 5932))]
(gdb) l
33 if (__rseq_abi.cpu_id != RSEQ_CPU_ID_UNINITIALIZED)
34 __libc_fatal ("glibc fatal error: "
35 "rseq already initialized for this thread\n");
36 ret = INTERNAL_SYSCALL_CALL (rseq, &__rseq_abi, sizeof (struct rseq),
37 0, RSEQ_SIG);
38 if (INTERNAL_SYSCALL_ERROR_P (ret))
39 {
40 const char *msg = NULL;
41
42 switch (INTERNAL_SYSCALL_ERRNO (ret))

(gdb) bt
#0 rseq_register_current_thread ()
at ../sysdeps/unix/sysv/linux/rseq-internal.h:38
#1 start_thread (arg=<optimized out>) at pthread_create.c:390
#2 0x00007f546b86d283 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

This looks like the earlier dlopen problem because during the creation
of new threads, all signals are blocked. This is required for
correctness of the rseq implementation, so that applications cannot
observe a thread state during which rseq is not registered.

I filed a Firefox bug here:

  <https://bugzilla.mozilla.org/show_bug.cgi?id=1651701>

How can we work together to fix this?

Thanks,
Florian