Localized VLC failure from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=887886 __pthread_register_cancel corrupts stack with rewrite of return value for var_AddCallback on instruction 0x00007ffff79a41d9 <+9>: mov %rax,0xc8(%rdi) Hardware watchpoint 7: *(long int*)0x7fffffffe258 Old value = 140737338902663 New value = 140737488348368 __GI___pthread_register_cancel (buf=0x7fffffffe190) at cleanup.c:32 32 in cleanup.c (gdb) disas Dump of assembler code for function __GI___pthread_register_cancel: 0x00007ffff79a41d0 <+0>: mov %fs:0x300,%rax 0x00007ffff79a41d9 <+9>: mov %rax,0xc8(%rdi) => 0x00007ffff79a41e0 <+16>: mov %fs:0x2f8,%rax 0x00007ffff79a41e9 <+25>: mov %rax,0xd0(%rdi) 0x00007ffff79a41f0 <+32>: mov %rdi,%fs:0x300 0x00007ffff79a41f9 <+41>: retq End of assembler dump. (gdb) c Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00007fffffffe4d0 in ?? () 0x00007fffffffe4d0: add %al,(%rax)
The offset of priv.data.cleanup changed, and old binaries have an insufficiently large stack allocation for the new offset.
It is simple to reproduce: 1. Compile nptl/tst-cleanup1.c with glibc 2.26. 2. Link tst-cleanup1.o with glibc 2.27. 3. Run tst-cleanup1. [hjl@gnu-6 pr22743]$ make gcc -O2 -c -o tst-cleanup1.o tst-cleanup1.c gcc -o tst-cleanup1 tst-cleanup1.o -lpthread gcc -L. -nostdlib -nostartfiles -o tst-cleanup1-dynamic \ -Wl,-dynamic-linker=/export/build/gnu/glibc/build-x86_64-linux/elf/ld-linux-x86-64.so.2 \ -Wl,-z,nocombreloc \ /export/build/gnu/glibc/build-x86_64-linux/csu/crt1.o /export/build/gnu/glibc/build-x86_64-linux/csu/crti.o \ `gcc --print-file-name=crtbegin.o` \ tst-cleanup1.o \ -Wl,-rpath=/export/build/gnu/glibc/build-x86_64-linux:/export/build/gnu/glibc/build-x86_64-linux/nptl:/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64:. \ \ \ /export/build/gnu/glibc/build-x86_64-linux/nptl/libpthread.so /export/build/gnu/glibc/build-x86_64-linux/nptl/libpthread_nonshared.a \ /export/build/gnu/glibc/build-x86_64-linux/libc.so.6 \ /export/build/gnu/glibc/build-x86_64-linux/elf/ld-linux-x86-64.so.2 \ /export/build/gnu/glibc/build-x86_64-linux/libc_nonshared.a \ -Wl,-as-needed /usr/lib/gcc/x86_64-redhat-linux/7/libgcc_s.so -Wl,--no-as-needed `gcc --print-file-name=crtend.o` \ /export/build/gnu/glibc/build-x86_64-linux/csu/crtn.o ./tst-cleanup1 ch (3) ch (2) ch (1) ./tst-cleanup1-dynamic ch (3) ch (2) ch (1) make: *** [Makefile:36: all] Segmentation fault [hjl@gnu-6 pr22743]$
A patch is available at https://sourceware.org/git/?p=glibc.git;a=commit;h=4b7fc470a6740808b41502d7431f91805e272d26 Andrew, please verify if it fixes the crash.
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "GNU C Library master sources". The branch, hjl/pr22743/master has been created at 10bba58b7eece4eac0db07c100217b709efd4727 (commit) - Log ----------------------------------------------------------------- https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=10bba58b7eece4eac0db07c100217b709efd4727 commit 10bba58b7eece4eac0db07c100217b709efd4727 Author: H.J. Lu <hjl.tools@gmail.com> Date: Wed Jan 24 15:27:49 2018 -0800 nptl: Update struct pthread_unwind_buf [BZ #22743] In glibc 2.27, the size of cancel_jmp_buf in struct pthread_unwind_buf has been increased to match the size of __jmp_buf_tag on Linux/x86 in order to save and restore shadow stack. Struct pthread_unwind_buf is used in <pthread.h>, whose address is passed from applications to libpthread. To access the private data in struct pthread_unwind_buf, which is placed after cancel_jmp_buf, in libpthread, we must know which struct pthread_unwind_buf, before glibc 27 and after glibc 2.27, is used in caller. If the size of caller's struct pthread_unwind_buf is smaller than what libpthread expects, libpthread will override caller's stack since struct pthread_unwind_buf is placed on caller's stack. We enable shadow stack at run-time only if program and all used shared objects, including dlopened ones, are shadow stack enabled, which means that they must be compiled with GCC 8 or above and glibc 2.27 or above. Since we need to save and restore shadow stack only if shadow stack is enabled, we can safely assume that caller is compiled with smaller struct pthread_unwind_buf on stack if shadow stack isn't enabled at run-time. For callers with larger struct pthread_unwind_buf, but shadow stack isn't enabled, we just have some unused space on caller's stack. struct pthread_unwind_buf is changed to union of 1. struct cancel_jmp_buf[1], which contains the common fields of struct updated_pthread_unwind_buf and struct compat_pthread_unwind_buf. 2. struct updated_pthread_unwind_buf, which is the updated layout of the cleanup buffer. 3. struct compat_pthread_unwind_buf, which is the compatible layout of the cleanup buffer. A macro, UNWIND_BUF_PRIV, is added to get the pointer to the priv field. By default, it uses the priv field of struct compat_pthread_unwind_buf. If a target defines NEED_SAVED_MASK_IN_CANCEL_JMP_BUF, it must provide its own version of UNEIND_BUF_PRIV to get the pointer to the priv field. On Linux/x86, it uses the priv field of struct compat_pthread_unwind_buf if shadow stack is disabled and struct updated_pthread_unwind_buf if shadow stack is enabled. [BZ #22743] * csu/libc-start.c (LIBC_START_MAIN): Use the updated version of the cleanup buffer. * nptl/cleanup.c (__pthread_register_cancel): Use UNWIND_BUF_PRIV to access the priv field in the cleanup buffer. (__pthread_unregister_cancel): Likewise. * nptl/cleanup_defer.c (__pthread_register_cancel_defer): Likewise. (__pthread_unregister_cancel_restore): Likewise. * nptl/unwind.c (unwind_stop): Likewise. (__pthread_unwind_next): Likewise. * nptl/descr.h (pthread_unwind_buf_data): New. (updated_pthread_unwind_buf): Likewise. (compat_pthread_unwind_buf): Likewise. (pthread_unwind_buf): Updated to use updated_pthread_unwind_buf and compat_pthread_unwind_buf. (UNWIND_BUF_PRIV): New. Macro to get pointer to the priv field in the cleanup buffer. * nptl/pthread_create.c (START_THREAD_DEFN): Use the updated version of the cleanup buffer. (__pthread_create_2_1): Use THREAD_COPY_ADDITONAL_INFO to copy additonal info if defined. * sysdeps/unix/sysv/linux/x86/nptl/pthreadP.h: Use the updated version of the cleanup buffer to check cancel_jmp_buf size. * sysdeps/unix/sysv/linux/x86/pthreaddef.h (THREAD_COPY_ADDITONAL_INFO): New. (UNWIND_BUF_PRIV): Likewise. -----------------------------------------------------------------------
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "GNU C Library master sources". The branch, master has been updated via 2ec0e7eade0ea1258acd5c6f5e5e9bfaeb5041a8 (commit) from 47c4b4b060db0290022dcc37cab7b5ff4bdb5c32 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=2ec0e7eade0ea1258acd5c6f5e5e9bfaeb5041a8 commit 2ec0e7eade0ea1258acd5c6f5e5e9bfaeb5041a8 Author: Carlos O'Donell <carlos@systemhalted.org> Date: Wed Jan 24 20:35:22 2018 -0800 Revert Intel CET changes to __jmp_buf_tag (Bug 22743) In commit cba595c350e52194e10c0006732e1991e3d0803b and commit f81ddabffd76ac9dd600b02adbf3e1dac4bb10ec, ABI compatibility with applications was broken by increasing the size of the on-stack allocated __pthread_unwind_buf_t beyond the oringal size. Applications only have the origianl space available for __pthread_unwind_register, and __pthread_unwind_next to use, any increase in the size of __pthread_unwind_buf_t causes these functions to write beyond the original structure into other on-stack variables leading to segmentation faults in common applications like vlc. The only workaround is to version those functions which operate on the old sized objects, but this must happen in glibc 2.28. Thank you to Andrew Senkevich, H.J. Lu, and Aurelien Jarno, for submitting reports and tracking the issue down. The commit reverts the above mentioned commits and testing on x86_64 shows that the ABI compatibility is restored. A tst-cleanup1 regression test linked with an older glibc now passes when run with the newly built glibc. Previously a tst-cleanup1 linked with an older glibc would segfault when run with an affected glibc build. Tested on x86_64 with no regressions. Signed-off-by: Carlos O'Donell <carlos@redhat.com> ----------------------------------------------------------------------- Summary of changes: ChangeLog | 30 ++++++++++++++++ bits/types/__cancel_jmp_buf_tag.h | 28 --------------- nptl/Makefile | 3 +- nptl/descr.h | 3 -- sysdeps/i386/nptl/tcb-offsets.sym | 1 - sysdeps/i386/nptl/tls.h | 4 -- sysdeps/nptl/pthread.h | 7 +++- sysdeps/unix/sysv/linux/hppa/pthread.h | 7 +++- .../linux/x86/bits/types/__cancel_jmp_buf_tag.h | 31 ----------------- sysdeps/unix/sysv/linux/x86/nptl/pthreadP.h | 36 -------------------- sysdeps/unix/sysv/linux/x86/pthreaddef.h | 22 ------------ sysdeps/x86_64/nptl/tcb-offsets.sym | 1 - sysdeps/x86_64/nptl/tls.h | 5 +-- 13 files changed, 42 insertions(+), 136 deletions(-) delete mode 100644 bits/types/__cancel_jmp_buf_tag.h delete mode 100644 sysdeps/unix/sysv/linux/x86/bits/types/__cancel_jmp_buf_tag.h delete mode 100644 sysdeps/unix/sysv/linux/x86/nptl/pthreadP.h delete mode 100644 sysdeps/unix/sysv/linux/x86/pthreaddef.h
This is fixed in 2.27 by backing out the ABI changes.
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "GNU C Library master sources". The branch, hjl/pr22743/master has been deleted was 10bba58b7eece4eac0db07c100217b709efd4727 - Log ----------------------------------------------------------------- 10bba58b7eece4eac0db07c100217b709efd4727 nptl: Update struct pthread_unwind_buf [BZ #22743] -----------------------------------------------------------------------
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "GNU C Library master sources". The branch, hjl/pr22743/master has been created at d139cd31adaf9f091a36027c8a452b016a05e95c (commit) - Log ----------------------------------------------------------------- https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=d139cd31adaf9f091a36027c8a452b016a05e95c commit d139cd31adaf9f091a36027c8a452b016a05e95c Author: H.J. Lu <hjl.tools@gmail.com> Date: Wed Jan 24 15:27:49 2018 -0800 nptl: Update struct pthread_unwind_buf [BZ #22743] In glibc 2.28, the size of cancel_jmp_buf in struct pthread_unwind_buf has been increased to match the size of __jmp_buf_tag on Linux/x86 in order to save and restore shadow stack register. pthread_unwind_buf is used in <pthread.h>, whose address is passed from applications to libpthread. To access the private data in struct pthread_unwind_buf, which is placed after cancel_jmp_buf, in libpthread, we must know which struct pthread_unwind_buf, before glibc 28 and after glibc 2.28, is used in caller. If the size of caller's struct pthread_unwind_buf is smaller than what libpthread expects, libpthread will override caller's stack since struct pthread_unwind_buf is placed on caller's stack. We enable shadow stack at run-time only if program and all used shared objects, including dlopened ones, are shadow stack enabled, which means that they must be compiled with GCC 8 or above and glibc 2.28 or above. Since we need to save and restore shadow stack register only if shadow stack is enabled, we can safely assume that caller is compiled with smaller struct pthread_unwind_buf on stack if shadow stack isn't enabled at run-time. For callers with larger struct pthread_unwind_buf, but shadow stack isn't enabled, we just have some unused space on caller's stack. struct pthread_unwind_buf is changed to union of 1. struct cancel_jmp_buf[1], which contains the common fields of struct full and struct compat_pthread_unwind_buf. 2. struct full_pthread_unwind_buf, which is the full layout of the cleanup buffer. 3. struct compat_pthread_unwind_buf, which is the compatible layout of the cleanup buffer. A macro, UNWIND_BUF_PRIV, is added to get the pointer to the priv field. By default, it uses the priv field of struct compat_pthread_unwind_buf. If a target defines NEED_SAVED_MASK_IN_CANCEL_JMP_BUF, it must provide its own version of UNEIND_BUF_PRIV to get the pointer to the priv field. On Linux/x86, it uses the priv field of struct compat_pthread_unwind_buf if shadow stack is disabled and struct full_pthread_unwind_buf if shadow stack is enabled. Note: There is an unused pointer space in pthread_unwind_buf_data. But it isn't unsuitable for saving and restoring shadow stack register since x32 is a 64-bit process with 32-bit software pointer and kernel may place x32 shadow stack above 4GB. We need to save and restore 64-bit shadow stack register for x32. [BZ #22743] * csu/libc-start.c (LIBC_START_MAIN): Use the full version of the cleanup buffer. * nptl/cleanup.c (__pthread_register_cancel): Use UNWIND_BUF_PRIV to access the priv field in the cleanup buffer. (__pthread_unregister_cancel): Likewise. * nptl/cleanup_defer.c (__pthread_register_cancel_defer): Likewise. (__pthread_unregister_cancel_restore): Likewise. * nptl/unwind.c (unwind_stop): Likewise. (__pthread_unwind_next): Likewise. * nptl/descr.h (pthread_unwind_buf_data): New. (full_pthread_unwind_buf): Likewise. (compat_pthread_unwind_buf): Likewise. (pthread_unwind_buf): Updated to use full_pthread_unwind_buf and compat_pthread_unwind_buf. (UNWIND_BUF_PRIV): New. Macro to get pointer to the priv field in the cleanup buffer. * nptl/pthread_create.c (START_THREAD_DEFN): Use the full version of the cleanup buffer. (__pthread_create_2_1): Use THREAD_COPY_ADDITONAL_INFO to copy additonal info if defined. * sysdeps/unix/sysv/linux/x86/nptl/pthreadP.h: Use the full version of the cleanup buffer to check cancel_jmp_buf size. * sysdeps/unix/sysv/linux/x86/pthreaddef.h (THREAD_COPY_ADDITONAL_INFO): New. (UNWIND_BUF_PRIV): Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=048fc0aaf81cd328631b3485e348e3e5723cb826 commit 048fc0aaf81cd328631b3485e348e3e5723cb826 Author: H.J. Lu <hjl.tools@gmail.com> Date: Fri Jan 26 05:19:15 2018 -0800 Revert "Revert Intel CET changes to __jmp_buf_tag (Bug 22743)" This reverts commit 2ec0e7eade0ea1258acd5c6f5e5e9bfaeb5041a8. This is needed to save and restore shadow stack register in setjmp and longjmp. [BZ #22563] * sysdeps/i386/nptl/tcb-offsets.sym (FEATURE_1_OFFSET): New. * sysdeps/i386/nptl/tls.h (tcbhead_t): Add feature_1. * sysdeps/x86_64/nptl/tcb-offsets.sym (FEATURE_1_OFFSET): New. * sysdeps/x86_64/nptl/tls.h (tcbhead_t): Rename __glibc_unused1 to feature_1. [BZ #22563] * bits/types/__cancel_jmp_buf_tag.h: New file. * sysdeps/unix/sysv/linux/x86/bits/types/__cancel_jmp_buf_tag.h * sysdeps/unix/sysv/linux/x86/pthreaddef.h: Likewise. * sysdeps/unix/sysv/linux/x86/nptl/pthreadP.h: Likewise. * nptl/Makefile (headers): Add bits/types/__cancel_jmp_buf_tag.h. * nptl/descr.h [NEED_SAVED_MASK_IN_CANCEL_JMP_BUF] (pthread_unwind_buf): Add saved_mask to cancel_jmp_buf. * sysdeps/nptl/pthread.h: Include <bits/types/__cancel_jmp_buf_tag.h>. (__pthread_unwind_buf_t): Use struct __cancel_jmp_buf_tag with __cancel_jmp_buf. * sysdeps/unix/sysv/linux/hppa/pthread.h: Likewise. -----------------------------------------------------------------------
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "GNU C Library master sources". The branch, hjl/pr22743/master has been deleted was d139cd31adaf9f091a36027c8a452b016a05e95c - Log ----------------------------------------------------------------- d139cd31adaf9f091a36027c8a452b016a05e95c nptl: Update struct pthread_unwind_buf [BZ #22743] -----------------------------------------------------------------------