Bug 24161 - __run_fork_handlers self-deadlocks in malloc/tst-mallocfork2
Summary: __run_fork_handlers self-deadlocks in malloc/tst-mallocfork2
Status: RESOLVED FIXED
Alias: None
Product: glibc
Classification: Unclassified
Component: nptl (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: 2.30
Assignee: Florian Weimer
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-02-04 06:48 UTC by Florian Weimer
Modified: 2019-02-08 12:14 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:
fweimer: security-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Weimer 2019-02-04 06:48:57 UTC
I get a hang like this occasionally:

(gdb) bt
#0  __lll_lock_wait_private ()
    at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:63
#1  0x00007f160c6f927a in __run_fork_handlers (who=(unknown: 209394016), 
    who@entry=atfork_run_prepare) at register-atfork.c:116
#2  0x00007f160c6b7897 in __libc_fork () at ../sysdeps/nptl/fork.c:58
#3  0x00000000004027d6 in sigusr1_handler (signo=<optimized out>)
    at tst-mallocfork2.c:80
#4  sigusr1_handler (signo=<optimized out>) at tst-mallocfork2.c:64
#5  <signal handler called>
#6  0x00007f160c6f92e4 in __run_fork_handlers (who=who@entry=atfork_run_parent)
    at register-atfork.c:136
#7  0x00007f160c6b79a2 in __libc_fork () at ../sysdeps/nptl/fork.c:152
#8  0x0000000000402567 in do_test () at tst-mallocfork2.c:156
#9  0x0000000000402dd2 in support_test_main (argc=1, argv=0x7ffc81ef1ab0, 
    config=config@entry=0x7ffc81ef1970) at support_test_main.c:350
#10 0x0000000000402362 in main (argc=<optimized out>, argv=<optimized out>)
    at ../support/test-driver.c:168

We need to extend the lock avoidance to atfork_lock.  Potentially introduced by:

commit 27761a1042daf01987e7d79636d0c41511c6df3c
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date:   Thu Feb 1 17:57:56 2018 -0200

    Refactor atfork handlers
Comment 1 Sourceware Commits 2019-02-08 11:47:07 UTC
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU C Library master sources".

The branch, master has been updated
       via  669ff911e2571f74a2668493e326ac9a505776bd (commit)
      from  d0bd87d4c0c11b93f79d1c3531ffb13b4b538190 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=669ff911e2571f74a2668493e326ac9a505776bd

commit 669ff911e2571f74a2668493e326ac9a505776bd
Author: Florian Weimer <fweimer@redhat.com>
Date:   Fri Feb 8 12:46:19 2019 +0100

    nptl: Avoid fork handler lock for async-signal-safe fork [BZ #24161]
    
    Commit 27761a1042daf01987e7d79636d0c41511c6df3c ("Refactor atfork
    handlers") introduced a lock, atfork_lock, around fork handler list
    accesses.  It turns out that this lock occasionally results in
    self-deadlocks in malloc/tst-mallocfork2:
    
    (gdb) bt
    #0  __lll_lock_wait_private ()
        at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:63
    #1  0x00007f160c6f927a in __run_fork_handlers (who=(unknown: 209394016),
        who@entry=atfork_run_prepare) at register-atfork.c:116
    #2  0x00007f160c6b7897 in __libc_fork () at ../sysdeps/nptl/fork.c:58
    #3  0x00000000004027d6 in sigusr1_handler (signo=<optimized out>)
        at tst-mallocfork2.c:80
    #4  sigusr1_handler (signo=<optimized out>) at tst-mallocfork2.c:64
    #5  <signal handler called>
    #6  0x00007f160c6f92e4 in __run_fork_handlers (who=who@entry=atfork_run_parent)
        at register-atfork.c:136
    #7  0x00007f160c6b79a2 in __libc_fork () at ../sysdeps/nptl/fork.c:152
    #8  0x0000000000402567 in do_test () at tst-mallocfork2.c:156
    #9  0x0000000000402dd2 in support_test_main (argc=1, argv=0x7ffc81ef1ab0,
        config=config@entry=0x7ffc81ef1970) at support_test_main.c:350
    #10 0x0000000000402362 in main (argc=<optimized out>, argv=<optimized out>)
        at ../support/test-driver.c:168
    
    If no locking happens in the single-threaded case (where fork is
    expected to be async-signal-safe), this deadlock is avoided.
    (pthread_atfork is not required to be async-signal-safe, so a fork
    call from a signal handler interrupting pthread_atfork is not
    a problem.)

-----------------------------------------------------------------------

Summary of changes:
 ChangeLog              |   10 ++++++++++
 nptl/register-atfork.c |    8 +++++---
 sysdeps/nptl/fork.c    |    6 +++---
 sysdeps/nptl/fork.h    |    8 +++++---
 4 files changed, 23 insertions(+), 9 deletions(-)
Comment 2 Florian Weimer 2019-02-08 11:49:05 UTC
Fixed in glibc 2.30.
Comment 3 Sourceware Commits 2019-02-08 12:12:54 UTC
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU C Library master sources".

The branch, release/2.28/master has been updated
       via  60f80624257ef84eacfd9b400bda1b5a5e8e7816 (commit)
      from  a9f60b1571cc7821cd6bc4dbeba12194d484e7f5 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=60f80624257ef84eacfd9b400bda1b5a5e8e7816

commit 60f80624257ef84eacfd9b400bda1b5a5e8e7816
Author: Florian Weimer <fweimer@redhat.com>
Date:   Fri Feb 8 12:55:21 2019 +0100

    nptl: Avoid fork handler lock for async-signal-safe fork [BZ #24161]
    
    Commit 27761a1042daf01987e7d79636d0c41511c6df3c ("Refactor atfork
    handlers") introduced a lock, atfork_lock, around fork handler list
    accesses.  It turns out that this lock occasionally results in
    self-deadlocks in malloc/tst-mallocfork2:
    
    (gdb) bt
    #0  __lll_lock_wait_private ()
        at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:63
    #1  0x00007f160c6f927a in __run_fork_handlers (who=(unknown: 209394016),
        who@entry=atfork_run_prepare) at register-atfork.c:116
    #2  0x00007f160c6b7897 in __libc_fork () at ../sysdeps/nptl/fork.c:58
    #3  0x00000000004027d6 in sigusr1_handler (signo=<optimized out>)
        at tst-mallocfork2.c:80
    #4  sigusr1_handler (signo=<optimized out>) at tst-mallocfork2.c:64
    #5  <signal handler called>
    #6  0x00007f160c6f92e4 in __run_fork_handlers (who=who@entry=atfork_run_parent)
        at register-atfork.c:136
    #7  0x00007f160c6b79a2 in __libc_fork () at ../sysdeps/nptl/fork.c:152
    #8  0x0000000000402567 in do_test () at tst-mallocfork2.c:156
    #9  0x0000000000402dd2 in support_test_main (argc=1, argv=0x7ffc81ef1ab0,
        config=config@entry=0x7ffc81ef1970) at support_test_main.c:350
    #10 0x0000000000402362 in main (argc=<optimized out>, argv=<optimized out>)
        at ../support/test-driver.c:168
    
    If no locking happens in the single-threaded case (where fork is
    expected to be async-signal-safe), this deadlock is avoided.
    (pthread_atfork is not required to be async-signal-safe, so a fork
    call from a signal handler interrupting pthread_atfork is not
    a problem.)
    
    (cherry picked from commit 669ff911e2571f74a2668493e326ac9a505776bd)

-----------------------------------------------------------------------

Summary of changes:
 ChangeLog              |   10 ++++++++++
 NEWS                   |    1 +
 nptl/register-atfork.c |    8 +++++---
 sysdeps/nptl/fork.c    |    6 +++---
 sysdeps/nptl/fork.h    |    8 +++++---
 5 files changed, 24 insertions(+), 9 deletions(-)
Comment 4 Sourceware Commits 2019-02-08 12:14:00 UTC
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU C Library master sources".

The branch, release/2.29/master has been updated
       via  c096b008d2671028c21ac8cf01f18a2083e73c44 (commit)
      from  44113a8ba24af23d7bbb174f9087a6b83a76289a (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c096b008d2671028c21ac8cf01f18a2083e73c44

commit c096b008d2671028c21ac8cf01f18a2083e73c44
Author: Florian Weimer <fweimer@redhat.com>
Date:   Fri Feb 8 12:54:41 2019 +0100

    nptl: Avoid fork handler lock for async-signal-safe fork [BZ #24161]
    
    Commit 27761a1042daf01987e7d79636d0c41511c6df3c ("Refactor atfork
    handlers") introduced a lock, atfork_lock, around fork handler list
    accesses.  It turns out that this lock occasionally results in
    self-deadlocks in malloc/tst-mallocfork2:
    
    (gdb) bt
    #0  __lll_lock_wait_private ()
        at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:63
    #1  0x00007f160c6f927a in __run_fork_handlers (who=(unknown: 209394016),
        who@entry=atfork_run_prepare) at register-atfork.c:116
    #2  0x00007f160c6b7897 in __libc_fork () at ../sysdeps/nptl/fork.c:58
    #3  0x00000000004027d6 in sigusr1_handler (signo=<optimized out>)
        at tst-mallocfork2.c:80
    #4  sigusr1_handler (signo=<optimized out>) at tst-mallocfork2.c:64
    #5  <signal handler called>
    #6  0x00007f160c6f92e4 in __run_fork_handlers (who=who@entry=atfork_run_parent)
        at register-atfork.c:136
    #7  0x00007f160c6b79a2 in __libc_fork () at ../sysdeps/nptl/fork.c:152
    #8  0x0000000000402567 in do_test () at tst-mallocfork2.c:156
    #9  0x0000000000402dd2 in support_test_main (argc=1, argv=0x7ffc81ef1ab0,
        config=config@entry=0x7ffc81ef1970) at support_test_main.c:350
    #10 0x0000000000402362 in main (argc=<optimized out>, argv=<optimized out>)
        at ../support/test-driver.c:168
    
    If no locking happens in the single-threaded case (where fork is
    expected to be async-signal-safe), this deadlock is avoided.
    (pthread_atfork is not required to be async-signal-safe, so a fork
    call from a signal handler interrupting pthread_atfork is not
    a problem.)
    
    (cherry picked from commit 669ff911e2571f74a2668493e326ac9a505776bd)

-----------------------------------------------------------------------

Summary of changes:
 ChangeLog              |   10 ++++++++++
 NEWS                   |    1 +
 nptl/register-atfork.c |    8 +++++---
 sysdeps/nptl/fork.c    |    6 +++---
 sysdeps/nptl/fork.h    |    8 +++++---
 5 files changed, 24 insertions(+), 9 deletions(-)