This is the mail archive of the
glibc-bugs@sourceware.org
mailing list for the glibc project.
[Bug malloc/19431] Deadlock between fflush, getdelim, and fork
- From: "cvs-commit at gcc dot gnu.org" <sourceware-bugzilla at sourceware dot org>
- To: glibc-bugs at sourceware dot org
- Date: Fri, 27 Jan 2017 15:32:42 +0000
- Subject: [Bug malloc/19431] Deadlock between fflush, getdelim, and fork
- Auto-submitted: auto-generated
- References: <bug-19431-131@http.sourceware.org/bugzilla/>
https://sourceware.org/bugzilla/show_bug.cgi?id=19431
--- Comment #9 from cvs-commit at gcc dot gnu.org <cvs-commit at gcc dot gnu.org> ---
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU C Library master sources".
The branch, linaro/2.21/master has been updated
via 1b248392fbadaa5a81ed220fdf2f7ec3b9939b9e (commit)
via cf161d39113047a81f969f277707b9d1679ecf69 (commit)
via a274119b0e6b4800163d8d1870995e597a55cb8b (commit)
via a18c25e0ee00bb37c724123d8b187050e9a367a0 (commit)
via 124ea50584d3d21fe4c2fe9383508efd0ac9010a (commit)
via 5a8692a64640bb648de6be9877dc072d728859bd (commit)
via 6a277a415c127f6ee9a70548c899a7ac536f4288 (commit)
via 9e1e8f68d59e33baa0eef9e8690d3cbaa5f67a24 (commit)
via dd08634b37f81f01441d842087b123d12b4003d4 (commit)
via 749c94d44c9aa063dfc7ad2b505c8e6d35e57345 (commit)
via b533edfeedf66e971b1b29affd8e2f1619987dc9 (commit)
via 4b31bd831ab205ff24ae3a2b8c4f8135c182c7d7 (commit)
via 50992336ac40625c363e1514890f92774c65054a (commit)
via 39ad1d12448adf0682ffb8b8ac146e99b68279bf (commit)
via 7f2efd6c269a2ab2eee80bffa5bcb2538aa89641 (commit)
from a68cafa11c500d8a49a3014c43c5152859d037ae (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=1b248392fbadaa5a81ed220fdf2f7ec3b9939b9e
commit 1b248392fbadaa5a81ed220fdf2f7ec3b9939b9e
Author: H.J. Lu <hjl.tools@gmail.com>
Date: Wed Jul 29 11:57:54 2015 -0700
Extend local PLT reference check
On x86, linker in binutils 2.26 and newer consolidates R_*_JUMP_SLOT with
R_*_GLOB_DAT relocation against the same symbol. This patch extends
local PLT reference check to support alternate relocations.
[BZ #18078]
* scripts/check-localplt.awk: Support alternate relocations.
* scripts/localplt.awk: Also check relocations in DT_RELA/DT_REL
sections.
* sysdeps/unix/sysv/linux/i386/localplt.data: Mark free and
malloc entries with + REL R_386_GLOB_DAT.
* sysdeps/x86_64/localplt.data: New file.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=cf161d39113047a81f969f277707b9d1679ecf69
commit cf161d39113047a81f969f277707b9d1679ecf69
Author: Alan Modra <amodra@gmail.com>
Date: Fri Feb 20 15:23:28 2015 +1030
Fix localplt test breakage with new readelf
Since 2014-11-24 binutils git commit bb4d2ac2, readelf has appended
the symbol version to symbols shown in reloc dumps.
[BZ #16512]
* scripts/localplt.awk: Strip off symbol version.
* NEWS: Mention bug fix.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=a274119b0e6b4800163d8d1870995e597a55cb8b
commit a274119b0e6b4800163d8d1870995e597a55cb8b
Author: Florian Weimer <fweimer@redhat.com>
Date: Fri Aug 26 22:40:27 2016 +0200
malloc: Simplify static malloc interposition [BZ #20432]
Existing interposed mallocs do not define the glibc-internal
fork callbacks (and they should not), so statically interposed
mallocs lead to link failures because the strong reference from
fork pulls in glibc's malloc, resulting in multiple definitions
of malloc-related symbols.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=a18c25e0ee00bb37c724123d8b187050e9a367a0
commit a18c25e0ee00bb37c724123d8b187050e9a367a0
Author: Florian Weimer <fweimer@redhat.com>
Date: Tue Aug 16 11:06:13 2016 +0200
nptl/tst-tls3-malloc: Force freeing of thread stacks
It turns out that due to the reduced stack size in tst-tls3 and the
(fixed) default stack cache size, allocated TLS variables are never
freed, so the test coverage for tst-tls3-malloc is less than complete.
This change increases the thread stack size for tst-tls3-malloc only,
to make sure thread stacks and TLS variables are freed.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=124ea50584d3d21fe4c2fe9383508efd0ac9010a
commit 124ea50584d3d21fe4c2fe9383508efd0ac9010a
Author: Florian Weimer <fweimer@redhat.com>
Date: Tue Aug 2 17:01:02 2016 +0200
malloc: Run tests without calling mallopt [BZ #19469]
The compiled tests no longer refer to the mallopt symbol
from their main functions. (Some tests still call mallopt
explicitly, which is fine.)
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=5a8692a64640bb648de6be9877dc072d728859bd
commit 5a8692a64640bb648de6be9877dc072d728859bd
Author: Florian Weimer <fweimer@redhat.com>
Date: Wed Aug 3 16:16:57 2016 +0200
elf: Do not use memalign for TCB/TLS blocks allocation [BZ #17730]
Instead, call malloc and explicitly align the pointer.
There is no external location to store the original (unaligned)
pointer, and this commit increases the allocation size to store
the pointer at a fixed location relative to the TCB pointer.
The manual alignment means that some space goes unused which
was previously made available for subsequent allocations.
However, in the TLS_DTV_AT_TP case, the manual alignment code
avoids aligning the pre-TCB to the TLS block alignment. (Even
while using memalign, the allocation had some unused padding
in front.)
This concludes the removal of memalign calls from the TLS code,
and the new tst-tls3-malloc test verifies that only core malloc
routines are used.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=6a277a415c127f6ee9a70548c899a7ac536f4288
commit 6a277a415c127f6ee9a70548c899a7ac536f4288
Author: Florian Weimer <fweimer@redhat.com>
Date: Wed Aug 3 16:15:38 2016 +0200
elf: Avoid using memalign for TLS allocations [BZ #17730]
Instead of a flag which indicates the pointer can be freed, dtv_t
now includes the pointer which should be freed. Due to padding,
the size of dtv_t does not increase.
To avoid using memalign, the new allocate_dtv_entry function
allocates a sufficiently large buffer so that a sub-buffer
can be found in it which starts with an aligned pointer. Both
the aligned and original pointers are kept, the latter for calling
free later.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=9e1e8f68d59e33baa0eef9e8690d3cbaa5f67a24
commit 9e1e8f68d59e33baa0eef9e8690d3cbaa5f67a24
Author: Alexandre Oliva <aoliva@redhat.com>
Date: Tue Mar 17 01:14:11 2015 -0300
Fix DTV race, assert, DTV_SURPLUS Static TLS limit, and nptl_db garbage
for ChangeLog
[BZ #17090]
[BZ #17620]
[BZ #17621]
[BZ #17628]
* NEWS: Update.
* elf/dl-tls.c (_dl_update_slotinfo): Clean up outdated DTV
entries with Static TLS too. Skip entries past the end of the
allocated DTV, from Alan Modra.
(tls_get_addr_tail): Update to glibc_likely/unlikely. Move
Static TLS DTV entry set up from...
(_dl_allocate_tls_init): ... here (fix modid assertion), ...
* elf/dl-reloc.c (_dl_nothread_init_static_tls): ... here...
* nptl/allocatestack.c (init_one_static_tls): ... and here...
* elf/dlopen.c (dl_open_worker): Drop l_tls_modid upper bound
for Static TLS.
* elf/tlsdeschtab.h (map_generation): Return size_t. Check
that the slot we find is associated with the given map before
using its generation count.
* nptl_db/db_info.c: Include ldsodefs.h.
(rtld_global, dtv_slotinfo_list, dtv_slotinfo): New typedefs.
* nptl_db/structs.def (DB_RTLD_VARIABLE): New macro.
(DB_MAIN_VARIABLE, DB_RTLD_GLOBAL_FIELD): Likewise.
(link_map::l_tls_offset): New struct field.
(dtv_t::counter): Likewise.
(rtld_global): New struct.
(_rtld_global): New rtld variable.
(dl_tls_dtv_slotinfo_list): New rtld global field.
(dtv_slotinfo_list): New struct.
(dtv_slotinfo): Likewise.
* nptl_db/td_symbol_list.c: Drop gnu/lib-names.h include.
(td_lookup): Rename to...
(td_mod_lookup): ... this. Use new mod parameter instead of
LIBPTHREAD_SO.
* nptl_db/td_thr_tlsbase.c: Include link.h.
(dtv_slotinfo_list, dtv_slotinfo): New functions.
(td_thr_tlsbase): Check DTV generation. Compute Static TLS
addresses even if the DTV is out of date or missing them.
* nptl_db/fetch-value.c (_td_locate_field): Do not refuse to
index zero-length arrays.
* nptl_db/thread_dbP.h: Include gnu/lib-names.h.
(td_lookup): Make it a macro implemented in terms of...
(td_mod_lookup): ... this declaration.
* nptl_db/db-symbols.awk (DB_RTLD_VARIABLE): Override.
(DB_MAIN_VARIABLE): Likewise.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=dd08634b37f81f01441d842087b123d12b4003d4
commit dd08634b37f81f01441d842087b123d12b4003d4
Author: Florian Weimer <fweimer@redhat.com>
Date: Mon Jun 20 14:31:40 2016 +0200
elf: Consolidate machine-agnostic DTV definitions in <dl-dtv.h>
Identical definitions of dtv_t and TLS_DTV_UNALLOCATED were
repeated for all architectures using DTVs.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=749c94d44c9aa063dfc7ad2b505c8e6d35e57345
commit 749c94d44c9aa063dfc7ad2b505c8e6d35e57345
Author: Florian Weimer <fweimer@redhat.com>
Date: Thu Apr 14 09:17:02 2016 +0200
malloc: Run fork handler as late as possible [BZ #19431]
Previously, a thread M invoking fork would acquire locks in this order:
(M1) malloc arena locks (in the registered fork handler)
(M2) libio list lock
A thread F invoking flush (NULL) would acquire locks in this order:
(F1) libio list lock
(F2) individual _IO_FILE locks
A thread G running getdelim would use this order:
(G1) _IO_FILE lock
(G2) malloc arena lock
After executing (M1), (F1), (G1), none of the threads can make progress.
This commit changes the fork lock order to:
(M'1) libio list lock
(M'2) malloc arena locks
It explicitly encodes the lock order in the implementations of fork,
and does not rely on the registration order, thus avoiding the deadlock.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=b533edfeedf66e971b1b29affd8e2f1619987dc9
commit b533edfeedf66e971b1b29affd8e2f1619987dc9
Author: Florian Weimer <fweimer@redhat.com>
Date: Mon Dec 21 16:42:46 2015 +0100
malloc: Fix list_lock/arena lock deadlock [BZ #19182]
* malloc/arena.c (list_lock): Document lock ordering requirements.
(free_list_lock): New lock.
(ptmalloc_lock_all): Comment on free_list_lock.
(ptmalloc_unlock_all2): Reinitialize free_list_lock.
(detach_arena): Update comment. free_list_lock is now needed.
(_int_new_arena): Use free_list_lock around detach_arena call.
Acquire arena lock after list_lock. Add comment, including FIXME
about incorrect synchronization.
(get_free_list): Switch to free_list_lock.
(reused_arena): Acquire free_list_lock around detach_arena call
and attached threads counter update. Add two FIXMEs about
incorrect synchronization.
(arena_thread_freeres): Switch to free_list_lock.
* malloc/malloc.c (struct malloc_state): Update comments to
mention free_list_lock.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=4b31bd831ab205ff24ae3a2b8c4f8135c182c7d7
commit 4b31bd831ab205ff24ae3a2b8c4f8135c182c7d7
Author: Florian Weimer <fweimer@redhat.com>
Date: Tue Nov 24 16:37:15 2015 +0100
Replace MUTEX_INITIALIZER with _LIBC_LOCK_INITIALIZER in generic code
* sysdeps/mach/hurd/libc-lock.h (_LIBC_LOCK_INITIALIZER): Define.
(__libc_lock_define_initialized): Use it.
* sysdeps/nptl/libc-lockP.h (_LIBC_LOCK_INITIALIZER): Define.
* malloc/arena.c (list_lock): Use _LIBC_LOCK_INITIALIZER.
* malloc/malloc.c (main_arena): Likewise.
* sysdeps/generic/malloc-machine.h (MUTEX_INITIALIZER): Remove.
* sysdeps/nptl/malloc-machine.h (MUTEX_INITIALIZER): Remove.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=50992336ac40625c363e1514890f92774c65054a
commit 50992336ac40625c363e1514890f92774c65054a
Author: Florian Weimer <fweimer@redhat.com>
Date: Wed Oct 28 19:32:46 2015 +0100
malloc: Prevent arena free_list from turning cyclic [BZ #19048]
[BZ# 19048]
* malloc/malloc.c (struct malloc_state): Update comment. Add
attached_threads member.
(main_arena): Initialize attached_threads.
* malloc/arena.c (list_lock): Update comment.
(ptmalloc_lock_all, ptmalloc_unlock_all): Likewise.
(ptmalloc_unlock_all2): Reinitialize arena reference counts.
(deattach_arena): New function.
(_int_new_arena): Initialize arena reference count and deattach
replaced arena.
(get_free_list, reused_arena): Update reference count and deattach
replaced arena.
(arena_thread_freeres): Update arena reference count and only put
unreferenced arenas on the free list.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=39ad1d12448adf0682ffb8b8ac146e99b68279bf
commit 39ad1d12448adf0682ffb8b8ac146e99b68279bf
Author: Florian Weimer <fweimer@redhat.com>
Date: Sat Oct 17 12:06:48 2015 +0200
malloc: Rewrite with explicit TLS access using __thread
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=7f2efd6c269a2ab2eee80bffa5bcb2538aa89641
commit 7f2efd6c269a2ab2eee80bffa5bcb2538aa89641
Author: Siddhesh Poyarekar <siddhesh@redhat.com>
Date: Wed Feb 18 11:06:05 2015 +0530
Consolidate arena_lookup and arena_lock into a single arena_get
This seems to have been left behind as an artifact of some old changes
and can now be merged. Verified that the only generated code change
on x86_64 is that of line numbers in asserts, like so:
@@ -27253,7 +27253,7 @@ Disassembly of section .text:
416f09: 48 89 42 20 mov %rax,0x20(%rdx)
416f0d: e9 7e f6 ff ff jmpq 416590 <_int_free+0x230>
416f12: b9 3f 9f 4a 00 mov $0x4a9f3f,%ecx
- 416f17: ba d5 0f 00 00 mov $0xfd5,%edx
+ 416f17: ba d6 0f 00 00 mov $0xfd6,%edx
416f1c: be a8 9b 4a 00 mov $0x4a9ba8,%esi
416f21: bf 6a 9c 4a 00 mov $0x4a9c6a,%edi
416f26: e8 45 e8 ff ff callq 415770 <__malloc_assert>
-----------------------------------------------------------------------
Summary of changes:
ChangeLog | 298 ++++++++++++++++++++++++++++
NEWS | 4 +-
csu/libc-tls.c | 2 +-
elf/dl-open.c | 12 +-
elf/dl-reloc.c | 6 -
elf/dl-tls.c | 209 ++++++++++++--------
elf/tlsdeschtab.h | 4 +-
include/libc-symbols.h | 15 ++
malloc/Makefile | 33 +++-
malloc/arena.c | 211 +++++++++++++-------
malloc/malloc-internal.h | 32 +++
malloc/malloc.c | 20 ++-
malloc/tst-interpose-aux-nothread.c | 20 ++
malloc/tst-interpose-aux-thread.c | 20 ++
malloc/tst-interpose-aux.c | 270 +++++++++++++++++++++++++
malloc/tst-interpose-aux.h | 30 +++
malloc/tst-interpose-nothread.c | 20 ++
malloc/tst-interpose-skeleton.c | 210 ++++++++++++++++++++
malloc/tst-interpose-static-nothread.c | 19 ++
malloc/tst-interpose-static-thread.c | 19 ++
malloc/tst-interpose-thread.c | 20 ++
malloc/tst-malloc-fork-deadlock.c | 220 ++++++++++++++++++++
manual/memory.texi | 12 --
nptl/Makefile | 8 +-
nptl/allocatestack.c | 13 +-
nptl/tst-tls3-malloc.c | 31 +++
nptl/tst-tls3.c | 10 +-
nptl_db/db-symbols.awk | 2 +
nptl_db/db_info.c | 4 +
nptl_db/fetch-value.c | 3 +-
nptl_db/structs.def | 39 ++++
nptl_db/td_symbol_list.c | 7 +-
nptl_db/td_thr_tlsbase.c | 172 ++++++++++++++++-
nptl_db/thread_dbP.h | 11 +-
scripts/check-localplt.awk | 40 ++++-
scripts/localplt.awk | 51 +++++-
sysdeps/aarch64/dl-tls.h | 3 -
sysdeps/aarch64/nptl/tls.h | 12 +-
sysdeps/alpha/dl-tls.h | 3 -
sysdeps/alpha/nptl/tls.h | 12 +-
sysdeps/arm/dl-tls.h | 3 -
sysdeps/arm/nptl/tls.h | 12 +-
sysdeps/generic/dl-dtv.h | 38 ++++
sysdeps/generic/malloc-machine.h | 8 -
sysdeps/hppa/dl-tls.h | 3 -
sysdeps/hppa/nptl/tls.h | 12 +-
sysdeps/i386/dl-tls.h | 3 -
sysdeps/i386/nptl/tls.h | 14 +--
sysdeps/ia64/dl-tls.h | 3 -
sysdeps/ia64/nptl/tls.h | 14 +--
sysdeps/m68k/dl-tls.h | 3 -
sysdeps/m68k/nptl/tls.h | 12 +-
sysdeps/mach/hurd/bits/libc-lock.h | 3 +-
sysdeps/mach/hurd/fork.c | 13 ++
sysdeps/mach/hurd/i386/tls.h | 12 +-
sysdeps/mach/hurd/malloc-machine.h | 10 -
sysdeps/microblaze/dl-tls.h | 3 -
sysdeps/microblaze/nptl/tls.h | 12 +-
sysdeps/mips/dl-tls.h | 3 -
sysdeps/mips/nptl/tls.h | 13 +-
sysdeps/nios2/dl-tls.h | 3 -
sysdeps/nios2/nptl/tls.h | 12 +-
sysdeps/nptl/bits/libc-lockP.h | 1 +
sysdeps/nptl/fork.c | 14 ++-
sysdeps/nptl/malloc-machine.h | 11 -
sysdeps/powerpc/dl-tls.h | 3 -
sysdeps/powerpc/nptl/tls.h | 12 +-
sysdeps/s390/dl-tls.h | 3 -
sysdeps/s390/nptl/tls.h | 14 +--
sysdeps/sh/dl-tls.h | 3 -
sysdeps/sh/nptl/tls.h | 12 +-
sysdeps/sparc/dl-tls.h | 3 -
sysdeps/sparc/nptl/tls.h | 12 +-
sysdeps/tile/dl-tls.h | 3 -
sysdeps/tile/nptl/tls.h | 12 +-
sysdeps/unix/sysv/linux/i386/localplt.data | 8 +-
sysdeps/x86_64/dl-tls.h | 3 -
sysdeps/x86_64/localplt.data | 19 ++
sysdeps/x86_64/nptl/tls.h | 13 +--
test-skeleton.c | 2 +
80 files changed, 1979 insertions(+), 505 deletions(-)
create mode 100644 malloc/malloc-internal.h
create mode 100644 malloc/tst-interpose-aux-nothread.c
create mode 100644 malloc/tst-interpose-aux-thread.c
create mode 100644 malloc/tst-interpose-aux.c
create mode 100644 malloc/tst-interpose-aux.h
create mode 100644 malloc/tst-interpose-nothread.c
create mode 100644 malloc/tst-interpose-skeleton.c
create mode 100644 malloc/tst-interpose-static-nothread.c
create mode 100644 malloc/tst-interpose-static-thread.c
create mode 100644 malloc/tst-interpose-thread.c
create mode 100644 malloc/tst-malloc-fork-deadlock.c
create mode 100644 nptl/tst-tls3-malloc.c
create mode 100644 sysdeps/generic/dl-dtv.h
create mode 100644 sysdeps/x86_64/localplt.data
--
You are receiving this mail because:
You are on the CC list for the bug.