[PATCH] malloc: Prevent arena free_list from turning cyclic [BZ #19048]

[Florian Weimer]

This is a preview of a fix for bug 19048.  As explained in the
description of that bug, the problem is that a certain sequence of
events causes free_list to turn cyclic.  If this happens, the
reused_arena fallback code is never used because get_free_list will
never return NULL again.  If the cycle in free_list is short, this can
lead to significant contention, and we will stop using arenas not on the
free list or already attached to a thread.

This patch introduces a counter to keep track to how many threads are
attached to an arena, and put it on the free list on thread exit if and
only if the exiting thread was the last one to use the arena.

There is a ??? marker on deattach_arena.  I did not want to change the
arena management algorithm in a fundamental way, but the reference count
can reach zero here, and we could put the arena on the free_list.  This
can happen if the other thread that caused contention for the current
thread (so that we switched arenas) exited.  This may not even be that
unlikely if a thread frees a lot of memory before exit.

Regarding algorithm changes: If applications create and destroy threads
continuously, it seems likely they will hit this bug.  After this fix
(with or with the deattach_arena enhancement mentioned above), the arena
selection behavior will be *very* different.  This is unavoidable, but
it is unclear if it will result in a performance gain across the board.

Regarding synchronization, I tried to make sure that all updates to the
attached_threads member happen under list_lock.  (There are some
existing data races, but that's a different issue.)  Only in
reused_arena, I had to take list_lock briefly to maintain the reference
count, but that's clearly not on the fast path.  In the other places, I
could reuse the existing list_lock synchronization.

Florian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-malloc-Prevent-arena-free_list-from-turning-cyclic-B.patch
Type: text/x-patch
Size: 6956 bytes
Desc: not available
URL: <http://sourceware.org/pipermail/libc-alpha/attachments/20151009/3169dddc/attachment.bin>