[Bug libc/11261] malloc uses excessive memory for multi-threaded applications

["siddhesh at redhat dot com" <sourceware-bugzilla at sourceware dot org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=11261

Siddhesh Poyarekar <siddhesh at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |siddhesh at redhat dot com

--- Comment #17 from Siddhesh Poyarekar <siddhesh at redhat dot com> ---
(In reply to Ondrej Bilka from comment #16)
> > Therefore the solution to a program with lots of threads is to limit the arenas > as a trade-off for memory.
> 
> That is a bandaid not a solution. Still there is no memory returned to
> system when one first does allocations and then allocates auxiliary memory
> like

You have not understood the bug report.

> void *calculate ()
> {
>   void **ary = malloc (1000000 * sizeof (void *))
>   for (i = 0; i < 1000000; i++) ary[i] = malloc (100);
>   for (i = 0; i <  999999; i++) free (ary [i]);
>   return ary[999999];
> }

This is a different problem from the current bug report, which is about too
many arenas getting created resulting in excessive address space usage
 and the MALLOC_ARENA_* variables not working to limit them.  Memory holes not
being freed has nothing to do with it. 

> When one acknowledges a bug a solution is relatively simple. Add a flag
> UNMAPPED for chunks which means that all pages completely contained in chunk
> were zeroed by madvise(s, n, MADV_DONTNEED).
> 
> You keep track of memory used and system and when their ratio is bigger than
> two you make chunks starting from largest ones UNMAPPED to decrease system
> charge.
> 
> This deals with RSS problem. A virtual space usage could still be excesive
> but that is smaller problem.

The problem you've described is different and I'm sure there's a bug report
open for it too.  madvise is not sufficient to free up commit charge; there's a
mail thread on libc-alpha that discusses this problem that you can search for
and read up on.  I think vm.overcommit_memory is one of the keywords to look
for.

-- 
You are receiving this mail because:
You are on the CC list for the bug.