[Bug malloc/26969] A common malloc pattern can make memory not given back to OS

[dimahabr at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26969

Dmitry <dimahabr at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dimahabr at gmail dot com

--- Comment #3 from Dmitry <dimahabr at gmail dot com> ---
This is a really common problem especially for long-running multithreaded
processes with a lot of arenas. Calling malloc_trim - not always an option, if
a service using C bindings and is written on high level languages. Also,
malloc_trim would cause unnecessary overhead, since it will be trimming and
locking all arenas. Additional confusion comes from man page for malloc_trim
which says it's called sometimes during free.

I was looking at the code and have the following suggestion for improvement:
the main idea is to call mtrim for arena in _int_free. To amortize performance
overhead: call it only if free is called for a chunk with size > than
FASTBIN_CONSOLIDATION_THRESHOLD, and we have a chance to free more memory than
let's say 3*TRIM_THRESHOLD. To understand how much we can return to OS we can
add 1 bit flag to chunk, set it to 0 after returning to OS, set to 1 otherwise.
This would not give 100% accurate result, but should give a good estimate.

I'd be glad to work on this functionality, if you think it makes sense. If you
have other suggestions also happy to discuss this. The main point - mtrim
should be called during free some times, otherwise current malloc is
unfortunately hard to use for long-running multithreaded workload, since it's
using 2-3x more RSS in comparison with jemalloc or tcmalloc.

-- 
You are receiving this mail because:
You are on the CC list for the bug.