[Bug malloc/33886] Fastbin can grow indefinitely and never release

[roche at httrack dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=33886

Xavier Roche <roche at httrack dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |roche at httrack dot com

--- Comment #5 from Xavier Roche <roche at httrack dot com> ---
We hit the same issue in production (search engine, 128 GB servers, 48+ cores).
After upgrading glibc versions, RSS jumped from ~12 GB to 100+ GB.
malloc_info() showed ~137 GB of free memory across 96 arenas, not returned to
the OS. Periodic malloc_trim(0) recovers 30-120 GB each time.

Standalone reproducer:
https://github.com/xroche/glibc-malloc-hoarding-reproducer

The stress tester runs 16 threads, with 256 MB of persistent data interleaved
with 10 GB of short-lived allocations. 983 MB freed-but-retained in arenas, 961
MB recovered by malloc_trim(0). Docker-based, three commands to run, feedback
welcome!

There's also a tentative patch against glibc trunk in the repo's patch/
directory. It adds a tunable glibc.malloc.madvise_threshold that calls
madvise(MADV_DONTNEED) on the page-aligned interior of large consolidated free
chunks during free(), reusing the mtrim() alignment logic. (Disabled by
default)

With threshold=64K, the reproducer RSS drops from 1247 MB to 296 MB on 261 MB
of live data.

Blog post with more context:
https://www.algolia.com/blog/engineering/when-allocators-are-hoarding-your-precious-memory

Disclosure: AI tools were used during research, analysis, and patch drafting. I
reviewed and tested everything, but I might have missed important details. The
core patch logic is very small though.

-- 
You are receiving this mail because:
You are on the CC list for the bug.