| Summary: | glibc fails to free memory | ||
|---|---|---|---|
| Product: | glibc | Reporter: | Hi-Angel <hi-angel> |
| Component: | malloc | Assignee: | Not yet assigned to anyone <unassigned> |
| Status: | RESOLVED NOTABUG | ||
| Severity: | normal | CC: | carlos |
| Priority: | P2 | ||
| Version: | 2.32 | ||
| Target Milestone: | --- | ||
| See Also: | https://sourceware.org/bugzilla/show_bug.cgi?id=14581 | ||
| Host: | Target: | ||
| Build: | Last reconfirmed: | ||
To add to the list of affected apps: Ruby¹, glib². And yeah, calling `malloc_trim(0);` next to each `free()` resolves the problem, but it is merely a workaround. Because it is glibc-specific function and one can't reasonably expect every developer of each existing app to be able to write code for each existing libc-implementation. Glibc `free()` should call `malloc_trime()` themselves whenever they deem appropriate. 1: https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html#investigating-fragmentation-at-the-memory-allocator-level 2: https://gitlab.gnome.org/GNOME/glib/-/issues/2057 (In reply to Hi-Angel from comment #1) > To add to the list of affected apps: Ruby¹, glib². > > And yeah, calling `malloc_trim(0);` next to each `free()` resolves the > problem, but it is merely a workaround. Because it is glibc-specific > function and one can't reasonably expect every developer of each existing > app to be able to write code for each existing libc-implementation. Glibc > `free()` should call `malloc_trime()` themselves whenever they deem > appropriate. > > 1: > https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory- > bloat.html#investigating-fragmentation-at-the-memory-allocator-level > 2: https://gitlab.gnome.org/GNOME/glib/-/issues/2057 The ruby case is a known issue with aligned allocations. https://sourceware.org/bugzilla/show_bug.cgi?id=14581 We have a patch to improve the ruby case significantly, but it needs more work. The gnome case is unrelated and is just about freeing back unused pages under memory pressure which is exactly what malloc_trim() will do when called. This particular issue is not a bug, the allocator is under no requirement to free back the 50MiB to the OS if it believes that the memory will be used again and that performance is improved by caching it in userspace. |
This is suspected to be the reason for this bugreport¹ and was discussed in glibc ML here². # Steps to reproduce (in terms of terminal commands) λ cat test.cpp // credits to https://stackoverflow.com/a/48652734/2388257#glibc-application-holding-onto-unused-memory-until-just-before-exit #include <cstdlib> #include <cstdio> #include <unistd.h> int main() { static const int N = 50000; static void *arr[N]; for (int i = 0; i < N; i++) arr[i] = std::malloc(1024); // reverse to simplify allocators job for (int i = N - 1; i >= 0; i--) std::free(arr[i]); puts("Measure"); sleep(1e5); } λ g++ test.cpp -o a λ ./a & [1] 647847 Measure λ smem -kc "name pid pss" | grep -P "\ba\b" a 647847 50.4M ## Expected Memory doesn't take up 50M because these 50M were freed in the point of measure ## Actual Memory takes up 50M despite that memory have been freed. # Links 1: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=45200 2: https://sourceware.org/pipermail/libc-help/2020-December/thread.html#005592