[PATCH 0/2] [RFC] malloc: tcache improvements

[Cupertino Miranda]

Hi everyone,

In this patch series I present some malloc improvements to tcache
support, which I kindly request for comments.

GLibC malloc is known for not scaling well for many thread applications which
allocate large amounts of data.
Although it does perform well, it results in strange higher memory
usage when comparing with other malloc implementations (example
tcmalloc).

https://sourceware.org/bugzilla/show_bug.cgi?id=11261
https://blog.cloudflare.com/the-effect-of-switching-to-tcmalloc-on-rocksdb-memory-use/
https://poonamparhar.github.io/glibc_fragmentation_work/

Within local analysis taken in production environment with mainstream
server applications, we have identified that glibc tends to allocate
much more virtual memory when in multi-threaded scenarios, and this extra
virtual memory allocation seems to be correlated with some excessive
usage of memory (RSS). The excessive use of memory with time results in
out-of-memory (OOM) situations and server application restarts.
Solution for these server applications, so far, is to use tcmalloc
allocator which does produce more stable runs.

As expected, the amount of VM allocated by glibc can be reduced to
practical levels by configuring it to use only one arena (arena_max=1),
however increasing thread contention and killing performance.

The presented patch is a first step to bring back concurrency, to scenarios
where max_arena=1 would provide better memory usage.
The solution was to improve concurrency by configurally allowing tcache
implementation to cache larger data sized chunks.
Please notice that the presented solution should bring no impact to
existing implementation of glibc malloc, unless enabled through tunable
manipulation.

The caching of large chunks might have a small impact on memory
, as these larger chunks will be kept back for the thread, up
until a specified thread tcache size of chunks is reached. Once reached
the older chunks are freed to the arena. However it would likely still
reserve less memory than the default settings (arena_max=8*nr_hw_threads).

Although possibly having impact in short term memory consumption, this
improvement brought a huge benefit in performance, when the arenas are
no longer playing a role in concurrency (arena_max=1).

Future patches will likely include pre-loading of caches and tcache
thread spliting/coalescing of chunks.
Last but not least is the per-core caching, which has already been
discussed and presented in GNU Cauldron 2024.

The glibc testsuite was also adapted to perform testing for the changed
tcache support.
Although everything appears to be working correctly, in practical usage,
there are some testsuite regressions, which at the moment I did not
check if they are real failures or just side effects.

Last but not least, I wonder if there is any glibc official benchmark
for malloc. If not, what would you recommend to evaluate performance
and memory footprint.

Looking forward to all your comments.

Best regards,
Cupertino

Cupertino Miranda (2):
  malloc: add tcache support for large chunk caching
  malloc: add testing for large tcache support.

 Rules                |  19 +++
 elf/dl-tunables.list |   3 +
 malloc/Makefile      |  11 ++
 malloc/arena.c       |   2 +
 malloc/malloc.c      | 383 ++++++++++++++++++++++++++++++++++++-------
 5 files changed, 360 insertions(+), 58 deletions(-)

-- 
2.39.5