[PATCH 0/2] [RFC] malloc: tcache improvements

[Cupertino Miranda]

Hi Carlos,

On 02-12-2024 21:14, Carlos O'Donell wrote:
> On 12/2/24 2:42 PM, Cupertino Miranda wrote:
>> In this patch series I present some malloc improvements to tcache
>> support, which I kindly request for comments.
> 
> Which workloads did you benchmark this with?

In the past few months I have been experimenting with mysql community 
edition in collaboration with local experts in the tool.
Ultimate goal is to bring glibc malloc back to tools like mysql.

> Can you share that benchmark data?

The attached charts below compare VmSIZE, RSS and TPC (performance) from 
default max_arena=8*32 (8*nr_cpus) (results on the left) to max_arena=1 
(results on the right).

The data is collected from a run of mysql server which allocates around 
32G of data. The remaining of the threads created more then around 2000 
threads, will reserve the rest.

<attached charts named *arena.png>

These charts shown the constant growth of RSS as well as the double 
allocation of VM.
With max_arena=1 (on the right) one can see that we get a much better VM 
and RSS result, being able to get an RSS stabilization.
On the performance side, arena_max=1 would give you almost half the 
performance (TPS).

After extending tcaches to allow bigger chunks, we were able to tune 
glibc to recover the performance loss of using arena_max=1.

<charts named large_tcache*.png>

> Have you looked at the state of the art in other allocators?

Yes, we have looked to tcmalloc which is what is currently in use by mysql.

> How large of a per-thread cache does jemalloc or tcmalloc permit?

Extracted from 
https://google.github.io/tcmalloc/design.html#tcmalloc-page-sizes

"Objects of size greater than the limit defined by kMaxSize are 
allocated directly from the backend. As such they are not cached in 
either the front or middle ends. Allocation requests for large object 
sizes are rounded up to the TCMalloc page size."

Depends on page size configuration, but max I seen is 256kB.

In any case, architecturally there is no limit.

>> Although it does perform well, it results in strange higher memory
>> usage when comparing with other malloc implementations (example
>> tcmalloc).
> 
> Have you root caused any of the RSS usage in the cases you've seen?

In order to have an RSS increase, the server needs to be running for 3 
days with heavy workload.

We know there are no memory leaks from application side.
Also there are no fragmentation problems that would not allow RSS growth 
not to stabilize (judging by the arena_max=1 stabilized results).
Also tried with an over-excessively high arena_max=(2*nr_threads), to 
check for possible concurrency race related problems on allocation (RSS 
kept growing faster)

Considering the high number of threads, our perception is that RSS 
growth is just an effect of existing multi-arena/heap architecture, 
which allocates much more VM. We think that eventually RSS would 
stabilize, however it would be much higher then other allocators which 
would be closer to RSS in arena_max=1.

> Do you have a heap state dumping utility to analyze the usage?

No. I have no idea how to analyse the heap in such a big example.

>> Last but not least, I wonder if there is any glibc official benchmark
>> for malloc. If not, what would you recommend to evaluate performance
>> and memory footprint.
> 
> The simplest official benchmark is 'make bench' which has a single threaded and
> multi-threaded benchmark, but these are not indicative real-world workloads.
> 
> Please look at the work DJ did during the original tcache development.
> https://sourceware.org/legacy-ml/libc-alpha/2017-01/msg00524.html

Thanks. Please notice my patch does not replace his work.

I am mostly looking for comments and perhaps collaboration to extend 
glibc malloc to allow it to support both existing architecture 
(centralized with multiple arenas/heaps) but also a more layered 
architecture (with single heap and multiple concurrency levels).
We believe both solutions would be possible without performance overhead 
in one another.

> Please review these tools for capturing traces:
> https://pagure.io/glibc-malloc-trace-utils
> 

Regards,
Cupertino
-------------- next part --------------
A non-text attachment was scrubbed...
Name: vmsize_max_arena.png
Type: image/png
Size: 157459 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20241205/3183d730/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rss_max_arena.png
Type: image/png
Size: 147781 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20241205/3183d730/attachment-0007.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tps_max_arena.png
Type: image/png
Size: 254160 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20241205/3183d730/attachment-0008.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: large_tcache_tps.png
Type: image/png
Size: 216990 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20241205/3183d730/attachment-0009.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: large_tcache_rss.png
Type: image/png
Size: 151079 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20241205/3183d730/attachment-0010.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: large_tcache_vmsize.png
Type: image/png
Size: 186421 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20241205/3183d730/attachment-0011.png>