[PATCH 0/2] [RFC] malloc: tcache improvements

[Cupertino Miranda]

Hi Wilco,

On 06-12-2024 19:01, Wilco Dijkstra wrote:
> Hi Cupertino,
> 
>> As expected, the amount of VM allocated by glibc can be reduced to
>> practical levels by configuring it to use only one arena (arena_max=1),
>> however increasing thread contention and killing performance.
> 
> Did you try other options besides the two extremes of arena_max=1 and
> arena_max=8*nr_hw_threads?
Yes, even with arena_max=2, RSS would keep growing. The lower the 
arena_max, the slower, but still fast enough. :(
I tried with other options as well, but without any relevant effect.

I also tried a much much bigger arena_max then default (4000 in 2000 
threads example), just considering that RSS issue was related to 
concurrency problems in glibc and that without the threads racing, RSS 
would stabilize.
That was not the case and RSS kept growing even faster.
> 
> This subject has come up before - it seems to me 8 times the number of
> cores is way overkill. That's 2048 arenas if you have 256 cores!
Experiments were with 32 cores.
> 
> In your tests, how many arenas do you actually end up with? Would you
> see improvements if say we limited arenas to say 1 arena per core or even
> less for higher core counts? Or do having 2 arenas already result in a huge
> increase in RSS?
Two arenas already results in a continuous grow of RSS.
> 
> As for tcache improvements, would it be sufficient to increase the minimum
> tcache size? Eg. doubling the array would be relatively simple and low cost.
Doubling the array would limit at the double. It simply does not scale 
properly. IMHO, that is really not enough. It would still not allow any 
proper tuning of the caches.
We really want to be able to cache "any" size chunk. I honestly think 
that we need to be able to allow caches to collect larger contiguous 
amounts of data and allow the cache to manage it, i.e. split and merge 
chunks without having to lock.
This would also be good in per-core allocation strategies.
> 
> My worry with extending the complexity of tcache is that small blocks will
> now get an extra penalty. 
I think the only overhead introduced to original tcache was a condition 
on the chunk size to make sure what caching mechanism it is dealing with 
(small or large). Otherwise performance is the same.

Now that I think about it, if we drop the 2 dimensional arrays from 
tcache implementation, we can make both implementation the same without 
that check overhead. Placing or grabbing chunks from those bins would 
just be a simple add/remove from tail/head of the linked-list, 
respectively. I don't think that would be much of a performance difference.
We can keep the counts per bin to preserve the semantics of previous 
implementation.

> So at some point someone will propose to add
> yet another cache layer to speed up small blocks again! Then we have fast
> tcache, large tcache, fastbins, small bins, large bins, unsorted bins and mmap...
> 
> Another possibility is to make the locking scheme more finegrained. The
> consolidation approach used currently requires taking malloc locks for very
> long periods - and that is the underlying cause for the concurrency issues.
I don't know precisely what are the concurrency issues you refer to. 
However, I tend to believe that, if you have finer grain locking you 
will have a higher chance to miss a lock in an essential location.

Also, I think if we would focus on fine grain tuning we will likely just 
tight the implementation to the existing architecture and lose the 
opportunity to make it a more configurable solution and address the 
requirements of these applications (mysql, jvm, etc.).
However that is likely unrelated to any tcache work.

Cheers,
Cupertino