malloc: Improve documentation of malloc tunables

[Dimitri John Ledkov]

On Thu, 12 Mar 2026 at 18:03, Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
>
> Hi Dimitri,
>
> > It would be very interesting if glibc malloc allocator could somehow
> > sense if it is in a container or not (even if by checking the
> > container environment variable) and for example choosing to lower
> > MALLOC_ARENA_MAX from the 8 times count of observable cpus to a fixed
> > lower value.
> > I wish containers would universally hint the weight and total amount
> > of cpu shares dynamically adjusted for the number of deployed
> > containers - but I don't believe this is available today.
>
> So is the issue you are seeing due to a bad default of MALLOC_ARENA_MAX?

I am relaying different sets of feedback, from very different
deployments with very different setups / workload / hardware / etc;
from different customers / end-users.

The MALLOC_ARENA_MAX=2 helped migrating of glibc 2.35 to 2.43 on x86_64

> Lots of arenas implies malloc isn't trying to use arenas efficiently. I think that
> can occur if you have extreme numbers of threads - the current malloc design
> tries to give each thread its own arena (which seems like a bad idea).

Yes, in this deployment there is JVM with many threads; inside
containers; and without MALLOC_ARENA_MAX=2 the performance degraded by
30% (total throughput, but also number of total and concurrent
requests being able to service). And the characteristics were similar
- very dense container deployment, low memory to cpu ratio, and yes
each container has a single process with many threads.

>
> Setting it to 2 might well be better as a default. However it's not obvious why
> we ever need more than say 64 arenas even on systems with lots of cores.

Is it arenas per process times 8 times cpu cores? Cause it feels like
in a container which observes all of the CPUs - yet actually cannot
really use all of them (there is contention / overcommit)

>
> > I think the desire to change the default mostly stems from lack of
> > good levers for applying the default tuning; or lack of automatic
> > dynamic  tuning based on a given environment.
>
> The goal is to provide good results out of the box. I don't believe in forcing
> people to fine tune every feature in their system in order to get it to work
> properly.
>
> Cheers,
> Wilco

I offered to set MALLOC_ARENA_MAX=2 by default universally for that
deployment and it had mixed perception - as if it might impact a
single large container allocated to run by-itself on a node;
effectively privileged / bare-metal container full speed ahead
workload.

It does feel like uncapped linear 8 times CPU cores is excessive after
a high enough number of cores. Was 8 times CPU cores been set back in
the day before 192 CPU cores public cloud instances generally
available; which with hyperthreading may appear as 384 cores; and
times by 8 would be 3072 arenas => which feels like very excessive
amount; especially when in practice this host actually runs 500 pods.

I wonder if we can keep 8 times CPU count; but cap it at 32 arenas
maybe? Unfortunately I don't have benchmark data that I could easily
trigger to simulate and measure the impact of changing this.

-- 
Regards,

Dimitri.