malloc: Improve documentation of malloc tunables

[Dimitri John Ledkov]

On Wed, 11 Mar 2026 at 14:07, Adhemerval Zanella Netto
<adhemerval.zanella@linaro.org> wrote:
>
>
>
> On 10/03/26 14:03, Wilco Dijkstra wrote:
> > Hi Adhemerval,
> >
> > On 10/03/26 12:56, Wilco Dijkstra wrote:
> >>
> >> Update default for tcache_count tunable.  Remove existing documentation and
> >> mention removal of fastbins in mxfast tunable.  Improve wording of hugetlb
> >> tunable, including default for AArch64.
> >>
> >> OK for commit?
> >
> >> It does not address the value returned by --list-tunables, which can be
> >> misleading; nor the issues about memory consumption raised by Dimitri.
> >
> > I'm testing a patch that shows it in --list-tunables, I'll post it when I get some
> > results back.
> >
> >> I really think we should be more conservative and *not* make this default
> >> and only enable through the tunable.  With system-wide tunables we will
> >> have an option to enable this as side-wide if user do want it.
> >
> > We have been super conservative for way too long already. It's around 15 years
> > since it was enabled by default for distros and 4 years since the hugetlb tunable
> > was added to GLIBC. How many applications actually set glibc.malloc.hugetlb?
> >
> > https://codesearch.debian.net/search?q=glibc.malloc.hugetlb&literal=1&perpkg=1
> >
> > So the question is, how many more years should we wait before developers
> > figure out how to use the tunables?
> >
> > Or we can use the distro settings and use THP when enabled. People can
> > disable it if it doesn't work well for them.
>
> I think the main point of contention here is that glibc is the default system-wide
> allocator, which users expect to work well across different workloads, and another
> expectation is that the same allocator prioritizes performance for specific workloads.
>
> I am focusing on the former because it is not clear that THP usage is beneficial
> across all possible workloads and environments (especially given some glibc
> ptmalloc shortcomings, such as fragmentation handling in some scenarios), or that
> changing the defaults will always benefit users.
>
> From Dimitri's report, I don't think it is an uncommon high-CPU container cluster
> with a low memory-to-CPU ratio. This is exactly what cloud providers are targeting
> for current Aarch64 deployments with current chips.


Most container deployments in such hardware / workload density
configurations do not currently use cpuset cgroups. Meaning all
containers typically see the host cpu count, rather than a cpuset
subset. The consequence is that one has to typically manually tweak
MALLOC_ARENA_MAX, most famously this heroku guidance
https://devcenter.heroku.com/articles/tuning-glibc-memory-behavior I
don't know how container/cgroups aware glibc is.

It would be very interesting if glibc malloc allocator could somehow
sense if it is in a container or not (even if by checking the
container environment variable) and for example choosing to lower
MALLOC_ARENA_MAX from the 8 times count of observable cpus to a fixed
lower value.
I wish containers would universally hint the weight and total amount
of cpu shares dynamically adjusted for the number of deployed
containers - but I don't believe this is available today.

Also, no config choice can be universal. The high-performance
deployments typically already tune these things. But it seems like the
number of workloads and glibc loads is no longer bare-metal heavy; but
is VM heavy and container heavy. (a given deployment is most often
likely to load glibc in a constrained container).

One size does not fit all; but more dynamic defaults would be nice.

I saw patches to add support for setting tunables via a config file.
It would be nice to ship multiple example config files for bare-metal,
vm, container => even if they just set malloc_arena_max 2. Then
distributions will get a hint to package these, and provide them as
config files with like update-alternatives. And possibly ship one
edition by default in "container" builds, another in "vm" builds, and
so on.

This approach worked really well for tuning ext4 with the e2fsprogs
default config file sensitive to the total size of filesystem w.r.t.
reserve space / inode reservation / etc. Such that creating ext4 on a
small sd-card results in a different performance profile, compared to
a multi-terabyte filesystem.

I think the desire to change the default mostly stems from lack of
good levers for applying the default tuning; or lack of automatic
dynamic  tuning based on a given environment.

>
> And it worries me that, in such scenarios, sysadmins will need to be aware of
> additional configuration to avoid potential pitfalls on aarch64, especially when
> comparing with other ABIs that also support THP but do not enable it by default.
>
> Checking on different memory allocators, it really depends on which kind of
> workload it targets:
>
> * TCMalloc [1] has baked in THP support and not switch to disable its usage. It
>   seems widely used by Google and MongoDB [2].
>
> * jemalloc [3] only enables it as an opt-in feature (with the MALLOC_CONF="thp..."
>   option).
>
> * mimalloc [4] also enabled THP usage and set it by default (MIMALLOC_ALLOW_THP).
>
> The TCMalloc case is interesting because Google uses its system-wide profile
> (GWP) to drive allocator development to avoid bias from microbenchmarks or specific
> benchmarks. It also takes into consideration, from the start, data structures and
> heuristics to avoid fragmentation across different workloads. The mimalloc in similar
> wrt to design (that takes in consideration THP).
>
> And I do not think we have that amount of research to certify that glibc THP usage
> yields the same performance gain across the myriad of workloads it is used for. We
> already have a fair number of corner issues [6][7][8][9][10] that might be exacerbated
> by using THP by default.
>
> Another worry is that this kind of change might create even more attrition and
> misconceptions, leading people to believe that a malloc replacement would be
> better than glibc’s.
>
> So I think it would be a good way to first advertise THP support and how to properly
> use the tunable, and I would say it is a failure on our part that this isn't more
> widespread. This kind of information is really very project-specific, and I don’t
> have a good answer on how to make it more widespread.
>
> [1] https://github.com/google/tcmalloc
> [2] https://www.mongodb.com/docs/manual/administration/tcmalloc-performance/
> [3] https://jemalloc.net/
> [4] https://github.com/microsoft/mimalloc
> [5] https://paulcavallaro.com/blog/tcmalloc-temeraire-hugepage-aware-allocator/https://paulcavallaro.com/blog/tcmalloc-temeraire-hugepage-aware-allocator/
> [6] https://sourceware.org/bugzilla/show_bug.cgi?id=31556
> [7] https://sourceware.org/bugzilla/show_bug.cgi?id=30769
> [8] https://sourceware.org/bugzilla/show_bug.cgi?id=15321
> [9] https://sourceware.org/bugzilla/show_bug.cgi?id=26969
> [10] https://sourceware.org/bugzilla/show_bug.cgi?id=21731
>
>


-- 
Regards,

Dimitri.