RFC: malloc automatic THP opt-in breaks madvise contract for container hosts

[Sun, Yutong]

Hi,

I'd like to raise a concern about the automatic Transparent Huge Pages
(THP) opt-in behavior introduced in malloc, specifically the commits:

  321e1fc7 - malloc: Enable 2MB THP by default on Aarch64
  1c588a21 - malloc: Improve thp_init
  bd569425 - malloc: Fix clang build after 1c588a2187

We maintain Bottlerocket, a Linux distribution purpose-built for running
containers. We've had to carry a revert patch for these changes [1] and
would like to discuss a proper upstream solution.

## The Problem

When the kernel is configured with THP in `madvise` mode, glibc now
automatically calls madvise(MADV_HUGEPAGE) on heap allocations for every
glibc-linked process. This changes the semantics of `madvise` mode: the
whole point of `madvise` (vs `always`) is that applications explicitly
opt in. glibc opting in on behalf of every process defeats this.

On a container host, the impact is particularly counterproductive. These
are measured values from a Bottlerocket node running Kubernetes pods on
aarch64:

  - kubelet:              ~24 MB AnonHugePages
  - containerd:           ~22 MB AnonHugePages
  - containerd-shim (per pod): ~22-28 MB AnonHugePages

The container shims — small management processes with minimal heap
working sets — receive huge page allocations, while the actual container
workloads that might benefit from THP do not. On a node running, say,
200 pods, shim processes alone would account for 4-5 GB of huge page
allocations. After applying our revert patch, all of these drop to 0 kB.

## Alternatives Considered

  - GLIBC_TUNABLES (glibc.malloc.hugetlb=0): Tunables are explicitly
    not part of the stable ABI [2] and the mechanism has had security
    vulnerabilities (CVE-2023-4911 [3]). As a distribution, we cannot
    depend on unstable interfaces for correctness.

  - Disabling THP system-wide: Too broad — penalizes workloads that
    genuinely benefit from THP.

  - Carrying a revert patch: This is what we're doing today [1]. It
    works but is not sustainable long-term.

## What We'd Like to Discuss

We recognize the THP optimization has value for workloads with large
heaps. The concern is that the current default removes choice from
distributions and operators who configure `madvise` mode precisely to
retain that control.

We're open to whatever approach the community prefers and happy to
contribute. We expect other distributions will encounter this as
glibc 2.43 rolls out more broadly.

[1] https://github.com/bottlerocket-os/bottlerocket-core-kit/pull/905
[2] https://sourceware.org/glibc/manual/2.43/html_mono/libc.html#Tunables
[3] https://lwn.net/Articles/947736/

Thanks,
Yutong
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20260417/b554bb4e/attachment-0001.htm>