RFC: malloc automatic THP opt-in breaks madvise contract for container hosts

Fri Apr 17 05:24:51 GMT 2026

On 4/16/26, 10:23 PM, "Sun, Yutong" <yutongsu@amazon.com <mailto:yutongsu@amazon.com>> wrote:

Thanks for the pointer. The per-binary opt-in idea is interesting.

To clarify our case — the issue is specifically malloc's thp_init()
auto-calling madvise(MADV_HUGEPAGE) when the kernel is in `madvise`
mode. Your proposal covers PT_LOAD segments which seems broader.
Would a binary without the property effectively get no THP at all
under `madvise` mode?

Also, are you hearing similar reports from other distros as 2.43 rolls out?

Again, as I mentioned we're carrying a revert patch [1] for now and happy to move to
whatever upstream solution lands.

[1] https://github.com/bottlerocket-os/bottlerocket-core-kit/pull/905 <https://github.com/bottlerocket-os/bottlerocket-core-kit/pull/905>

Thanks,
Yutong

On 4/16/26, 10:00 PM, "H.J. Lu" <hjl.tools@gmail.com <mailto:hjl.tools@gmail.com> <mailto:hjl.tools@gmail.com <mailto:hjl.tools@gmail.com>>> wrote:

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

On Fri, Apr 17, 2026 at 12:48 PM H.J. Lu <hjl.tools@gmail.com <mailto:hjl.tools@gmail.com> <mailto:hjl.tools@gmail.com <mailto:hjl.tools@gmail.com>>> wrote:
>
> On Fri, Apr 17, 2026 at 12:33 PM Sun, Yutong <yutongsu@amazon.com <mailto:yutongsu@amazon.com> <mailto:yutongsu@amazon.com <mailto:yutongsu@amazon.com>>> wrote:
> >
> > Hi,
> >
> >
> >
> > I'd like to raise a concern about the automatic Transparent Huge Pages
> >
> > (THP) opt-in behavior introduced in malloc, specifically the commits:
> >
> >
> >
> > 321e1fc7 - malloc: Enable 2MB THP by default on Aarch64
> >
> > 1c588a21 - malloc: Improve thp_init
> >
> > bd569425 - malloc: Fix clang build after 1c588a2187
> >
> >
> >
> > We maintain Bottlerocket, a Linux distribution purpose-built for running
> >
> > containers. We've had to carry a revert patch for these changes [1] and
> >
> > would like to discuss a proper upstream solution.
> >
> >
> >
> > ## The Problem
> >
> >
> >
> > When the kernel is configured with THP in `madvise` mode, glibc now
> >
> > automatically calls madvise(MADV_HUGEPAGE) on heap allocations for every
> >
> > glibc-linked process. This changes the semantics of `madvise` mode: the
> >
> > whole point of `madvise` (vs `always`) is that applications explicitly
> >
> > opt in. glibc opting in on behalf of every process defeats this.
> >
> >
> >
> > On a container host, the impact is particularly counterproductive. These
> >
> > are measured values from a Bottlerocket node running Kubernetes pods on
> >
> > aarch64:
> >
> >
> >
> > - kubelet: ~24 MB AnonHugePages
> >
> > - containerd: ~22 MB AnonHugePages
> >
> > - containerd-shim (per pod): ~22-28 MB AnonHugePages
> >
> >
> >
> > The container shims — small management processes with minimal heap
> >
> > working sets — receive huge page allocations, while the actual container
> >
> > workloads that might benefit from THP do not. On a node running, say,
> >
> > 200 pods, shim processes alone would account for 4-5 GB of huge page
> >
> > allocations. After applying our revert patch, all of these drop to 0 kB.
> >
> >
> >
> > ## Alternatives Considered
> >
> >
> >
> > - GLIBC_TUNABLES (glibc.malloc.hugetlb=0): Tunables are explicitly
> >
> > not part of the stable ABI [2] and the mechanism has had security
> >
> > vulnerabilities (CVE-2023-4911 [3]). As a distribution, we cannot
> >
> > depend on unstable interfaces for correctness.
> >
> >
> >
> > - Disabling THP system-wide: Too broad — penalizes workloads that
> >
> > genuinely benefit from THP.
> >
> >
> >
> > - Carrying a revert patch: This is what we're doing today [1]. It
> >
> > works but is not sustainable long-term.
> >
> >
> >
> > ## What We'd Like to Discuss
> >
> >
> >
> > We recognize the THP optimization has value for workloads with large
> >
> > heaps. The concern is that the current default removes choice from
> >
> > distributions and operators who configure `madvise` mode precisely to
> >
> > retain that control.
> >
> >
> >
> > We're open to whatever approach the community prefers and happy to
> >
> > contribute. We expect other distributions will encounter this as
> >
> > glibc 2.43 rolls out more broadly.
> >
> >
> >
> > [1] https://github.com/bottlerocket-os/bottlerocket-core-kit/pull/905 <https://github.com/bottlerocket-os/bottlerocket-core-kit/pull/905> <https://github.com/bottlerocket-os/bottlerocket-core-kit/pull/905> <https://github.com/bottlerocket-os/bottlerocket-core-kit/pull/905>>
> >
> > [2] https://sourceware.org/glibc/manual/2.43/html_mono/libc.html#Tunables <https://sourceware.org/glibc/manual/2.43/html_mono/libc.html#Tunables> <https://sourceware.org/glibc/manual/2.43/html_mono/libc.html#Tunables> <https://sourceware.org/glibc/manual/2.43/html_mono/libc.html#Tunables>>
> >
> > [3] https://lwn.net/Articles/947736/ <https://lwn.net/Articles/947736/> <https://lwn.net/Articles/947736/> <https://lwn.net/Articles/947736/>>
> >
> >
> >
> > Thanks,
> >
> > Yutong
>
> Will my GNU_PROPERTY_1_NEEDED_TRANSPARENT_HUGEPAGE proposal:
>
> https://inbox.sourceware.org/gnu-gabi/CAMe9rOqRQLJ_03gfy1PvJ2-fPuMMGXBNCx2_paC+ZM_n_-5jsQ@mail.gmail.com <mailto:CAMe9rOqRQLJ_03gfy1PvJ2-fPuMMGXBNCx2_paC+ZM_n_-5jsQ@mail.gmail.com> <mailto:CAMe9rOqRQLJ_03gfy1PvJ2-fPuMMGXBNCx2_paC+ZM_n_-5jsQ@mail.gmail.com <mailto:CAMe9rOqRQLJ_03gfy1PvJ2-fPuMMGXBNCx2_paC+ZM_n_-5jsQ@mail.gmail.com>>/T/#mf03cb1820f19c60967b3ffac7f90c0677c915def
>
> help you? My proposal is to opt-in THP for read-only segments. We can
> add another bit to opt-in THP for other memories, like read-write segments
> and heap.
>
>

I opened:

https://sourceware.org/bugzilla/show_bug.cgi?id=34081 <https://sourceware.org/bugzilla/show_bug.cgi?id=34081> <https://sourceware.org/bugzilla/show_bug.cgi?id=34081> <https://sourceware.org/bugzilla/show_bug.cgi?id=34081>>

--
H.J.