[PATCH v8] elf: Support THP segment load with madvise enabled THP
H.J. Lu
hjl.tools@gmail.com
Fri May 15 23:39:11 GMT 2026
On Fri, May 15, 2026 at 06:21:50PM -0300, Adhemerval Zanella Netto wrote:
>
>
> On 14/05/26 18:58, H.J. Lu wrote:
> > On Fri, May 15, 2026 at 1:31 AM Adhemerval Zanella Netto
> > <adhemerval.zanella@linaro.org> wrote:
> >> On 13/05/26 22:31, H.J. Lu wrote:
> >>> Changes in v7:
> >>>
> >>> Rebased against
> >>>
> >>> commit 389ecf52bc75e6378b2881415b03f3c00a3ef891
> >>> Author: Shamil Abdulaev <ashamil435@gmail.com>
> >>> Date: Wed May 13 07:52:40 2026 +0200
> >>>
> >>> libio: Fix race in _IO_new_file_init_internal initialization order
> >>> [BZ #33785]
> >>>
> >>>
> >>> --
> >>> H.J.
> >>> --> From 0172a61ad9a870882dcec0e1a82e646ed23c5614 Mon Sep 17 00:00:00 2001
> >>> From: "H.J. Lu" <hjl.tools@gmail.com>
> >>> Date: Mon, 13 Apr 2026 08:23:05 +0800
> >>> Subject: [PATCH v7] elf: Support THP segment load with madvise enabled THP
> >>>
> >>> The current THP segment load approach works only when THP is enabled
> >>> with always in the kernel. If THP is enabled with madvise in the
> >>> kernel, to enable THP segment load in an application, madvise should
> >>> be called with MADV_HUGEPAGE on all THP eligible PT_LOAD segments:
> >>>
> >>> 1. Define DL_MAP_DEFAULT_THP_PAGESIZE in hugepages.h and default it to 0.
> >>> 2. Update _dl_map_segment_align to support madvise THP mode. This fixes
> >>> BZ #34079.
> >>> 3. Call _dl_executable_postprocess in rtld_setup_main_map for dynamic
> >>> executables and in LIBC_START_MAIN for static executables, which calls
> >>> madvise with MADV_HUGEPAGE on all THP eligible PT_LOAD segments in
> >>> executable. This fixes BZ #34080 for both dynamic and static executables.
> >>
> >> _dl_executable_postprocess is invoked on main_map, never on _dl_rtld_map.
> >> The loader will most likely never be elective for Transparent Hugepage (THP):
> >
> > We can't call madvise on ld.so:
> >
> > madvise(0x7fcbc294b000, 33564605, MADV_HUGEPAGE) = -1 ENOMEM (Cannot
> > allocate memory)
>
> I think we need to use the expect text segment size:
>
> $ gdb posix/tst-mmap-offset
> Reading symbols from posix/tst-mmap-offset...
> (gdb) b main
> [...]
> (gdb) r
> [...]
> (gdb) info proc
> [...]
> 0x7ffff7fc5000 0x7ffff7fc6000 0x1000 0x0 r--p [...]elf/ld.so
> 0x7ffff7fc6000 0x7ffff7fef000 0x29000 0x1000 r-xp [...]elf/ld.so
> 0x7ffff7fef000 0x7ffff7ffb000 0xc000 0x2a000 r--p [...]elf/ld.so
> 0x7ffff7ffb000 0x7ffff7ffd000 0x2000 0x35000 r--p [...]elf/ld.so
> 0x7ffff7ffd000 0x7ffff7ffe000 0x1000 0x37000 rw-p [...]elf/ld.so
> 0x7ffff7ffe000 0x7ffff7fff000 0x1000 0x0 rw-p
> 0x7ffffffdd000 0x7ffffffff000 0x22000 0x0 rw-p [stack]
> 0xffffffffff600000 0xffffffffff601000 0x1000 0x0 --xp [vsyscall]
> (gdb) call madvise(0x7ffff7fc6000, 0x29000, 14)
> '__GI_madvise' has unknown return type; cast the call to its declared return type
> (gdb) call (int)madvise(0x7ffff7fc6000, 0x29000, 14)
> $1 = 0
>
> It should not matter for x86_64, but for aarch64 and other ISA with mTHP
> (multi-size Transparent Huge Pages) could use different page size other
> than 2MB it could yield some gains.
>
> But we do not any support for mTHP now, so I don't think it is worth.
We can add it later.
>
> Some comments below, I will try to take another look at this weekend.
>
> > --- /dev/null
> > +++ b/sysdeps/generic/dl-load-post.h
> > @@ -0,0 +1,31 @@
> > +/* _dl_postprocess_loadcmd_extra. Generic version.
> > + Copyright (C) 2026 Free Software Foundation, Inc.
> > + Copyright The GNU Toolchain Authors.
> > + This file is part of the GNU C Library.
> > +
> > + The GNU C Library is free software; you can redistribute it and/or
> > + modify it under the terms of the GNU Lesser General Public
> > + License as published by the Free Software Foundation; either
> > + version 2.1 of the License, or (at your option) any later version.
> > +
> > + The GNU C Library is distributed in the hope that it will be useful,
> > + but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > + Lesser General Public License for more details.
> > +
> > + You should have received a copy of the GNU Lesser General Public
> > + License along with the GNU C Library; if not, see
> > + <https://www.gnu.org/licenses/>. */
> > +
> > +#include <dl-load.h>
Will be removed.
> It would be better to add include guards, since dl-load.h also include
> dl-load-post.h. Better if the dl-load-post.h can be move after
> the 'struct loadcmd' definition.
dl-load-post.h is included right after the 'struct loadcmd' definition
in dl-load.h.
>
> > + "DL_MAP_DEFAULT_THP_PAGESIZE <= MAX_THP_PAGESIZE");
> > +
> > + /* NB: Accessing /sys/kernel/mm files is quite expensive and the file
> > + may not be accessible in containers. If DL_MAP_DEFAULT_THP_PAGESIZE
> > + is non-zero, assume THP mode is madvise and always call madvise.
> > + Since madvise is a fast systemcall, it adds only a small overhead
> > + compared to the cost of accessing /sys/kernel/mm files. */
> > + if (DL_MAP_DEFAULT_THP_PAGESIZE != 0)
> > + {
> > + GL(dl_elf_thp_pagesize) = DL_MAP_DEFAULT_THP_PAGESIZE;
> > + GL(dl_thp_mode) = thp_mode_madvise;
>
> I think we can avoid issuing the madvise for the 'always' THP mode by checking
> the __get_thp_mode:
>
> if (DL_MAP_DEFAULT_THP_PAGESIZE != 0)
> {
> enum thp_mode_t m = __get_thp_mode ();
> GL(dl_thp_mode) = (m == thp_mode_not_supported) ? thp_mode_madvise : m;
> }
>
> And then the 'GL(dl_elf_thp_control) = 1' below '/* NB: Enable THP if THP is
> always enabled in the kernel. */' can be removed. On _dl_executable_postprocess
> we can also:
>
> @@ -93,10 +95,10 @@ _dl_executable_postprocess (struct link_map *main_map,
> if (!need_madvise || GL(dl_elf_thp_control) <= 0)
> return;
>
> - /* NB: If DL_MAP_DEFAULT_THP_PAGESIZE is non-zero, dl_thp_mode is set
> - to thp_mode_madvise. */
> - if (DL_MAP_DEFAULT_THP_PAGESIZE == 0
> - && GL(dl_thp_mode) != thp_mode_madvise)
> + /* Only "madvise" mode benefits from an explicit MADV_HUGEPAGE: in
> + "always" mode the kernel handles THP automatically for properly
> + aligned mappings, and in "never" mode the call is a no-op. */
> + if (GL(dl_thp_mode) != thp_mode_madvise)
> return;
>
> /* When we get here, the main executable have been mapped in. Call
>
>
> This will end with:
>
> * kernel 'always': PMD alignment only, no madvise from either path (not
> required).
> * kernel 'madvise', kernel-loaded / static: madvise from postprocess only.
> * kernel 'madvise', ld.so /foo: madvise from _dl_map_segments only.
> * kernel 'never': no work — _dl_postprocess_loadcmd_extra_needed() returns
> false and the postprocess loop is gated off by mode != madvise.
> * no /sys (containers): falls back to today's behavior.
>
> This will cost us a couple of syscall from __get_thp_mode, so it will
> depends whether madvise is costly than accessing sysfs (I am not sure).
I'd like to keep it ASIS before we have data to show which one is
faster. For the longer term, we should add AT_XXXs for THP mode as
well as THP page sizes to avoid syscalls.
>
>
> > + }
> > + else
> > + {
> > + GL(dl_elf_thp_pagesize) = __get_thp_size ();
> > + GL(dl_thp_mode) = __get_thp_mode ();
> > + /* We cap the huge page size at MAX_THP_PAGESIZE to avoid
> > + over-aligning on systems with very large normal pages
> > + (like 64K pages with 512M huge pages). */
> > + if (GL(dl_elf_thp_pagesize) > MAX_THP_PAGESIZE)
> > + GL(dl_elf_thp_control) = 0;
> > + /* NB: Enable THP if THP is always enabled in the kernel. */
> > + else if (GL(dl_thp_mode) == thp_mode_always)
> > + GL(dl_elf_thp_control) = 1;
> > + }
> > +}
> > +
> > +static inline void
> > +_dl_executable_postprocess (struct link_map *main_map,
> > + const ElfW(Phdr) *phdr, ElfW(Word) phnum)
> > +{
> > + if (GL(dl_elf_thp_control) == -1)
> > + _dl_get_thp_config ();
>
> This still trigger two madvise for the case of direct loader invocation,
> which is wasteful:
>
> $ strace -e madvise \
> env GLIBC_TUNABLES=glibc.elf.thp=1 \
> elf/ld.so --library-path . \
> elf/tst-thp-1 --direct 2>&1 | grep MADV_HUGEPAGE
> madvise(0x7784e5200000, 33566720, MADV_HUGEPAGE) = 0
> madvise(0x7784e5200000, 33563701, MADV_HUGEPAGE) = 0
>
> I think you need something like:
>
> diff --git a/sysdeps/unix/sysv/linux/dl-exec-post.h b/sysdeps/unix/sysv/linux/dl-exec-post.h
> index b7dc7be8120..cd1f7c458e1 100644
> --- a/sysdeps/unix/sysv/linux/dl-exec-post.h
> +++ b/sysdeps/unix/sysv/linux/dl-exec-post.h
> @@ -63,7 +63,15 @@ static inline void
> _dl_executable_postprocess (struct link_map *main_map,
> const ElfW(Phdr) *phdr, ElfW(Word) phnum)
> {
> - if (GL(dl_elf_thp_control) == -1)
> + /* If THP state was not yet initialized, the main executable was mapped by
> + the kernel; in that case this function is the only place that can apply
> + MADV_HUGEPAGE to the main executable's segments. Otherwise,
> + _dl_get_thp_config has already run earlier in dl_main and
> + _dl_map_segments has just mapped the main executable, so
> + _dl_postprocess_loadcmd_extra has already done the madvise pass; do
> + not repeat it here. */
> + bool need_madvise = (GL(dl_elf_thp_control) == -1);
> + if (need_madvise)
> _dl_get_thp_config ();
>
> /* NB: In static executable, PT_GNU_PROPERTY is processed in target
> @@ -82,7 +90,7 @@ _dl_executable_postprocess (struct link_map *main_map,
> #endif
>
> /* Return if THP segment load isn't enabled. */
> - if (GL(dl_elf_thp_control) <= 0)
> + if (!need_madvise || GL(dl_elf_thp_control) <= 0)
> return;
>
> /* NB: If DL_MAP_DEFAULT_THP_PAGESIZE is non-zero, dl_thp_mode is set
>
> with this applied:
>
> $ strace -e madvise \
> env GLIBC_TUNABLES=glibc.elf.thp=1 \
> elf/ld.so --library-path . \
> elf/tst-thp-1 --direct 2>&1 | grep MADV_HUGEPAGE
> madvise(0x7b7730a00000, 33566720, MADV_HUGEPAGE) = 0
I changed it to
if (GL(dl_elf_thp_control) != -1)
return;
_dl_get_thp_config ();
/* Return if THP segment load isn't enabled. */
if (GL(dl_elf_thp_control) <= 0)
return;
> > +
> > +# Test whether strace is available in the test environment. If not, skip
> > +# the test.
> > +${test_wrapper_env} ${run_program_env} \
> > + /bin/sh -c "command -v strace" || exit 77
>
> I think we should add a test to check if strace actually works, to avoid
> filtering ptrace (yama ptrace_scope >= 1, seccomp, or chroots that block
> ptrace):
>
> # Verify strace is not just present, but works in this environment.
> ${test_wrapper_env} ${run_program_env} \
> strace -e trace=none -- /bin/true >/dev/null 2>&1 || exit 77
>
Fixed:
/bin/sh -c \
"${test_wrapper_env} ${run_program_env} \
strace -e trace=none -- /bin/true" > /dev/null 2>&1 || exit 77
> I also think we should add some timeout to avoid strace blocking indefinitely,
> like iconv/tst-iconv_prog.sh does.
Fixed:
TIMEOUTFACTOR=${TIMEOUTFACTOR:-1}
...
if /bin/sh -c \
"timeout -k 4 $((3*$TIMEOUTFACTOR)) \
${test_wrapper_env} ${run_program_env} strace ${rtld} \
--library-path ${library_path} ${test_prog} --direct 2>&1 \
| grep -E \"madvise.*, MADV_HUGEPAGE\""; then
>
> > +
> > +# Finally the actual test inside the test environment, using the just
> > +# build ld.so and new libraries to run the THP test under strace.
> > +if /bin/sh -c \
> > + "${test_wrapper_env} ${run_program_env} strace ${rtld} \
> > + --library-path ${library_path} ${test_prog} --direct 2>&1 \
> > + | grep -E \"madvise(.*, MADV_HUGEPAGE)\""; then
>
> The (...) are accidental ERE groups; escape or drop them.
>
Changed to
grep -E \"madvise.*, MADV_HUGEPAGE\""; then
I will post the v9 patch next week with your feedbacks over the weekend.
Thanks.
H.J.
More information about the Libc-alpha
mailing list