[PATCH v8] elf: Support THP segment load with madvise enabled THP
H.J. Lu
hjl.tools@gmail.com
Thu May 14 21:58:15 GMT 2026
On Fri, May 15, 2026 at 1:31 AM Adhemerval Zanella Netto
<adhemerval.zanella@linaro.org> wrote:
> On 13/05/26 22:31, H.J. Lu wrote:
> > Changes in v7:
> >
> > Rebased against
> >
> > commit 389ecf52bc75e6378b2881415b03f3c00a3ef891
> > Author: Shamil Abdulaev <ashamil435@gmail.com>
> > Date: Wed May 13 07:52:40 2026 +0200
> >
> > libio: Fix race in _IO_new_file_init_internal initialization order
> > [BZ #33785]
> >
> >
> > --
> > H.J.
> > --> From 0172a61ad9a870882dcec0e1a82e646ed23c5614 Mon Sep 17 00:00:00 2001
> > From: "H.J. Lu" <hjl.tools@gmail.com>
> > Date: Mon, 13 Apr 2026 08:23:05 +0800
> > Subject: [PATCH v7] elf: Support THP segment load with madvise enabled THP
> >
> > The current THP segment load approach works only when THP is enabled
> > with always in the kernel. If THP is enabled with madvise in the
> > kernel, to enable THP segment load in an application, madvise should
> > be called with MADV_HUGEPAGE on all THP eligible PT_LOAD segments:
> >
> > 1. Define DL_MAP_DEFAULT_THP_PAGESIZE in hugepages.h and default it to 0.
> > 2. Update _dl_map_segment_align to support madvise THP mode. This fixes
> > BZ #34079.
> > 3. Call _dl_executable_postprocess in rtld_setup_main_map for dynamic
> > executables and in LIBC_START_MAIN for static executables, which calls
> > madvise with MADV_HUGEPAGE on all THP eligible PT_LOAD segments in
> > executable. This fixes BZ #34080 for both dynamic and static executables.
>
> _dl_executable_postprocess is invoked on main_map, never on _dl_rtld_map.
> The loader will most likely never be elective for Transparent Hugepage (THP):
We can't call madvise on ld.so:
madvise(0x7fcbc294b000, 33564605, MADV_HUGEPAGE) = -1 ENOMEM (Cannot
allocate memory)
> aarch64-linux-gnu $ readelf -lW elf/ld.so | awk '/^[[:space:]]*LOAD/ && /E/ {print $6}' | \
> while read h; do printf "RX MemSiz: %d\n" "$h"; done
> RX MemSiz: 164904
> x86_64-linux-gnu $ readelf -lW elf/ld.so | awk '/^[[:space:]]*LOAD/ && /E/ {print $6}' | \
> while read h; do printf "RX MemSiz: %d\n" "$h"; done
> RX MemSiz: 164497
>
> but I think for completeness we should also check it.
>
> > 4. Call _dl_postprocess_loadcmd_extra in _dl_postprocess_loadcmd, which
> > calls madvise with MADV_HUGEPAGE on all THP eligible PT_LOAD segments
> > when loading an object after they have been mapped in. This fixes
> > BZ #34080 for shared objects.
> > 5. Set the maximum page alignment on THP tests to THP page size as the
> > default maximum page alignment may be smaller than THP page size.
> > 6. Add tests to verify that large executable PT_LOAD segments in
> > executables are mapped at addresses aligned to THP page size when the
> > kernel is configured to use THP in "always" mode or "madvise" mode by
> > inspecting /proc/self/maps to check that the mapping address is aligned
> > to THP page size reported by the kernel. Also verify that madvise is
> > called with MADV_HUGEPAGE when the glibc tunable glibc.elf.thp=1 is used
> > and madvise isn't called with MADV_HUGEPAGE when the glibc tunable
> > glibc.elf.thp=0 is used.
>
> There is an inconsistency where madvise will be applied for shared libraries
> even when the tunable is not set:
>
> $ uname -m
> x86_64
> $ cat /sys/kernel/mm/transparent_hugepage/enabled
> [always] madvise never
> $ strace -f -e madvise,mmap,openat ./elf/tst-thp-align --direct
> [...]
> openat(AT_FDCWD, "[...]/x86_64-linux-gnu/elf/tst-thp-size-mod.so", O_RDONLY|O_CLOEXEC) = 3
> mmap(NULL, 33562640, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ad94f000000
> madvise(0x7ad94f000000, 33558528, MADV_HUGEPAGE) = 0
>
> Where I would expect that no madvise will be required or applied.
Fixed in the v8 patch:
diff --git a/sysdeps/unix/sysv/linux/dl-exec-post.h b/sysdeps/unix/sysv/linux/dl
-exec-post.h
index 367b20ae2a..b7dc7be812 100644
--- a/sysdeps/unix/sysv/linux/dl-exec-post.h
+++ b/sysdeps/unix/sysv/linux/dl-exec-post.h
@@ -24,6 +24,8 @@ _dl_get_thp_config (void)
if (TUNABLE_IS_INITIALIZED_FULL (glibc, elf, thp))
GL(dl_elf_thp_control) = TUNABLE_GET_FULL (glibc, elf, thp, int32_t,
NULL);
+ else
+ GL(dl_elf_thp_control) = 0;
> This does not happens on aarch64:
>
> $ uname -m
> aarch64
> $ cat /sys/kernel/mm/transparent_hugepage/enabled
> [always] madvise never
> $ strace -f -e madvise,mmap,openat ./elf/tst-thp-align --direct
> [...]
> openat(AT_FDCWD, "/[...]/build/aarch64-linux-gnu/elf/tst-thp-size-mod.so", O_RDONLY|O_CLOEXEC) = 3
> mmap(NULL, 39845920, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_DENYWRITE, -1, 0) = 0xffffa745f000
> mmap(0xffffa7600000, 37748768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xffffa7600000
> mmap(0xffffa99ff000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21ff000) = 0xffffa99ff000
> [...]
If DL_MAP_DEFAULT_THP_PAGESIZE is defined, assume kernel THP madvise mode.
If kernel THP mode is always or never, there is an extra madvise call
which has no impact. DL_MAP_DEFAULT_THP_PAGESIZE is defined for x86 and
64-bit loongarch. Since DL_MAP_DEFAULT_THP_PAGESIZE isn't defined
for aarch64, /sys/kernel/mm/transparent_hugepage/enabled is checked and
madvise isn't called.
> It seems that the two enabled checks at:
>
> * Executable, in sysdeps/unix/sysv/linux/dl-exec-post.h:80:
>
> if (GL(dl_elf_thp_control) <= 0)
> return;
>
> * Dependency / dlopen, in sysdeps/unix/sysv/linux/dl-load-post.h:23:
>
> return GL(dl_thp_mode) == thp_mode_madvise;
After dl_thp_mode is set according to dl_elf_thp_control,
dl_thp_mode should be checked. The bug in v7 is missing:
+ else
+ GL(dl_elf_thp_control) = 0;
> Use different control variables, either _dl_postprocess_loadcmd_extra_needed
> should additionally require dl_elf_thp_control > 0, or _dl_get_thp_config
> should auto-promote dl_elf_thp_control to 1 in the DL_MAP_DEFAULT_THP_PAGESIZE != 0
> and thp_mode == thp_mode_madvise paths.
>
> The tests only exercise thp=1 / thp=0, so this gap escapes coverage.
>
> >
> > Skip these tests if THP page size cannot be determined or if THP is not
> > enabled in "always" mode nor "madvise" mode.
> >
> > Quote WANG Rui <wangrui@loongson.cn>:
> >
> > From benchmarking a clang build of the Linux kernel on x86_64 with
> > your patch in THP madvise mode, I observed that iTLB misses were
> > reduced, similar to what we see in THP always mode.
> >
> > NB: Some THP tests fail on arm due to limitations of arm32 kABI:
> >
> > https://sourceware.org/bugzilla/show_bug.cgi?id=34096
> >
>
Changes in v8:
1. Set GL(dl_elf_thp_control) to 0 if no tunables.
2. Add strace tests for tst-thp-align.
3. Add --direct to strace test to check madvise on DSO.
--
H.J.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: v8-0001-elf-Support-THP-segment-load-with-madvise-enabled.patch
Type: application/x-patch
Size: 51295 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20260515/886059be/attachment-0001.bin>
More information about the Libc-alpha
mailing list