[PATCH v2 2/2] elf: Support THP segment load with madvise enabled THP

[H.J. Lu]

On Fri, Apr 17, 2026 at 12:11 PM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> The current THP segment load approach works only when THP is enabled
> with always in the kernel.  If THP is enabled with madvise in the
> kernel, to enable THP segment load in an application, madvise should
> be called with MADV_HUGEPAGE on all THP eligible PT_LOAD segments:
>
> 1. Remove _dl_map_segment_align.  Call _dl_map_segment_adjust to adjust
> alignments of PT_LOAD segments, which marks and aligns only THP eligible
> PT_LOAD segments to THP page size when loading an object. This fixes
> BZ #34079.
> 2. Rename DL_MAP_DEFAULT_THP_PAGESIZE to DL_DEFAULT_THP_PAGESIZE.  If
> DL_DEFAULT_THP_PAGESIZE is defined, assume that THP is enabled with
> madvise in the kernel.  Otherwise, call __get_thp_size to get THP page
> size and call __get_thp_mode to get THP mode.
> 3. Call _dl_executable_postprocess in rtld_setup_main_map for dynamic
> executables and in LIBC_START_MAIN for static executables, which calls
> madvise with MADV_HUGEPAGE on all THP eligible PT_LOAD segments in
> executable.  This fixes BZ #34080 for both dynamic and static executables.
> 4. Call _dl_postprocess_loadcmd_extra in _dl_postprocess_loadcmd, which
> calls madvise with MADV_HUGEPAGE on all THP eligible PT_LOAD segments
> when loading an object after they have been mapped in.  This fixes
> BZ #34080 for shared objects.
> 5. Add tests to verify that large executable PT_LOAD segments in
> executables and shared libraries are mapped at addresses aligned to THP
> page size when the kernel is configured to use THP in "always" mode or
> "madvise" mode by inspecting /proc/self/maps to check that the mapping
> address is aligned to THP page size reported by the kernel [1].  Also
> verify that madvise is called with MADV_HUGEPAGE when the glibc tunable
> glibc.elf.thp=1 is used and madvise isn't called with MADV_HUGEPAGE when
> the glibc tunable glibc.elf.thp=0 is used.
>
> Skip these tests if THP page size cannot be determined or if THP is not
> enabled in "always" mode nor "madvise" mode.
>
> Quote WANG Rui <wangrui@loongson.cn>:
>
> From benchmarking a clang build of the Linux kernel on x86_64 with
> your patch in THP madvise mode, I observed that iTLB misses were
> reduced, similar to what we see in THP always mode.
>
> [1] This is developed by WANG Rui <wangrui@loongson.cn>.
>
> Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
> Co-developed-by: WANG Rui <wangrui@loongson.cn>

Changes from v1:

1. Rebased against the v10 THP test patch from WANG Rui <wangrui@loongson.cn>.
2. Keep DL_MAP_DEFAULT_THP_PAGESIZE.

-- 
H.J.
---
The current THP segment load approach works only when THP is enabled
with always in the kernel.  If THP is enabled with madvise in the
kernel, to enable THP segment load in an application, madvise should
be called with MADV_HUGEPAGE on all THP eligible PT_LOAD segments:

1. Remove _dl_map_segment_align.  Call _dl_map_segment_adjust to adjust
alignments of PT_LOAD segments, which marks and aligns only THP eligible
PT_LOAD segments to THP page size when loading an object. This fixes
BZ #34079.
2. Call _dl_executable_postprocess in rtld_setup_main_map for dynamic
executables and in LIBC_START_MAIN for static executables, which calls
madvise with MADV_HUGEPAGE on all THP eligible PT_LOAD segments in
executable.  This fixes BZ #34080 for both dynamic and static executables.
3. Call _dl_postprocess_loadcmd_extra in _dl_postprocess_loadcmd, which
calls madvise with MADV_HUGEPAGE on all THP eligible PT_LOAD segments
when loading an object after they have been mapped in.  This fixes
BZ #34080 for shared objects.
4. Add tests to verify that large executable PT_LOAD segments in
executables and shared libraries are mapped at addresses aligned to THP
page size when the kernel is configured to use THP in "always" mode or
"madvise" mode by inspecting /proc/self/maps to check that the mapping
address is aligned to THP page size reported by the kernel [1].  Also
verify that madvise is called with MADV_HUGEPAGE when the glibc tunable
glibc.elf.thp=1 is used and madvise isn't called with MADV_HUGEPAGE when
the glibc tunable glibc.elf.thp=0 is used.

Skip these tests if THP page size cannot be determined or if THP is not
enabled in "always" mode nor "madvise" mode.

Quote WANG Rui <wangrui@loongson.cn>:

>From benchmarking a clang build of the Linux kernel on x86_64 with
your patch in THP madvise mode, I observed that iTLB misses were
reduced, similar to what we see in THP always mode.

[1] This is developed by WANG Rui <wangrui@loongson.cn>.

Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Co-developed-by: WANG Rui <wangrui@loongson.cn>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: v2-0002-elf-Support-THP-segment-load-with-madvise-enabled.patch
Type: text/x-patch
Size: 50994 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20260417/1db6794e/attachment-0001.bin>