[PATCH v6 5/6] elf: Align large load segments to PMD huge page size for THP

[WANG Rui]

Hi Wilco,

On Thu, Mar 12, 2026 at 1:45 AM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
>
> Hi Rui,
>
> > David Hildenbrand mentioned that READ_ONLY_THP_FOR_FS will likely go
> > away soon. [1]
>
> I double checked, and while that config flag may go, the feature is essentially
> a default now. Even without any extra alignment, iTLB misses are significantly
> lower when you use [madvise] or [always], so it is already using large/huge
> pages automatically.

I double checked this behaviour.

CONFIG_READ_ONLY_THP_FOR_FS=y + THP mode madvise:

Even if the start virtual address and file offset of a read-only
executable PT_LOAD segment are hugepage-aligned, the mapping will not
be collapsed into huge pages automatically unless the region is
explicitly marked with madvise(MADV_HUGEPAGE).

After switching THP mode from "always" to "madvise" and dropping the
page cache (echo 1 > /proc/sys/vm/drop_caches), I no longer observe
any non-zero FilePmdMapped entries in smaps.

CONFIG_READ_ONLY_THP_FOR_FS=y + THP mode always:

A file mapping can only be collapsed into huge pages when the virtual
address and the file offset of the region are both hugepage-aligned
and the region is large enough to cover a huge page.

Also note that even if the start address and file offset of a PT_LOAD
segment are not hugepage-aligned, the segment might still contain a
hugepage-aligned subrange. In that case only that subrange can be
mapped with huge pages, while the head of the segment remains mapped
with normal pages.

This patch increases the chance of getting huge page mappings by
aligning the virtual address of PT_LOAD segments whose file offset is
already hugepage-aligned (by increasing p_align).

For 2MB huge pages, many programs have text sizes only slightly larger
than one huge page, so aligning the start virtual address often
becomes the key condition to get a huge page mapping.

For larger huge pages (e.g. 32MB), the unaligned head region can be
close to the huge page size itself, which makes the potential
performance impact even more significant.

Thanks,
Rui