[PATCH v8 5/6] elf: Align large load segments to PMD huge page size for THP

[H.J. Lu]

On Sun, Apr 12, 2026 at 6:14 PM WANG Rui <wangrui@loongson.cn> wrote:
>
> On Sun, Apr 12, 2026 at 7:05 AM H.J. Lu <hjl.tools@gmail.com> wrote:
> >
> > On Sat, Apr 11, 2026 at 8:15 PM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
> > >
> > > Hi HJ,
> > >
> > > > A couple issues:
> > > >
> > > > 1.  We shouldn't assume what are in /sys/kernel/mm/transparent_hugepage/enabled
> > > > nor /sys/kernel/mm/transparent_hugepage/hpage_pmd_size.   We can
> > > > add a system call or AT_XXX values to get them.
> > >
> > > Indeed, reading those files turned out to be a bad idea. Adding AT_XXX sounds like
> > > a good idea - it's cheap for the loader to process a few extra entries.
> > >
> > > However until we have a way to easily get the THP settings at startup, we need an
> > > alternative without reading those files. We also need to override the PMD size if
> > > it is too large - the current patch limits it to 32MB and for AArch64 I force the THP size
> > > to 2MB for all base page sizes.
> >
> > There is an AT_PAGE_SHIFT_MASK proposal for x86:
> >
> > https://lore.kernel.org/lkml/ecb049aa-bcac-45c7-bbb1-4612d094935a@p183/
> >
> > which covers
> >
> > /sys/kernel/mm/hugepages
> >
> > It can be used by target to decide the optimal huge page size.  But we
> > need to cover
> > all targets as well as
> >
> > /sys/kernel/mm/transparent_hugepage/enabled
> >
> > > > 2.  We should support THP enabled with madvise.  If the current kernel doesn't
> > > > support it, it should be updated.
> > >
> > > In the future, sure, but it is not a requirement for Rui's patch series.
> >
> > Since THP has significant drawbacks for certain workloads, some applications,
> > like databases, recommend not setting THP to always.  We should support madvise
> > THP to get it right from the start.
> >
> > > > 3. Glibc can map PIE and DSO to address aligned to THP page size after
> > > >
> > > > commit 718fdd87b1b98ef88e883a37d9c18867256fa5a4
> > > > Author: Rongwei Wang <rongwei.wang@linux.alibaba.com>
> > > > Date:   Fri Dec 10 20:39:10 2021 +0800
> > > >
> > > >    elf: Properly align PT_LOAD segments [BZ #28676]
> > > >
> > > > If THP is enabled with madvise, should kernel map PIE to address aligned to
> > > > THP page size just like THP enabled with always?  If not,  should we add a
> > > > marker in PIE to let kernel know that a PIE should be mapped to address
> > > > aligned to THP page size?
> > >
> > > The kernel should do the alignment when it believes there is an expected benefit -
> > > the exact cases for which that is true are likely to change over time depending on how
> > > it is implemented in the kernel. If we use AT_XXX as the mechanism, it could actually
> > > give GLIBC what it believes is the best alignment for THP (potentially the alignment for
> > > file and anonymous mappings might be different).
> > >
> >
> > But kernel may not know if THP is good for performance for every
> > application.  Al
>
> From my perspective, it is actually harder for applications or
> programmers to make a well-informed decision about adding such a
> marker than it is for the kernel.
> The kernel has a more comprehensive view of the system's runtime
> state, memory pressure, workload patterns, and overall system
> behavior, while an individual application typically lacks that global
> context.
>
> > mode of "google linux transparent huge pages" shows
> >
> > Common Issues & Trade-offs
> >
> > Latency Spikes: The background process (khugepaged) that consolidates pages
> > can cause temporary system freezes or high CPU usage.
>
> Regarding the concerns cited from the general discussions around THP
> trade-offs, I think we should be careful not to apply those
> conclusions too broadly without considering the specific details of
> this patch.
>
> > Memory Waste: Using a 2MB page for only 4KB of data can lead to internal
> > fragmentation and higher memory consumption.
>
> In the current impl, there is already a filtering condition that
> requires the text segment length to exceed the huge page size.
> So the scenario of using a 2MB page to map only 4KB of data does not apply here.
>
> > Database Incompatibility: Many databases (e.g., Oracle, MongoDB, SAP ASE)
> > recommend disabling THP because their non-contiguous access patterns often
> > trigger performance degradations.
>
> Similarly, the database examples commonly refer to memory mappings of
> database **data**, which often have non-contiguous access patterns.
> This patch, however, targets executable **code** segments, whose
> access characteristics are fundamentally different.
>
> Therefore, rather than relying on high-level summaries of THP
> behavior, it would be more productive to evaluate the impact based on
> the concrete behavior of code mappings in this specific context.

The issue is that the current implementation requires and assumes
THP enabled with always which impacts ALL memory mapping,
including data.

>
> >
> > A madvise THP kernel aligning binaries to the optimal huge page size only for
> > binaries with a THP marker can mitigate the THP performance issues.
>
> Thanks,
> Rui
>


-- 
H.J.