[PATCH v8 5/6] elf: Align large load segments to PMD huge page size for THP

[H.J. Lu]

On Sun, Apr 12, 2026 at 6:51 PM WANG Rui <wangrui@loongson.cn> wrote:
>
> On Sat, Apr 11, 2026 at 8:15 PM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
> >
> > Hi HJ,
> >
> > > A couple issues:
> > >
> > > 1.  We shouldn't assume what are in /sys/kernel/mm/transparent_hugepage/enabled
> > > nor /sys/kernel/mm/transparent_hugepage/hpage_pmd_size.   We can
> > > add a system call or AT_XXX values to get them.
> >
> > Indeed, reading those files turned out to be a bad idea. Adding AT_XXX sounds like
> > a good idea - it's cheap for the loader to process a few extra entries.
> >
> > However until we have a way to easily get the THP settings at startup, we need an
> > alternative without reading those files. We also need to override the PMD size if
> > it is too large - the current patch limits it to 32MB and for AArch64 I force the THP size
> > to 2MB for all base page sizes.
> >
> > > 2.  We should support THP enabled with madvise.  If the current kernel doesn't
> > > support it, it should be updated.
> >
> > In the future, sure, but it is not a requirement for Rui's patch series.
>
> I agree. This patch is intentionally conservative and only improves
> segment alignment while passively working with THP in "always" mode.
>
> We can move forward incrementally and evaluate the behavior in
> practice before considering a more proactive use of madvise.

1. Since the THP setting can be changed at run-time in the user space,
it is wrong for glibc to assume that THP is enabled with always.
2. There are potential performance issues with THP enabled with always.

I propose the following:

1.  Always check the THP setting at load-time.
2.  Check the tunable once at load-time.
3.  If THP isn't disabled by the tunable,
      3a.  Load shared libraries at THP compatible addresses.
      3b.  If THP is enabled with madvise, issue madvise on all THP
             compatible executable load segments, including main
             executable and shared libraries.
4.  Select an application and build it as PDE with THP compatible executable
load segments.  Use PDE, instead of PIE, ensures that executable load
segments have addresses suitable for THP under the existing kernel.
5. Compare its performance under never THP kernel, always kernel and
madvise THP kernel.

I can provide such glibc changes on top of yours.

madvise THP kernel should have similar iTLB performance as always THP
kernel.

If the results are positive, we can make kernel changes to provide faster
THP setting access and load PIE at THP compatible addresses.

> Thanks,
> Rui
>
> >
> > > 3. Glibc can map PIE and DSO to address aligned to THP page size after
> > >
> > > commit 718fdd87b1b98ef88e883a37d9c18867256fa5a4
> > > Author: Rongwei Wang <rongwei.wang@linux.alibaba.com>
> > > Date:   Fri Dec 10 20:39:10 2021 +0800
> > >
> > >    elf: Properly align PT_LOAD segments [BZ #28676]
> > >
> > > If THP is enabled with madvise, should kernel map PIE to address aligned to
> > > THP page size just like THP enabled with always?  If not,  should we add a
> > > marker in PIE to let kernel know that a PIE should be mapped to address
> > > aligned to THP page size?
> >
> > The kernel should do the alignment when it believes there is an expected benefit -
> > the exact cases for which that is true are likely to change over time depending on how
> > it is implemented in the kernel. If we use AT_XXX as the mechanism, it could actually
> > give GLIBC what it believes is the best alignment for THP (potentially the alignment for
> > file and anonymous mappings might be different).
> >
> > Cheers,
> > Wilco
>


-- 
H.J.