[PATCH v6 5/6] elf: Align large load segments to PMD huge page size for THP

[Adhemerval Zanella Netto]

On 09/03/26 22:01, WANG Rui wrote:
> Hi,
> 
> On Tue, Mar 10, 2026 at 2:25 AM Adhemerval Zanella Netto
> <adhemerval.zanella@linaro.org> wrote:
>>
>> On 09/03/26 14:36, Wilco Dijkstra wrote:
>>> Hi Adhemerval,
>>>
>>>> There is also the potential memory usage increase of using this tunable as
>>>> default, as we saw a report for the malloc one on AArch64.  I think it would
>>>> less than the malloc, but on multi-layer environment with a lot of process
>>>> this could compound to a significant value.
>>>>
>>>> I think we should *not* make this the default for now.
>>>
>>> Would it round up a 33MB load segment to 64MB? If so, that could increase
>>> memory usage if you had lots of processes of that size. However the OS could
>>> trivially drop any readonly hugepage at the first hint of memory pressure and
>>> revert to base pages. So the fallback mechanism should be pretty robust.
>>
>> Even if _dl_map_segment_align only considers !PROT_WRITE and aligned to the
>> thp_size, _dl_map_object_from_fd will align all PT_LOAD segments:
>>
>> 1228     /* Align all PT_LOAD segments to the maximum p_align.  */
>> 1229     for (size_t i = 0; i < nloadcmds; i++)
>> 1230       loadcmds[i].mapalign = p_align_max;
>>
>> We can refactor it to just align some specific segments.
>>
>> But even with change, I would not do a system-wide change and rely on kernel
>> OOM mechanism to proper handle memory pressure. We already had some complains
>> for the AArch64 malloc one (which I think we should revert) so I think we
>> should be conservative here and make this a opt-in feature.
> 
> Regarding virtual address space: increasing the alignment can indeed
> introduce larger gaps between shared library mappings, which wastes
> some virtual address space. On 64-bit this is basically a non-issue.
> On 32-bit we avoid extreme cases by capping it with MAX_THP_PAGESIZE.
> 
> As for physical memory pressure, if a mapping happens to satisfy the
> THP size and alignment requirements, the kernel may still collapse it
> into a huge page even when only a few normal pages are actually
> populated. In that case more physical memory will be allocated.
> 
> From my understanding, the kernel THP collapse decision should
> ultimately be responsible for this, and the control we have here is
> actually quite weak.

Besides higher memory usage and possible increase of number of memory
mappings (due gapping filling in _dl_map_segments), it also has the side 
effect of silent lowering ASLR bits.  We had some issue in the past
(check here [1] for some history), so I think we should avoid doing 
changing the mapping alignment without users explicitly asking for this.

> 
>> In fact, I think the best option for this THP enablement would to work
>> towards adding a ELF marking as hint to enable it.  It can also help on
>> the static and binary/ld.so case, since it is up to the kernel to the
>> alignment.
> 
> I would suggest that we avoid requiring changes to ELF binaries for
> this. Otherwise a large number of existing binaries would simply miss
> this opt.

Other system uses similar strategies to accomplish it, AIX with
XCOFF Auxiliary Header and Solaris with MPSS and PF_SUNW_LARGEPAGE.

Having a proper ELF marking will close the missing kernel gap for
static and the loader, where it requires using extra tools like BOLT
hugify tool to accomplish.

Also, having the tunable it would be always possible to enable it
and with a system-wide tunable enforce it on the system.

[1] https://grsecurity.net/toolchain_necromancy_past_mistakes_haunting_aslr#:~:text=The%20part%20that%20says%20%E2%80%9Climits,us%20to%20our%20next%20suspect.