[PATCH v2] malloc: Enable 2MB THP by default on Aarch64

[Dev Jain]

On 17/12/25 10:31 pm, Wilco Dijkstra wrote:
> Hi Adhemerval,
>
>>> So without making the huge page support significantly smarter (and correct),
>>> I don't believe such huge page sizes work at all for GLIBC.
>> I agree that this 512mb is impractical for most if not all workloads. I was just
>> curious if we have tried to actually use it to see if there is any conformance
>> issue or performance differences with spec.
> I know a colleague tried SPEC with 512MB huge pages, and he was complaining
> that malloc would use up huge pages for alignment padding or internal heap
> and fail without errors to use huge pages in the cases he wanted it. You need
> to supply far more huge pages than you think...
>
>> I was thinking is use the reported values of /sys/kernel/mm/hugepages/ instead
>> of /sys/kernel/mm/transparent_hugepage/ because it was how libhugetlbfs used to
>> work while reserving huge pages.  The mTHP does allow a lot of more page size
>> and I agree that this unlikely to be useful for most cases.
>>
>> A possible /sys/kernel/mm/hugepages/ side selection does make sense for for
>> glibc.malloc.hugetlb=2 though.
> I think you just want the smallest huge page rather than going with something
> really huge.
>
> Like with caches, you get almost all of the gains from the first few doublings - 
> nobody uses a 1MB L1 cache!
>
>> And it seems that there are some users trying hugetlb=2, specially for the case
>> where the workload used to rely on libhugetlbfs (and they are not supported anymore
>> since we remove the malloc hooks).  I did fixed some issues [1], after a coworker
>> reported some issues with an specific workload.
> Yes that particular issue affects THP as well. There are some interesting alignment
> issues around the very first sbrk() call, where we won't get an aligned huge page until
> we allocate more. I'm wondering whether we could use mmap instead (placed near
> the sbrk region) or just sbrk() till we are aligned and never use the initial unaligned
> part (not sure it is legal to unmap it - Dev?).

As discussed before, I do not think that Linux ever had a heap VMA which was unaligned.
The initial program break *may* be unaligned, but when the VMA gets created, it is always aligned.
Linux does not treat the heap VMA as a special VMA (for example, it does treat the stack VMA as
a special VMA), and the rule for VMAs is that their start and size is always aligned to the
page size the kernel is working on. I am pretty sure the hole is in the glibc implementation
of sbrk(), which converts the sbrk call to a brk() Linux syscall, or the absolutely horrible
alignment code in malloc.

As far as unmapping the heap VMA is concerned, I believe it is legal since Linux treats it
as all other VMAs. But I am concerned about the user-kernel boundary here. It is a known
fact for decades that the heap is a contigous span of virtual space (although that gets
broken already because malloc() allocates secondary heaps in the main arena too - so probably
just a matter of implementation).

>
>> What I am not sure if we will start to get reports that this change increase resource
>> usage and will need to reevaluate the default strategy.  I noted that you sent a
>> change to override DEFAULT_THP_PAGESIZE with glibc.malloc.hugetlb=0 [2], which is on
>> my backlog and I would like to have it for 2.43.
> It can increase memory usage of course, but that can be improved by moving towards
> fewer and larger heaps. Linux does not have to use huge pages if it is low on memory.
> But having an override like you suggested seemed like a good idea.
>
>> I still think we will need to evaluate it to make it the default for other
>> ISA that also support hugepages. But I think we can start with aarch64 and
>> see how it goes on next release.
> Agreed. It's also the only way to find out if there are other issues with GLIBCs THP
> implementation - using it in 1 or 2 applications is not the same as it being the
> default for all.
>
> Cheers,
> Wilco