[PATCH v2] malloc: Enable 2MB THP by default on Aarch64

[Wilco Dijkstra]

Hi Adhemerval,

>> The idea was to avoid using the glibc.malloc.hugetlb=1 code path by default since
>> that does several expensive system calls at startup even if you never use malloc.
>> We could easily use glibc.malloc.hugetlb=0 as the disable option (eg. set thp_mode
>> to never in that case and check that in thp_init() instead).
>
> I don't think making the 2MB as the only option for AArch64 a problem, although
> afaik it does only work as expected on Linux 6.9+ with Contiguous PTE enabled.
>
> It also has the side effect of disabling  glibc.malloc.hugetlb=1 support on
> kernel older than 6.9 with 64kb page size.  I am not sure if using 512M is
> really 'unusable' and 'impractical' as Dev has stated in the patch; specially
> because the tunable is optional. But also I am not sure how wildly this configuration
> is used, nor the performance implications. Do we have SPECcpu2017 numbers of
> the performance difference between using 2MB and 512MB mTHP?

For SPEC runs one uses jemalloc with real hugepages, not THP. I don't believe anyone
ever tried to use GLIBC for SPEC - that's a great incentive to improve GLIBC malloc!

512MB THP is not useful at all if all processes will now start to use 512MB of memory
after a small malloc. Even as a real hugepage it is too large since it will simply round up
a 513MB allocation to 1024MB, wasting half the memory...

So without making the huge page support significantly smarter (and correct),
I don't believe such huge page sizes work at all for GLIBC.

> I though initially to add a way to specify the expected huge page size to use
> along with the tunable, something like glibc.malloc.hugetlb=1:2097152.  If
> the value is set it would bypass the hpage_pmd_size parsing.  I am not sure
> how useful it would be; although it would give users a way to check different
> page size with Contiguous PTE.

Or we could use values in the range of 3..4095 for mTHP. However I'm not
convinced infinite configurability is useful in practice - is there any application
actually using glibc.malloc.hugetlb?

> Another change that I am not fully confortable is that aarch64 now behaves slight
> different than other ABIs and that this is a very server oriented tunable. It is
> not clear when users starts to use glibc on aarm64 desktop environments if this
> will incurs in higher memory usage compared to other ABIs.

AArch64 on Linux+GLIBC is very server oriented of course - even "embedded" uses
typically have 16+ high-end cores...

But consider the last time any malloc settings were changed. The top-pad, trim and
mmap thresholds are all unchanged since 2002 - a time when computers had just
256-512MB of memory. If we assume 8GB for a low-end PC in 2025, that's 16 times
larger! So isn't it about time to increase all these limits and make use of huge pages?

Cheers,
Wilco