[PATCH] elf: Align large load segments to PMD huge page size for THP

[WANG Rui]

Mapping segments that are at least the size of a PMD huge page to
huge-page-aligned addresses helps make them eligible for Transparent
Huge Pages (THP).

This patch updates the segment mapping logic in the generic ELF loader
to dynamically calculate the PMD huge page size based on the runtime
normal page size. On 64-bit systems, the number of page table entries
is (page_size / sizeof(void *)). Multiplying this by the page size
yields the PMD huge page size (e.g., a 4K normal page yields a 2M huge
page, and a 16K normal page yields a 32M huge page).

For load segments where both the virtual address and file offset are
already aligned to this huge page size (e.g., zero), and the segment
size is large enough, we explicitly bump its alignment to the huge
page size. This optimization significantly reduces TLB pressure and
improves performance for large objects.

To prevent excessive address space padding and memory waste on
architectures with very large normal pages (e.g., 64K pages yielding
512M huge pages), the optimization is capped and only applies if the
calculated huge page size is 32M or less.

* elf/dl-load.c (_dl_map_object_from_fd): Bump p_align_max to the
calculated PMD huge page size (capped at 32M) for large load segments
on 64-bit systems.

Signed-off-by: WANG Rui <wangrui@loongson.cn>
---
 elf/dl-load.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/elf/dl-load.c b/elf/dl-load.c
index 7355eef8e7..a8db4f3abc 100644
--- a/elf/dl-load.c
+++ b/elf/dl-load.c
@@ -1171,6 +1171,23 @@ _dl_map_object_from_fd (const char *name, const char *origname, int fd,
 
 	  /* Optimize a common case.  */
 	  c->prot = pf_to_prot (ph->p_flags);
+
+#ifdef __LP64__
+	  /* Dynamically calculate the PMD huge page size based on the normal
+	     page size (e.g., 2M for 4K pages, 32M for 16K pages). Aligning
+	     load segments that are large enough to the PMD size helps improve
+	     THP eligibility and reduces TLB pressure.
+	     We cap the huge page size at 32M to avoid over-aligning on systems
+	     with very large normal pages (like 64K pages with 512M huge pages). */
+	  size_t page_size = GLRO(dl_pagesize);
+	  size_t huge_page_size = page_size * (page_size / sizeof (void *));
+	  size_t huge_page_mask = huge_page_size - 1;
+	  if (huge_page_size <= (32 * 1024 * 1024)
+	      && ((c->mapstart | c->mapoff) & huge_page_mask) == 0
+	      && (c->mapend - c->mapstart) >= huge_page_size
+	      && p_align_max < huge_page_size)
+	    p_align_max = huge_page_size;
+#endif
 	  break;
 
 	case PT_TLS:
-- 
2.53.0