[PATCH] malloc: Do not call madvise if heap's oldsize >= THP size

[Dev Jain]

Linux handles virtual memory in Virtual Memory Areas (VMAs). The
madvise(MADV_HUGEPAGE) call works on a VMA granularity, which sets the
VM_HUGEPAGE flag on the VMA. This flag is invariant of the mprotect()
syscall which is used in growing the secondary heaps. Therefore, we
need to call madvise() only when we are sure that VM_HUGEPAGE was not
previously set, which is only in the case when h->size < mp_.thp_pagesize.
---
 malloc/arena.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/malloc/arena.c b/malloc/arena.c
index 4cd79d7244..2551cb7749 100644
--- a/malloc/arena.c
+++ b/malloc/arena.c
@@ -482,7 +482,10 @@ grow_heap (heap_info *h, long diff)
       h->mprotect_size = new_size;
     }
 
-  madvise_thp (h, new_size);
+  /* mprotect preserves MADV_HUGEPAGE semantics - this means that if the old
+     region was marked with MADV_HUGEPAGE, the new region will retain that.  */
+  if (h->size < mp_.thp_pagesize)
+    madvise_thp (h, new_size);
 
   h->size = new_size;
   LIBC_PROBE (memory_heap_more, 2, h, h->size);
-- 
2.43.0