[RFC] Are HugeTLB pages supported for building and running glibc?

[Adhemerval Zanella Netto]

On 14/08/25 12:27, William Hunt wrote:
> When I enable a small number of HugeTLB pages such as 20 and try to run glibc 
> subdir tests with HugeTLB pages (i.e. glibc.malloc.hugetlb=2) on master branch,
> the system glibc crashes showing SIGBUS errors in stdlib. This happens even for
> subdirs that don't have exclusions for hugetlb2 test variants, such as elf. 
> 
> Even when running with a very large number of HugeTLB pages such as 16384, 
> there are usually some stdlib tests that fail, such as tst-arc4random-thread. 
> Below are some example results for x86-64, where stdlib/tst-arc4random-thread, 
> stdlib/isomac and stdlib/tst-atexit fail.
> 
> 		=== Summary of results ===
>       3 FAIL
>    7040 PASS
>      89 UNSUPPORTED
>      16 XFAIL
>       4 XPASS
> 
> Since tests are ran such that tunables are globally set, I am unsure whether 
> normal tests don't reset to glibc.malloc.hugetlb=0, or whether building and 
> running tests with huge pages was ever supported for glibc. As I'm developing 
> a branch that enables HugeTLB support for mremap and cannot get all tests to
> consistently pass, I would be curious if this is something I can be expected
> to fix?

I think this is an inherent problem on how hugepage reservation works on Linux.
>From Documentation/mm/hugetlbfs_reserv.rst:

  Huge pages as described at Documentation/admin-guide/mm/hugetlbpage.rst are
  typically preallocated for application use.  These huge pages are instantiated
  in a task's address space at page fault time if the VMA indicates huge pages
  are to be used.  If no huge page exists at page fault time, the task is sent
  a SIGBUS and often dies an unhappy death.

My understanding is if concurrent processes allocates more pages than the
available (HugePages_Free) they can be subject to SIGBUS when accessing the pages.
Assuming a system with 512 2MB huge pages reserved:

$ cat /proc/meminfo | grep ^HugePages
HugePages_Total:     512
HugePages_Free:      512
HugePages_Rsvd:        0
HugePages_Surp:        0
$ cat test.c

#include <assert.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/wait.h>
#include <unistd.h>
#include <stdint.h>
#include <stdio.h>
#include <signal.h>

static const size_t hugepage_size = 2097152;
static const size_t npages = 512;
static const size_t pages_size = npages * hugepage_size;

static char *pages;

static void
sigbus_handler (int s)
{
  char msg[512];
  ssize_t msgsize = snprintf (msg, sizeof msg, "[%jd] SIGBUS\n",
                              (intmax_t) getpid ());
  write (STDERR_FILENO, msg, msgsize);
  _exit (EXIT_FAILURE);
}

static void
touch_pages ()
{
  for (int i = 0; i < npages; i++)
    pages[i*hugepage_size] = 0xcc;
  printf ("[%jd] touch_pages\n", (intmax_t) getpid ());
}

int main ()
{
  sigaction (SIGBUS, &(struct sigaction) { .sa_handler = sigbus_handler },
             NULL);
  pages = mmap (NULL,
                pages_size,
                PROT_READ | PROT_WRITE,
                MAP_ANONYMOUS | MAP_PRIVATE | MAP_HUGETLB,
                0,
                0);
  assert (pages != MAP_FAILED);

  pid_t pid = fork ();
  assert (pid != -1);

  touch_pages ();

  if (pid != 0)
    waitpid (pid, NULL, 0);
}
$ ./t
[4034938] SIGBUS
[4034937] touch_pages

You can mitigate this to add overcommit pages hugepages to the pool:

$ echo 512 | sudo tee /proc/sys/vm/nr_overcommit_hugepages
$ cat /proc/meminfo | grep ^HugePages
HugePages_Total:     512
HugePages_Free:      512
HugePages_Rsvd:        0
HugePages_Surp:        0
$ ./t
[4035239] touch_pages
[4035238] touch_pages

The Documentation/admin-guide/mm/hugetlbpage.rst have a lot of more information.

It also seems that MAP_HUGETLB does not seems to be influenced by memory
overcommit (vm.overcommit_memory = 2), at least it seems that COW is always
enabled even at fork (meaning that SIGBUG will be generated for any
value of vm.overcommit_memory).