RFC: replace ptmalloc2

[Eric Wong]

Will Newton <will.newton@linaro.org> wrote:
> On 17 October 2014 10:03, Eric Wong <normalperson@yhbt.net> wrote:
> > * Ability to take advantage of THP, but not inflate memory usage.
> >   This is important for folks who run many threads w/o overcommit.
> >   I think software like MySQL with 10s-100s of threads on a handful of
> >   CPUs is here to stay.  Getting folks to remember knobs like
> >   MALLOC_ARENA_MAX is annoying and tiring.
> 
> Do you have an idea in mind how malloc could integrate with THP?

(my untested/unverified ideas)

There should probably be two levels of allocation:
1. for large requests to the OS (>= 2M for x86-64)
2. per-thread arenas taking memory from the global pool

So the goal would be to keep as many >= 2M contiguous chunks as possible
and avoid unmapping smaller chunks (or maybe any middle chunks) which
can generate holes.

Per-thread arenas should avoid mmap/munmap/sbrk syscalls directly,
because:
* they'd encounter mmap_sem contention in the kernel anyways
* they lose the ability (in dlmalloc 2.8, at least) to detect+merge
  contiguous MAP_ANON segments from separate mmap calls.

dlmalloc (2.8 at least) tries to detect contiguous MAP_ANON segments
from separate mmap calls and merge them.  This usually works well with
Linux (unless the underlying application is doing its own MAP_ANON).