What are the advantages and disadvantages if we always use mmap to allocate memory instead of malloc ?

[Siddhesh Poyarekar]

On Wed, 12 Aug 2020 at 17:05, 孙世龙 sunshilong via Libc-help
<libc-help@sourceware.org> wrote:
> As per the Linux Programmer Manual, which says:
> For  allocations  greater than or equal to the limit specified (in bytes)
> by M_MMAP_THRESHOLD that can't be satisfied from the free list,
> the memory-allocation functions employ mmap(2) instead of
> increasing the program break using sbrk(2).
>
> I can draw the conclusion that malloc(3) may invoke mmap(2) when
> allocating a huge block memory.

That description is many years old.  It is kinda the same in
principle, i.e. allocation from the heap vs allocation of a single
block of memory using mmap can be controlled by M_MMAP_THRESHOLD, but
the definition of the "heap" has changed in glibc over the years.

The "heap" in question is called an arena in glibc parlance and it can
be expanded or contracted either by brk() or by mmap() depending on
various factors.  The key point here though is not which syscall is
used for allocation, but that a syscall is used at all.

> So I think there may be something wrong with your conclusion(i.e.
> mmap calls malloc to allocate a big block).

I think he said the opposite, i.e. malloc calls mmap to allocate a
single big block.

> What do you think about it? Am I missing something?

I think the point you're missing is the fact that the key performance
benefit of using malloc is the possibility of getting allocations
without having to go through a syscall at all.  System calls are quite
expensive because they involve a context switch into the kernel and
back, so malloc economizes those calls by requesting large blocks of
memory at once and then giving out smaller blocks from it whenever a
user calls malloc.  For very large blocks of memory (> 128K), it is
generally more optimal to serve it with a dedicated mmap'd block of
its own instead of giving it out from the cached blocks.

When you replace malloc with mmap, it would mean that *all* your
allocations will be served by mmap, meaning that every single
allocation call would result in a system call, thus potentially
reducing performance.  Further, even if you request 16 bytes from the
kernel using mmap, you'll get a minimum of a page (i.e. 4K or 64K
depending on the architecture), thus wasting the rest of the
allocation.

The example application you cited could be a case where the program is
doing its own memory management, i.e. requesting large blocks from the
kernel and then handing out small blocks from it to different parts of
the program.

Siddhesh
-- 
http://siddhesh.in