Re: RFC: replace ptmalloc2

[Rich Felker <dalias at libc dot org>]

On Thu, Oct 09, 2014 at 06:07:43PM -0700, JÃrn Engel wrote:
> > >   often called in 4k-granularities.  Each call takes the mmap_sem
> > >   writeable and potentially splits off new vmas.  Way too expensize to
> > >   do in small granularities.
> > >   It gets better when looking at the other arenas.  Memory is
> > >   allocated via mmap(PROT_NONE), so every mprotect() will split off
> > >   new vmas.  Potentially some of them can get merged later on.  But
> > >   current Linux kernels contain at least one bug, so this doesn't
> > >   always happen.
> > >   If someone is arguing in favor of PROT_NONE as a debug- or
> > >   security-measure, I wonder why we don't have the equivalent for the
> > >   main arena.  Do we really want the worst of both worlds?
> > 
> > mprotect usage is not just a diagnostic or security measure, it is
> > primarily there to reduce the commit charge of the process.  This is
> > what keeps the actual memory usage low for processes despite having
> > large address space usage.
> 
> Interesting.  I assume that "commit charge" would be memory that shows
> up as VmRSS in /proc/$PID/status.  In that case, how does mprotect usage
> reduce it?  Mmap() returns virtual memory without any mapping,
> individual pages get faulted in on demand and mprotect shouldn't make
> any difference.
> 
> Clearly there is something here I don't understand.  At least I hope so.

Commit charge is the way you account for memory usage on a system with
virtual memory in order that you don't run out of physical backing to
instantiate all virtual memory that gets allocated. In the default
Linux configuration without strict commit accounting
(vm.overcommit_memory==0), it's just part of a heuristic, but with
overcommit disabled (vm.overcommit_memory==2), it puts a hard limit on
the amount of memory you can allocate. Any writable (or
previously-writable-and-modified) private mapping, or shared memory
that's not backed by a file/device, contributes to commit charge.

Rich