This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: RFC: replace ptmalloc2


On Thu, Oct 09, 2014 at 02:54:47PM -0700, JÃrn Engel wrote:
> Possible improvements found by source code inspection and via
> testcases:
> - Everything mentioned in
>   https://www.facebook.com/notes/facebook-engineering/scalable-memory-allocation-using-jemalloc/480222803919
> - Arenas are a bad choice for per-thread caches.
> - mprotect_size seems to be responsible for silly behaviour.  When
>   extending the main arena with sbrk(), one could immediately
>   mprotect() the entire extension and be done.  Instead mprotect() is
>   often called in 4k-granularities.  Each call takes the mmap_sem
>   writeable and potentially splits off new vmas.  Way too expensize to
>   do in small granularities.

FYI locklessinc.com's demo showing that their allocator is better than
glibc's depends on this bug. So they might be unhappy if it's fixed.
;-)

>   It gets better when looking at the other arenas.  Memory is
>   allocated via mmap(PROT_NONE), so every mprotect() will split off
>   new vmas.  Potentially some of them can get merged later on.  But
>   current Linux kernels contain at least one bug, so this doesn't
>   always happen.

This was a serious bug in some kernels that renders them unusable. It
only occurs if a particular config option related to checkpointing is
enabled. The kernel's intent is that adjacent anonymous VMA's ALWAYS
be merged into a single VMA; if this doesn't happen it's a kernel bug,
and glibc should not pessimize other aspects to work around it.

>   If someone is arguing in favor of PROT_NONE as a debug- or
>   security-measure, I wonder why we don't have the equivalent for the
>   main arena.  Do we really want the worst of both worlds?

It's neither debug nor security. It's the only way you can get
contiguous memory without preallocating it (and thereby consuming
commit charge).

Rich


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]