This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: improving malloc
On 1/6/2013 3:05 PM, OndÅej BÃlka wrote:
> I also want to comment other suggestions from wiki. Here is first
>
> Chris Metcalf:
> * Hooks to allow arenas to be created using huge pages (either default
> size or particular sizes)
>
> This has problem that huge pages increase memory usage. When application
> overcommit memory then small pages are advantage.
Yes, so this is something that applications should request explicitly. For hardware with relatively smaller TLBs, huge pages are relatively more advantageous.
> Second problem is fork. It is wasteful to copy huge page when only small
> part changes.
Agreed - typically you don't want to COW huge pages unnecessarily.
> My ideas how to address this is either:
>
> 1) Use new function(say huge_malloc) so programmer is aware of these
> drawbacks.
Alternately, if we want to think about exposing the underlying arena implementation and allow explicit arena creation and allocate-from-arena functionality, we could support requesting huge pages (and other memory attributes where relevant) at the per-arena level. The dlmalloc code that glibc derives from [1] has an option to expose "mspace_malloc" methods and the like.
> 2) Profiling. One could track how many pages allocations used/ if fork
> happened and then decide if huge pages are profitable.
>
> It has drawback that programmer must do profiling. When programmer
> does this he can add suitable mallopt.
Again, it may be useful to do this at a per-arena level, e.g. mspace_mallopt in dlmalloc.
> Huge pages are much better with garbage collection runtimes as heap
> compactification reduces first problem.
Good point, and probably a good argument for providing an option for process-wide huge page use, e.g. a Java runtime or the equivalent.
[1] ftp://g.oswego.edu/pub/misc/malloc.c
--
Chris Metcalf, Tilera Corp.
http://www.tilera.com