This is the mail archive of the
mailing list for the glibc project.
Re: improving malloc
On 1/6/2013 3:05 PM, OndÅej BÃlka wrote:
> I also want to comment other suggestions from wiki. Here is first
> Chris Metcalf:
> * Hooks to allow arenas to be created using huge pages (either default
> size or particular sizes)
> This has problem that huge pages increase memory usage. When application
> overcommit memory then small pages are advantage.
Yes, so this is something that applications should request explicitly. For hardware with relatively smaller TLBs, huge pages are relatively more advantageous.
> Second problem is fork. It is wasteful to copy huge page when only small
> part changes.
Agreed - typically you don't want to COW huge pages unnecessarily.
> My ideas how to address this is either:
> 1) Use new function(say huge_malloc) so programmer is aware of these
Alternately, if we want to think about exposing the underlying arena implementation and allow explicit arena creation and allocate-from-arena functionality, we could support requesting huge pages (and other memory attributes where relevant) at the per-arena level. The dlmalloc code that glibc derives from  has an option to expose "mspace_malloc" methods and the like.
> 2) Profiling. One could track how many pages allocations used/ if fork
> happened and then decide if huge pages are profitable.
> It has drawback that programmer must do profiling. When programmer
> does this he can add suitable mallopt.
Again, it may be useful to do this at a per-arena level, e.g. mspace_mallopt in dlmalloc.
> Huge pages are much better with garbage collection runtimes as heap
> compactification reduces first problem.
Good point, and probably a good argument for providing an option for process-wide huge page use, e.g. a Java runtime or the equivalent.
Chris Metcalf, Tilera Corp.