This is the mail archive of the mailing list for the glibc project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: improving malloc

On 1/6/2013 3:05 PM, OndÅej BÃlka wrote:
> I also want to comment other suggestions from wiki. Here is first
> Chris Metcalf: 
> * Hooks to allow arenas to be created using huge pages (either default
> size or particular sizes)  
> This has problem that huge pages increase memory usage. When application
> overcommit memory then small pages are advantage. 

Yes, so this is something that applications should request explicitly.  For hardware with relatively smaller TLBs, huge pages are relatively more advantageous.

> Second problem is fork. It is wasteful to copy huge page when only small 
> part changes.

Agreed - typically you don't want to COW huge pages unnecessarily.

> My ideas how to address this is either:
> 1) Use new function(say huge_malloc) so programmer is aware of these
> drawbacks.

Alternately, if we want to think about exposing the underlying arena implementation and allow explicit arena creation and allocate-from-arena functionality, we could support requesting huge pages (and other memory attributes where relevant) at the per-arena level.  The dlmalloc code that glibc derives from [1] has an option to expose "mspace_malloc" methods and the like.

> 2) Profiling. One could track how many pages allocations used/ if fork
> happened and then decide if huge pages are profitable. 
> It has drawback that programmer must do profiling. When programmer
> does this he can add suitable mallopt. 

Again, it may be useful to do this at a per-arena level, e.g. mspace_mallopt in dlmalloc.

> Huge pages are much better with garbage collection runtimes as heap
> compactification reduces first problem.

Good point, and probably a good argument for providing an option for process-wide huge page use, e.g. a Java runtime or the equivalent.


Chris Metcalf, Tilera Corp.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]