Re: malloc using mprotect

[Jakub Jelinek <jakub at redhat dot com>]

On Fri, Aug 10, 2001 at 11:12:31AM +0200, Wolfram Gloger wrote:
> Hello,
> 
> > What's the exact reason for using mmap(..., PROT_NONE, ...) first and then
> > mprotect (..., PROT_READ|PROT_WRITE) chunk by chunk instead of
> 
> It's actually happening in rather bis chunks, _not_ individually for
> every malloc()ed chunk larger than a page.

I tried
while (1) malloc(1024);
and once it eated the brk heap, the syscall sequence was mmap, munmap half o
f it, mprotect first 32K, then mprotect 4K with increasing addresses, then
again mmap etc.

> > mmap(..., PROT_READ|PROT_WRITE, ...) first and nothing afterwards?
> > Is it primarily to catch bogus programs?
> 
> No, not primarily.
> 
> > Or is it primarily so that the whole heap does not count immediately into
> > rlimits, but slowly as malloc areas are requested?
> 
> Yes, that is one of the main reasons.  After all, the heaps are quite
> huge (or at least were quite huge in 1996 :).
> 
> > If we explain it to Linus, I guess he could accept mprotect(2) patches to do
> > some merging in the common case.
> 
> What exactly is the problem?

mprotect does not do any segment merging in 2.4, so each single page
consumes one vma. It is slower to have 65000 vmas than say 50 vmas which are
really needed, but the main problem is that although you have enough memory,
you really cannot malloc all you could without mprotect - say with the above
test on 4GB IA-32, you can malloc some 1.2GB, although you should be able to
malloc 2.8GB. The number of vmas has upper limit (2<<16 in Linus kernel's,
in -ac kernels it is just the default and it is adjustable through
/proc/sys/vm/max_map_count).
But Linus does not want to do anything with mprotect merging unless we
convince him why glibc needs to do the mprotect stuff.

	Jakub