Malloc Problem in glibc 2.2

[Wolfram Gloger]

Hello,

> looking at a problem with getrlimit/setrlimit, I noticed that malloc
> in glibc 2.2 doesn't work as expected.  The appended program gives
> with a glibc 2.1 system:
> 
> gromit:/tmp/pr:[0]$ ./rlimit 
> rlim_cur 1240, rlim_max: 2147483647
> malloc returned: (nil)
> 
> But the current glibc 2.2 version does:
> $ LD_LIBRARY_PATH=. elf/ld-linux.so.2 /tmp/pr/rlimit
> rlim_cur 1240, rlim_max: 2147483647
> malloc returned: 0x40117008

I consider this a feature, not a bug :-).  The increasing bunch of
lucky people with >2GB 32bit systems will probably agree with me, too.
All you have done with rlimit is to limit the `classical' heap even
further than the `normal' 1GB, so the non-linear heap is now used.

> Why is glibc 2.2 using mmap and glibc 2.1 not?

If sbrk() would fail on 2.1, there could only be n_mmaps_max further
allocations.  No matter how large we would increase n_mmaps_max, I'm
sure someone would complain that her machine still can't allocate all
the available memory.

So, I changed the meaning of n_mmaps_max to mean the following limit:

In 2.1:
n_mmaps_max was a hard limit on the number of chunks alocated with
mmap().  Beyond the limit, if sbrk() failed, malloc would return 0.

In 2.2:
Until n_mmaps_max is reached, mmap() is used in preference of sbrk().
When n_mmaps_max is reached or passed, sbrk() is _always_ tried first.
However, mmap() is now always tried as a last resort before giving up
and returning 0.

So, the parameter is still useful for avoiding mmap() on a system
where it's very costly (Linux surely isn't such a system), but it
doesn't limit the total number of chunks that can be allocated.

> Do you have any idea what's broken here?  This is with Linux
> 2.4.0-test4, a current gcc (one or two weeks old) and the current
> glibc sources.
...
>   rlim.rlim_cur = 1240;
>   assert(-1 != setrlimit(RLIMIT_DATA, &rlim));

IMHO RLIMIT_DATA is totally useless to limit the amount of memory
allocated by a process, as it can be easily circumvented with mmap().
Why should malloc() obey that limit?  Also, it didn't do so in 2.1,
either.

The only effective rlimit to control memory usage is RLIMIT_AS.

Regards,
Wolfram.