free() does not physically trim/reclaim memory

[Shuxin Yang]

Hi, Goffredo:

     Thank you very much for the speedy response. Please see the 
following interleaving comment.

On 09/01/2016 02:14 PM, Goffredo Baroncelli wrote:
> On 2016-09-01 20:24, shuxin yang wrote:
>> Hi, There:
>>
>>      My environment is Linux with "Ubuntu GLIBC 2.21-0ubuntu4" (this is what ldd --version gives).
>>
>>      In my application, I need to call mmap() a block right after BSS in order to prevent heap from growing.
>> Then, the subsequent "malloc(not-very-big-size)" is to carve a block from a mmap()-ed block.
>> It seems to me that the corresponding free() does not physically reclaim the memory unless I
>> explicitly call malloc_trim().
>>
>>      Could you please shed some light on this issue?
>>
>>      I reproduce the problem with the following snippet, and observe RSS size using command
>> "smem  -P "a\.out"
> Disclaimer: I am not a libc expert; I have only played a bit with your code.
>
> 1) free(3) calls malloc_trim(3) if the block to free is greater than 128K. See mallopt(3), and the M_TRIM_THRESHOLD parameter. If you set M_TRIM_THRESHOLD to 126K or you allocate block greater 128k, you will observer the behavior that you expect.
It somehow does not work at my side: none of following approaches works:
   - invoke the example by : LLOC_TRIM_THRESHOLD_=$((126 * 1024)) ./a.out
   - call 'mallopt(M_TRIM_THRESHOLD, 126 * 1024)' right after main() is 
entered.

Am I missing something here?

If I set MALLOC_MMAP_THRESHOLD_ to 126k, then the memory can be 
reclaimed immediately.
(As far as I can understand the code, it is because each malloc(127k) 
ends up mmap() a "chunk"
  flagged as "mapped chunk", when free() is called, "mapped chunk" can 
be easily deallocated)

>
> 2) this is a minor thing: it is not sufficient to allocate memory with malloc, but you have also to access it in order to make a real allocation.
> If you add a memset(p[i], 0, 127*1024); after the malloc(), you can see that the RSS go from about 17MB to about 500MB. In fact 4096*127*1024 = 508MB.
>
Yes, you are absolutely right!

Thanks

Shuxin