| Summary: | missing call or documentation for malloc_trim() ... | ||
|---|---|---|---|
| Product: | glibc | Reporter: | Sandor Kovacs <sakovacs> |
| Component: | libc | Assignee: | Ulrich Drepper <drepper.fsp> |
| Status: | RESOLVED INVALID | ||
| Severity: | normal | CC: | glibc-bugs |
| Priority: | P2 | Flags: | fweimer:
security-
|
| Version: | 2.3.5 | ||
| Target Milestone: | --- | ||
| Host: | x86_64-pc-linux-gnu | Target: | x86_64-pc-linux-gnu |
| Build: | x86_64-pc-linux-gnu | Last reconfirmed: | |
*** Bug 2530 has been marked as a duplicate of this bug. *** Why don't you make your little program write out the data:
#include <unistd.h>
#include <stdlib.h>
#include <malloc.h>
#include <stdio.h>
int
main (int argc, char *argv[])
{
char cmd[100];
snprintf (cmd, sizeof (cmd), "grep heap /proc/%d/maps || echo no heap",
getpid ());
void *before = sbrk (0);
system (cmd);
int max = 10000000;
char **p1 = (char **) malloc (max * sizeof (char *));
for (int x = 0; x < max; x++)
{
p1[x] = (char *) malloc (29);
}
for (int x = 0; x < max; x++)
{
free (p1[x]);
}
free (p1);
void *middle = sbrk (0);
system (cmd);
malloc_trim (0);
void *after = sbrk (0);
system (cmd);
printf ("before = %p, middle = %p, after = %p\n", before, middle, after);
return 0;
}
When I run this I see
no heap
00501000-1cee1000 rw-p 00501000 00:00 0 [heap]
00501000-00502000 rw-p 00501000 00:00 0 [heap]
before = 0x501000, middle = 0x1cee1000, after = 0x502000
This clearly shows that
a) libc returns all memory if malloc_trim is used
b) the kernel frees all memory.
So, what's the problem?
> a) libc returns all memory if malloc_trim is used I agree. That's what I said too: >... if you call malloc_trim(0) right before the usleep() the RSS goes back to > less than 1M (this is a simple case, no fregmentation, all memory could be > returned to the OS). b) the kernel frees all memory. I agree with this as well. > So, what's the problem? All I was saying is that either put malloc_trim() should be referenced in the malloc/free man page or free() should call it from time to time -- I since read that this should be the case. So a simple "see also" entry would help a lot for many guys having similar issues. Cheers, Sandor The man pages are not maintained in glibc. Tell this to the man page maintainer. |
Let me describe my issue and you can decide whether it is a bug in libc or just missing documentation on the man pages. I've spent a couple of weeks debugging my program, it processed big files and allocated lots of small memory blocks in the process, free()'d the memory afterwards and RSS still didn't decrease. I obviously thought it was a memory leak, valgrinded, etc, until I came down to this very simple program: #include <unistd.h> #include <malloc.h> int main(int argc, char *argv[]) { int max = 10000000; char** p1 = (char**) malloc(max * sizeof(char*)); for (int x = 0; x < max; x++) { p1[x] = (char*) malloc(29); } for (int x = 0; x < max; x++) { free(p1[x]); } free(p1); usleep(20000 * 1000); // so that we have time to check the RSS.. return 0; } This program runs up to ~500Mb on my system, and even after free()'ing the RSS does not shrink. I think for most of you it is obvious now, that the issue is that free() does not return memory to the system. Unfortunately on the man page there is no reference to this behaviour, also no reference to malloc_trim() call that can return some of the memory (depending on the fragmentation). In this example code, if you call malloc_trim(0) right before the usleep() the RSS goes back to less than 1M (this is a simple case, no fregmentation, all memory could be returned to the OS). Now, I know that malloc_trim() does not guarantee the return of any memory, but I would think that A) the linux free() implementation should call it from time to time (based on the allocated/free memory ratio?.. I'm not the proper person to define such an algorithm) B) the man pages should contain the fact that free() never returns allocated pages to the OS and that on linux malloc_trim() call can be used to try to return pages. I just would like to avoid others spending hours debugging their programs while there is nothing to debug. Looking forward to hear your opinion, Cheers, Sandor