[Bug libc/3455] New: ptmalloc fails if foreign sbrk and no mmap and size is greater than trim threshold

["cseddon at sequencedesign dot com" <sourceware-bugzilla at sourceware dot org>]

While ensuring that a program built on RHL72 would run on RHEL3, I discovered
that the program would run out of memory prematurely.

I obtained a version of ptmalloc, and, while debugging, determined that a call
to _int_free() from sYSMALLOc() was being made to free up old_top().  This
results in a call to sYSTRIm() to trim off the any unused memory.  However, in
that flow, it ends up clearing out all of the newly-allocated memory that was
just requested.

(The MMAP base note is due to an internal requirement, and the 65535 value is
 one taken from a third-party's sbrk() call, which caused our problem)

Here is a sample program:
#include <malloc.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h> // sbrk

const int oneMeg = 1024*1024;
//const int oneMeg = 1024;
void* ourMalloc(size_t size) {
    void* retVal = malloc(size);
    if (retVal > (void*)0x80000000) {
        printf("MMAP base exceeded\n");
    }
    return retVal;
}

main() {
    void* p;
    void* firstP = 0;
    void* lastP;
    int totalAllocs = 0;
    mallopt(M_MMAP_MAX, -1);
    while (p = malloc(oneMeg)) {
        if (!firstP) {
            firstP = p;
            // introduce a 'foreign sbrk'
            void* brk = sbrk(0);
            printf("b4 sbrk: %p\n", brk);
            sbrk(65535);
            brk = sbrk(0);
            printf("after sbrk: %p\n", brk);
        }
        lastP = p; 
        totalAllocs++;
        memset(p, 'a', oneMeg);
    }
    printf("started @ %p\n", firstP);
    printf("ended @ %p\n", lastP);
    printf("total: %u\n", totalAllocs * oneMeg);
}

This programs runs fine on RHL72:

cseddon@power-linux-01:~ % ./a.out
b4 sbrk: 0x804988c
after sbrk: 0x805988b
started @ 0x40164008
ended @ 0x3fe5b488
total: 3075473408

On RHEL3, however:

cseddon@platform-rhel3-01:~ % ~/a.out
b4 sbrk: 0x816b000
after sbrk: 0x817afff
started @ 0x8049898
ended @ 0x8049898
total: 1048576

And, when run with trimming shut off it again allocates the max memory:

cseddon@platform-rhel3-01:~ % setenv MALLOC_TRIM_THRESHOLD_ -1
cseddon@platform-rhel3-01:~ % a.out
b4 sbrk: 0x816b000
after sbrk: 0x817afff
started @ 0x8049898
ended @ 0xbfe9a008
total: 3068133376


Our workaround is to shut off trimming usin mallopt(M_TRIM_THRESHOLD, -1)

-charlie

-- 
           Summary: ptmalloc fails if foreign sbrk and no mmap and size is
                    greater than  trim threshold
           Product: glibc
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
        AssignedTo: drepper at redhat dot com
        ReportedBy: cseddon at sequencedesign dot com
                CC: glibc-bugs at sources dot redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=3455

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.