Memory leak in gcc 3.4.4 / ARM with mplayer

Robert Schwebel
Tue Jan 3 21:01:00 GMT 2006


I observed some kind of a memory leak while gcc-3.4.4, built with
crosstool-0.38 for ARM, is trying to compile libavcodec/snow.c,
especially this part: 

static void spatial_decompose97i(DWTELEM *buffer, int width, int height, int stride){
    int y;
    DWTELEM *b0= buffer + mirror(-4-1, height-1)*stride;
    DWTELEM *b1= buffer + mirror(-4  , height-1)*stride;
    DWTELEM *b2= buffer + mirror(-4+1, height-1)*stride;
    DWTELEM *b3= buffer + mirror(-4+2, height-1)*stride;

    for(y=-4; y<height; y+=2){
        DWTELEM *b4= buffer + mirror(y+3, height-1)*stride;
        DWTELEM *b5= buffer + mirror(y+4, height-1)*stride;

        if(b3 <= b5)     horizontal_decompose97i(b4, width);
        if(y+4 < height) horizontal_decompose97i(b5, width);

        if(b3 <= b5) vertical_decompose97iH0(b3, b4, b5, width);
        if(b2 <= b4) vertical_decompose97iL0(b2, b3, b4, width);
        if(b1 <= b3) vertical_decompose97iH1(b1, b2, b3, width);
        if(b0 <= b2) vertical_decompose97iL1(b0, b1, b2, width);



void ff_spatial_dwt(DWTELEM *buffer, int width, int height, int stride, int type, int decomposition_count){
    int level;

    for(level=0; level<decomposition_count; level++){
        case 0: spatial_decompose97i(buffer, width>>level, height>>level, stride<<level); break;
        case 1: spatial_decompose53i(buffer, width>>level, height>>level, stride<<level); break;
        case 2: spatial_decomposeX  (buffer, width>>level, height>>level, stride<<level); break;

When the compiler (cc1) hits the code in ff_spatial_dwt(), memory
consumption goes up until the OOM killer does it's work. This was not
the case with gcc-3.3.3, built with crosstool as well. 

Disabling one of the "case [0|1|2]" lines in ff_spatial_dwt() makes the
bug go away, as well as changing the optimisation from -O4 to -O2 (-O3
does still crash). 

I'm not sure if it is reproducable with patchless compilers, but before
I try it and spend hours with toolchain compiling, I'd like to ask if
anyone has seen a similar problem before. 

 Dipl.-Ing. Robert Schwebel |
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9

Want more information?  See the CrossGCC FAQ,
Want to unsubscribe? Send a note to

More information about the crossgcc mailing list