Bug 23489 - "gcc -lmcheck" aborts on free when using posix_memalign
Summary: "gcc -lmcheck" aborts on free when using posix_memalign
Status: RESOLVED FIXED
Alias: None
Product: glibc
Classification: Unclassified
Component: libc (show other bugs)
Version: 2.19
: P2 normal
Target Milestone: 2.34
Assignee: Siddhesh Poyarekar
URL:
Keywords:
Depends on:
Blocks: 22050 28068
  Show dependency treegraph
 
Reported: 2018-08-06 23:54 UTC by ben
Modified: 2021-07-22 14:34 UTC (History)
6 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed: 2018-08-07 00:00:00
fweimer: security-


Attachments
minimal example (.i version) (6.35 KB, text/plain)
2018-08-06 23:54 UTC, ben
Details
readelf a.out (3.70 KB, text/plain)
2018-08-10 05:42 UTC, Manuel Lauss
Details
readelf libc-2.28.so (54.53 KB, text/plain)
2018-08-10 05:43 UTC, Manuel Lauss
Details
readelf libmcheck.a (1.33 KB, text/plain)
2018-08-10 05:43 UTC, Manuel Lauss
Details

Note You need to log in before you can comment on or make changes to this bug.
Description ben 2018-08-06 23:54:22 UTC
Created attachment 11166 [details]
minimal example (.i version)

Created attachment 44513 [details]
Minimal example (.i version)

Simple C programs using `posix_memalign()` and `free()`, compiled with `gcc -lmcheck`, cause the resulting binary to raise a runtime error such as `munmap_chunk(): invalid pointer`.

For example,

    #include <stdlib.h>
    #include <stdio.h>

    int main()
    {
        void *p = 0;
        int ret = posix_memalign(&p, sizeof(void*), 0);
        printf("ret was %d, pointer is %p\n", ret, p);
        free(p);
        return 0;
    }


The full example including headers is attached.

Note that "POSIX requires that memory obtained from posix_memalign() can be  freed using free".

Output (my system):

    $ gcc test.c -lmcheck -ggdb
    $ ./a.out
    ret was 0, pointer is 0xcb6090
    *** Error in `./a.out': munmap_chunk(): invalid pointer: 0x0000000000cb6060 ***
    Aborted

Output (NVIDIA developer):

    $ gcc test.c -lmcheck -ggdb
    $ ./a.out
    ret was 0, pointer is 0x558c32d102e0
    munmap_chunk(): invalid pointer
    Aborted (core dumped)

Expected output:

    $ gcc test.c -lmcheck -ggdb
    $ ./a.out
    ret was 0, pointer is 0x<some-address>

This was initially discovered due to posix_memalign's use in the Nvidia graphics driver -- discussion here, three people report same symptoms: https://devtalk.nvidia.com/default/topic/1037171/linux/memory-error-in-glx-programs-when-linked-with-gcc-lmcheck/

For me, here is the GCC version that raises this. I am aware this is an old version of GCC as provided by what is currently Debian odstable (Jessie), but others have reported the same results.

    $ gcc -v
    Using built-in specs.
    COLLECT_GCC=gcc
    COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.9/lto-wrapper
    Target: x86_64-linux-gnu
    Configured with: ../src/configure -v --with-pkgversion='Debian 4.9.2-10+deb8u1' --with-bugurl=file:///usr/share/doc/gcc-4.9/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.9 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.9 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.9-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.9-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.9-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --with-arch-32=i586 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
    Thread model: posix
    gcc version 4.9.2 (Debian 4.9.2-10+deb8u1) 

Obviously, I am using an old version of GCC/glibc but I hope this bug report may be of use to anyone using a search engine to lookup a problem with this possible lmcheck false positive.
Comment 1 ben 2018-08-06 23:56:54 UTC
Forgot to add

$ ldd --version
ldd (Debian GLIBC 2.19-18+deb8u10) 2.19
Comment 2 Carlos O'Donell 2018-08-07 00:07:07 UTC
(In reply to ben from comment #1)
> Forgot to add
> 
> $ ldd --version
> ldd (Debian GLIBC 2.19-18+deb8u10) 2.19

This is quite old.

With gcc 7, and glibc 2.26 I don't see anything wrong with -lmcheck.

This could be a compiler bug in your distribution, please file this with Debian.
Comment 3 Hussam Al-Tayeb 2018-08-07 05:36:56 UTC
I see this bug with gcc version 7.3.1 20180804 and glibc 2.27 (using a custom Arch Linux derivative).
Comment 4 Hussam Al-Tayeb 2018-08-07 05:39:30 UTC
An NVIDIA developer confirmed he reproduces the same issue here as well. https://devtalk.nvidia.com/default/topic/1037171/linux/memory-error-in-glx-programs-when-linked-with-gcc-lmcheck/post/5274323/#5274323
Comment 5 Carlos O'Donell 2018-08-07 11:28:12 UTC
(In reply to Hussam Al-Tayeb from comment #3)
> I see this bug with gcc version 7.3.1 20180804 and glibc 2.27 (using a
> custom Arch Linux derivative).

Are you certain you can reproduce this with 2.27?

We fixed this in 2.27, which was a bug that existed from 2.24->2.26:
https://sourceware.org/bugzilla/show_bug.cgi?id=22050

Can you debug your program and report the value of __malloc_initialize_hook when you are within a malloc API function?

e.g.

Breakpoint 2, main () at test.c:6
6	        void *p = 0;
(gdb) c
Continuing.
(gdb) p __malloc_initialize_hook
$1 = (void (*)(void)) 0x400610 <turn_on_mcheck>
(gdb)

It really looks like something in the hook setup is wrong.
Comment 6 Hussam Al-Tayeb 2018-08-07 11:40:34 UTC
Do I need a debug build of glibc for this?
I am getting

(gdb) run
Starting program: /home/hussam/cache/workspace/c/a.out 

Breakpoint 1, main () at memalign.c:5
5	    void *p = 0;
(gdb) c
Continuing.

Breakpoint 2, main () at memalign.c:6
6	    int ret = posix_memalign(&p, sizeof(void*), 0);
(gdb) p __malloc_initialize_hook
No symbol "__malloc_initialize_hook" in current context.
(gdb)
Comment 7 Hussam Al-Tayeb 2018-08-07 11:45:41 UTC
/usr/bin/ldd --version
ldd (GNU libc) 2.27
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper

This revision in particular https://sourceware.org/git/?p=glibc.git;a=commit;h=2dab17550dd464bcf531088efa81a9d595167f2a
Comment 8 Manuel Lauss 2018-08-07 18:24:24 UTC
Is anything special required for mcheck to work (glibc config option, ..)?

gentoo glibc-2.29 (git 92a4cba76076), gcc-8.2:

starting program: ./a.out 

Breakpoint 1, main () at test.c:6
6               void *p = 0;
(gdb) p __malloc_initialize_hook
$1 = (void (*)(void)) 0x0
(gdb) c
Continuing.
ret was 0, pointer is 0x4052e0
munmap_chunk(): invalid pointer

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50        return ret;


$ /lib64/libc.so.6 
GNU C Library (Gentoo 2.28 p1) stable release version 2.28.
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 8.2.0.
libc ABIs: UNIQUE IFUNC ABSOLUTE
Comment 9 Carlos O'Donell 2018-08-07 20:09:37 UTC
(In reply to Manuel Lauss from comment #8)
> Is anything special required for mcheck to work (glibc config option, ..)?

No.

> gentoo glibc-2.29 (git 92a4cba76076), gcc-8.2:
> 
> starting program: ./a.out 
> 
> Breakpoint 1, main () at test.c:6
> 6               void *p = 0;
> (gdb) p __malloc_initialize_hook
> $1 = (void (*)(void)) 0x0

This is odd, it's as if mcheck was never setup in the first place. I wonder if you have some other hook setting this to zero.

Can you debug the *first* malloc called from libc.so.6? You'll have to skip forward until you're calling malloc from malloc.c and not dl-minimal.c (bootstrap malloc).

You really want to put a breakpoint on 'turn_on_mcheck', and if it never triggers, then you've got a toolchain problem in gentoo.

What do your test binary and libc.so.6 look like?

readelf -a -W test | grep malloc
    70: 0000000000601040     8 OBJECT  GLOBAL DEFAULT   23 __malloc_initialize_hook

readelf -a -W /lib64/libc.so.6 | grep malloc_initialize
00000000003afe78  0000072000000006 R_X86_64_GLOB_DAT      00000000003b28d0 __malloc_initialize_hook@GLIBC_2.2.5 + 0
  1824: 00000000003b28d0     8 OBJECT  WEAK   DEFAULT   35 __malloc_initialize_hook@GLIBC_2.2.5
  3662: 00000000003b0244     4 OBJECT  LOCAL  DEFAULT   34 __libc_malloc_initialized
  6226: 00000000003b28d0     8 OBJECT  WEAK   DEFAULT   35 __malloc_initialize_hook@GLIBC_2.2.5
Comment 10 Manuel Lauss 2018-08-08 08:31:48 UTC
$ readelf -W -a a.out | grep malloc
     8: 0000000000404050     8 OBJECT  GLOBAL DEFAULT   23 __malloc_initialize_hook@GLIBC_2.2.5
    43: 0000000000404050     8 OBJECT  GLOBAL DEFAULT   23 __malloc_initialize_hook@GLIBC_2.2.5

$ readelf -W -a /lib64/libc.so.6 | grep malloc         
00000000001c6e30  000004d900000006 R_X86_64_GLOB_DAT      000000000008b6d0 malloc@@GLIBC_2.2.5 + 0
00000000001c6e68  0000075200000006 R_X86_64_GLOB_DAT      00000000001c98f0 __malloc_initialize_hook@GLIBC_2.2.5 + 0
00000000001c6ef0  0000047600000006 R_X86_64_GLOB_DAT      00000000001c7c30 __malloc_hook@@GLIBC_2.2.5 + 0
   465: 000000000008b6d0   852 FUNC    GLOBAL DEFAULT   12 __libc_malloc@@GLIBC_2.2.5
   575: 000000000008d550   132 FUNC    WEAK   DEFAULT   12 malloc_info@@GLIBC_2.10
   859: 000000000008d0d0   495 FUNC    WEAK   DEFAULT   12 malloc_stats@@GLIBC_2.2.5
  1041: 0000000000143830    62 FUNC    GLOBAL DEFAULT   12 malloc_get_state@GLIBC_2.2.5
  1142: 00000000001c7c30     8 OBJECT  WEAK   DEFAULT   29 __malloc_hook@@GLIBC_2.2.5
  1241: 000000000008b6d0   852 FUNC    GLOBAL DEFAULT   12 malloc@@GLIBC_2.2.5
  1282: 000000000008cea0   243 FUNC    WEAK   DEFAULT   12 malloc_usable_size@@GLIBC_2.2.5
  1541: 000000000008cbb0   743 FUNC    WEAK   DEFAULT   12 malloc_trim@@GLIBC_2.2.5
  1874: 00000000001c98f0     8 OBJECT  WEAK   DEFAULT   30 __malloc_initialize_hook@GLIBC_2.2.5
  2187: 0000000000143870   289 FUNC    GLOBAL DEFAULT   12 malloc_set_state@GLIBC_2.2.5


breakpoint on 'turn_on_mcheck' never triggers, but apparently mcheck's 'mallochook'
is working:


(gdb) b turn_on_mcheck
Function "turn_on_mcheck" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (turn_on_mcheck) pending.
(gdb) b mallochook
Function "mallochook" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 2 (mallochook) pending.
(gdb) r
Starting program: ./a.out 

Breakpoint 2, mallochook (size=49, caller=0x7ffff7e3f9dd <memalignhook+333>) at mcheck.c:202
202     {
(gdb) c
Continuing.

Breakpoint 2, mallochook (size=1024, caller=0x7ffff7e24f9c <__GI__IO_file_doallocate+140>) at mcheck.c:202
202     {
(gdb) c
Continuing.
ret was 0, pointer is 0x4052e0
munmap_chunk(): invalid pointer

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50        return ret;
(gdb) 


glibc is configured thusly:
configure --enable-stack-protector=all --enable-stackguard-randomization --disable-cet --without-selinux --without-cvs --enable-bind-now --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --disable-profile --without-gd --with-headers=/usr/include --prefix=/usr --sysconfdir=/etc --localstatedir=/var --libdir=$(prefix)/lib64 --mandir=$(prefix)/share/man --infodir=$(prefix)/share/info --libexecdir=$(libdir)/misc/glibc --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion=Gentoo 2.28 p1 --disable-systemtap --disable-nscd --disable-werror --without-stack-protector --enable-kernel=4.16.0 --disable-timezone-tools

Gentoo also applies 5 patches to the build (to work around testsuite failures with sandbox), which I revert.
Comment 11 Manuel Lauss 2018-08-08 09:12:41 UTC
'turn_on_mcheck'  does not even exist in the final library, but it's there
in mcheck-init.o in the build directory. 


readelf -Wa /lib64/libc.so.6 | grep mcheck
  1071: 000000000008dea0   260 FUNC    GLOBAL DEFAULT   12 mcheck@@GLIBC_2.2.5
  1357: 000000000008d820   112 FUNC    GLOBAL DEFAULT   12 mcheck_check_all@@GLIBC_2.2.5
  2165: 000000000008dfb0    65 FUNC    GLOBAL DEFAULT   12 mcheck_pedantic@@GLIBC_2.2.5


readelf -Wa mcheck-init.o | egrep -h "malloc|mcheck"
000000000000002b  0000001100000004 R_X86_64_PLT32         0000000000000000 mcheck - 4
0000000000000a92  0000000f00000001 R_X86_64_64            0000000000000000 __malloc_initialize_hook + 0
     1: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS mcheck-init.c
     5: 0000000000000000    52 FUNC    LOCAL  DEFAULT    1 turn_on_mcheck
    15: 0000000000000000     8 OBJECT  GLOBAL DEFAULT    5 __malloc_initialize_hook
    17: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND mcheck
    19: 0000000000000000     8 OBJECT  GLOBAL DEFAULT    5 __malloc_initialize_hook@GLIBC_2.2.5
Comment 12 Carlos O'Donell 2018-08-09 16:29:42 UTC
(In reply to Manuel Lauss from comment #11)
> 'turn_on_mcheck'  does not even exist in the final library, but it's there
> in mcheck-init.o in the build directory. 

It exists only in libmcheck.a.

e.g.

With glibc-devel installed:

readelf -a -W /usr/lib/libmcheck.a | grep turn_on_mcheck
     5: 00000000    27 FUNC    LOCAL  DEFAULT    2 turn_on_mcheck

(In reply to Manuel Lauss from comment #10)
> $ readelf -W -a a.out | grep malloc
>      8: 0000000000404050     8 OBJECT  GLOBAL DEFAULT   23
> __malloc_initialize_hook@GLIBC_2.2.5
>     43: 0000000000404050     8 OBJECT  GLOBAL DEFAULT   23
> __malloc_initialize_hook@GLIBC_2.2.5

This can't be right. It should not be a versioned symbol in the executable,
since it's unversioned in the archive.

Exactly how did you build this executable?

Please attach the output of `readelf -a -W ./a.out` to this issue.

Please attach the output of `readelf -a -W /usr/lib64/libmcheck.a` to this issue.

> $ readelf -W -a /lib64/libc.so.6 | grep malloc 

If you actually grep'd for malloc there should be many more results than this.
        
Please attach the output of `readelf -a -W /lib64/libc.so.6` to this issue. 
 
> breakpoint on 'turn_on_mcheck' never triggers, but apparently mcheck's

That's what's wrong.

> 'mallochook'
> is working:

Right, it's "half enabled" and that will crash your application.

I suspect something is wrong with your toolchain.
Comment 13 Manuel Lauss 2018-08-10 05:42:43 UTC
Created attachment 11171 [details]
readelf a.out
Comment 14 Manuel Lauss 2018-08-10 05:43:06 UTC
Created attachment 11172 [details]
readelf libc-2.28.so
Comment 15 Manuel Lauss 2018-08-10 05:43:26 UTC
Created attachment 11173 [details]
readelf libmcheck.a
Comment 16 Manuel Lauss 2018-08-10 06:07:27 UTC
(In reply to Carlos O'Donell from comment #12)
> (In reply to Manuel Lauss from comment #11)
> > 'turn_on_mcheck'  does not even exist in the final library, but it's there
> > in mcheck-init.o in the build directory. 
> 
> It exists only in libmcheck.a.

Can confirm that,


> Exactly how did you build this executable?

$ gcc -v -lmcheck -ggdb -Og -pipe test.c
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/8.2.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /tmp-ram/portage/sys-devel/gcc-8.2.0/work/gcc-8.2.0/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/8.2.0 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/8.2.0 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/8.2.0/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/8.2.0/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/include/g++-v8 --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/8.2.0/python --enable-languages=c,c++ --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 8.2.0 p1.3' --disable-esp --enable-libstdcxx-time --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-multilib --with-multilib-list=m32,m64 --disable-altivec --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libmudflap --disable-libssp --disable-libmpx --disable-systemtap --disable-vtable-verify --disable-libvtv --disable-libquadmath --enable-lto --with-isl --disable-isl-version-check --disable-libsanitizer --disable-default-pie --disable-default-ssp --with-glibc-version=2.27 --with-arch=znver1 --with-tune=znver1
Thread model: posix
gcc version 8.2.0 (Gentoo 8.2.0 p1.3) 
COLLECT_GCC_OPTIONS='-v' '-ggdb' '-Og' '-pipe' '-mtune=znver1' '-march=znver1'
 /usr/libexec/gcc/x86_64-pc-linux-gnu/8.2.0/cc1 -quiet -v test.c -quiet -dumpbase test.c -mtune=znver1 -march=znver1 -auxbase test -ggdb -Og -version -o - |
 /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../../x86_64-pc-linux-gnu/bin/as -v --64 -o /tmp/.private/mano/ccHBxdLA.o
GNU assembler version 2.31.51 (x86_64-pc-linux-gnu) using BFD version (Gentoo 9999 p1) 2.31.51.20180731
GNU C17 (Gentoo 8.2.0 p1.3) version 8.2.0 (x86_64-pc-linux-gnu)
        compiled by GNU C version 8.2.0, GMP version 6.1.2, MPFR version 4.0.1, MPC version 1.1.0, isl version isl-0.20-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../../x86_64-pc-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/include
 /usr/local/include
 /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/include-fixed
 /usr/include
End of search list.
GNU C17 (Gentoo 8.2.0 p1.3) version 8.2.0 (x86_64-pc-linux-gnu)
        compiled by GNU C version 8.2.0, GMP version 6.1.2, MPFR version 4.0.1, MPC version 1.1.0, isl version isl-0.20-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 19b312d39d18a0244da7d32dd0eafd38
COMPILER_PATH=/usr/libexec/gcc/x86_64-pc-linux-gnu/8.2.0/:/usr/libexec/gcc/x86_64-pc-linux-gnu/8.2.0/:/usr/libexec/gcc/x86_64-pc-linux-gnu/:/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/:/usr/lib/gcc/x86_64-pc-linux-gnu/:/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../../x86_64-pc-linux-gnu/bin/
LIBRARY_PATH=/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/:/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../../x86_64-pc-linux-gnu/lib/:/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-ggdb' '-Og' '-pipe' '-mtune=znver1' '-march=znver1'
 /usr/libexec/gcc/x86_64-pc-linux-gnu/8.2.0/collect2 -plugin /usr/libexec/gcc/x86_64-pc-linux-gnu/8.2.0/liblto_plugin.so -plugin-opt=/usr/libexec/gcc/x86_64-pc-linux-gnu/8.2.0/lto-wrapper -plugin-opt=-fresolution=/tmp/.private/mano/ccAO6PAJ.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/crtbegin.o -L/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0 -L/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../../x86_64-pc-linux-gnu/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../.. -lmcheck /tmp/.private/mano/ccHBxdLA.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/crtend.o /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../../lib64/crtn.o
COLLECT_GCC_OPTIONS='-v' '-ggdb' '-Og' '-pipe' '-mtune=znver1' '-march=znver1'


> Please attach the output of `readelf -a -W ./a.out` to this issue.
 
> Please attach the output of `readelf -a -W /usr/lib64/libmcheck.a` to this
> issue.
         
> Please attach the output of `readelf -a -W /lib64/libc.so.6` to this issue. 

Done,

 
> I suspect something is wrong with your toolchain.

hm, ok.


Thank you!
    Manuel
Comment 17 Manuel Lauss 2018-08-10 13:51:07 UTC
Seems like gentoo is stripping the "turn_on_mcheck" func from libmcheck.a
when it installs it.  Fixing that, I get now:

Breakpoint 1, __GI___libc_malloc (bytes=0) at malloc.c:3012
3012    {
(gdb) p __malloc_initialize_hook
$1 = (void (*)(void)) 0x401160 <turn_on_mcheck>

(gdb) b turn_on_mcheck
Breakpoint 1 at 0x401160: file mcheck-init.c, line 27.
(gdb) r
Starting program: ./a.out 

Breakpoint 1, turn_on_mcheck () at mcheck-init.c:27
27      {
(gdb) c
Continuing.
ret was 0, pointer is 0x4052e0
munmap_chunk(): invalid pointer

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50        return ret;
(gdb) c
Continuing.


Something is still not right.
Comment 18 Carlos O'Donell 2018-08-11 03:45:33 UTC
In your libmcheck.a you have:

Symbol table '.symtab' contains 13 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 SECTION LOCAL  DEFAULT    1 
     2: 0000000000000000     0 SECTION LOCAL  DEFAULT    3 
     3: 0000000000000000     0 SECTION LOCAL  DEFAULT    4 
     4: 0000000000000000     0 SECTION LOCAL  DEFAULT    5 
     5: 0000000000000000     0 SECTION LOCAL  DEFAULT    7 
     6: 0000000000000000     0 SECTION LOCAL  DEFAULT    8 
     7: 0000000000000000     0 SECTION LOCAL  DEFAULT    9 
     8: 0000000000000000     0 SECTION LOCAL  DEFAULT   11 
     9: 0000000000000000     8 OBJECT  GLOBAL DEFAULT    5 __malloc_initialize_hook
    10: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND mcheck
    11: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __stack_chk_fail
    12: 0000000000000000     8 OBJECT  GLOBAL DEFAULT    5 __malloc_initialize_hook@GLIBC_2.2.5

The __malloc_initialize_hook@GLIBC_2.2.5 is a versioned reference in a static object. This is completely wrong. You have a broken binutils in your build that is breaking this.

You are going to have to go back to the Gentoo team and find out why this is the case.

You should see something like this:

Symbol table '.symtab' contains 18 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS mcheck-init.c
     2: 0000000000000000     0 SECTION LOCAL  DEFAULT    1 
     3: 0000000000000000     0 SECTION LOCAL  DEFAULT    3 
     4: 0000000000000000     0 SECTION LOCAL  DEFAULT    4 
     5: 0000000000000000     7 FUNC    LOCAL  DEFAULT    1 turn_on_mcheck
     6: 0000000000000000     0 SECTION LOCAL  DEFAULT    5 
     7: 0000000000000000     0 SECTION LOCAL  DEFAULT    7 
     8: 0000000000000000     0 SECTION LOCAL  DEFAULT    9 
     9: 0000000000000000     0 SECTION LOCAL  DEFAULT   10 
    10: 0000000000000000     0 SECTION LOCAL  DEFAULT   12 
    11: 0000000000000000     0 SECTION LOCAL  DEFAULT   14 
    12: 0000000000000000     0 SECTION LOCAL  DEFAULT   16 
    13: 0000000000000000     0 SECTION LOCAL  DEFAULT   17 
    14: 0000000000000000     0 SECTION LOCAL  DEFAULT   15 
    15: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _GLOBAL_OFFSET_TABLE_
    16: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND mcheck
    17: 0000000000000000     8 OBJECT  GLOBAL DEFAULT    5 __malloc_initialize_hook

Which is what I have in Fedora.

I suggest reviewing the results of your binutils build to look for failures in the testsuite versus what upstream failures are present.

At this point it looks like we've triaged this to the point where the distro maintainers should take over.
Comment 19 Hussam Al-Tayeb 2018-08-11 12:39:38 UTC
What the configure options for binutils on Fedora?
Comment 20 Carlos O'Donell 2018-08-11 13:30:45 UTC
(In reply to Hussam Al-Tayeb from comment #19)
> What the configure options for binutils on Fedora?

+ ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --quiet --build=x86_64-redhat-linux --host=x86_64-redhat-linux --target=x86_64-redhat-linux --enable-gold=default --enable-ld --with-sysroot=/ --enable-shared --enable-deterministic-archives=no --enable-lto --enable-compressed-debug-sections=none --enable-generate-build-notes=no --enable-targets=x86_64-pep --enable-relro=yes --enable-plugins --with-bugurl=http://bugzilla.redhat.com/bugzilla/

https://kojipkgs.fedoraproject.org//packages/binutils/2.31.1/10.fc29/data/logs/x86_64/build.log

It's probably less about which exact configure options you use, and more about which exact upstream version you have and if it had bugs and if you fixed them with backports.

I'm going to mark this RESOLVED/INVALID. Please reach out to your distribution and file bugs with them.
Comment 21 Hussam Al-Tayeb 2018-08-11 14:14:53 UTC
I am using https://sourceware.org/git/?p=binutils-gdb.git;a=shortlog;h=refs/heads/binutils-2_30-branch
with no backports. Are there any specific patches I need to apply on top of the branch?
Comment 22 Dmitry V. Levin 2018-08-11 17:39:00 UTC
I take the liberty to reopen this because I see exactly the same issue in ALT with __malloc_initialize_hook@GLIBC_2.2.5 in libmcheck.a built from release/2.27/master branch using binutils built from 20180528 snapshot of 2.30.0 branch.

The issue therefore is not distro-specific.  If this happens to be a fixed bug in upstream binutils, I think we have to identify the earliest version of binutils that works properly and update glibc build requirements.
Comment 23 Andreas Schwab 2018-08-11 18:08:33 UTC
The version suffix is not a bug, __malloc_initialize_hook is a compat symbol.
Comment 24 Hussam Al-Tayeb 2018-09-22 07:17:18 UTC
Any news regarding this issue?
Comment 25 ben 2020-02-13 18:27:43 UTC
Just to note that I'm still getting this today on an up to date Debian system.

$ uname -a
Linux myhostname 4.19.0-5-amd64 #1 SMP Debian 4.19.37-5 (2019-06-19) x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 10 (buster)
Release:	10
Codename:	buster

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/8/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 8.3.0-6' --with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-8 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 8.3.0 (Debian 8.3.0-6) 


$ ldd --version
ldd (Debian GLIBC 2.28-10) 2.28
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
Comment 26 Siddhesh Poyarekar 2021-07-13 15:46:10 UTC
I can reproduce this and it should get fixed once the patchset[1] to move malloc debugging out into a separate DSO is pushed.

[1] https://patchwork.sourceware.org/project/glibc/list/?series=2486
Comment 27 Siddhesh Poyarekar 2021-07-22 13:20:15 UTC
commit c142eb253f3814f46527e9b37484041dd85702cf
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
Date:   Thu Jul 22 18:38:02 2021 +0530

    mcheck: Wean away from malloc hooks [BZ #23489]
    
    Split the mcheck implementation into the debugging hooks and API so
    that the API can be replicated in libc and libc_malloc_debug.so.  The
    libc APIs always result in failure.
    
    The mcheck implementation has also been moved entirely into
    libc_malloc_debug.so and with it, all of the hook initialization code
    can now be moved into the debug library.  Now the initialization can
    be done independently of libc internals.
    
    With this patch, libc_malloc_debug.so can no longer be used with older
    libcs, which is not its goal anyway.  tst-vfork3 breaks due to this
    since it spawns shell scripts, which in turn execute using the system
    glibc.  Move the test to tests-container so that only the built glibc
    is used.
    
    This move also fixes bugs in the mcheck version of memalign and
    realloc, thus allowing removal of the tests from tests-mcheck
    exclusion list.
    
    Reviewed-by: Carlos O'Donell <carlos@redhat.com>
    Tested-by: Carlos O'Donell <carlos@redhat.com>
Comment 28 Hussam Al-Tayeb 2021-07-22 14:27:23 UTC
(In reply to Siddhesh Poyarekar from comment #26)
> I can reproduce this and it should get fixed once the patchset[1] to move
> malloc debugging out into a separate DSO is pushed.
> 
> [1] https://patchwork.sourceware.org/project/glibc/list/?series=2486

Can this be safely backported to the 2.33 branch?
Comment 29 Siddhesh Poyarekar 2021-07-22 14:34:17 UTC
I'm afraid not, the patchset introduces an ABI change.  It may be possible to isolate and backport some fixes for mcheck.