Since glibc 2.14 the malloc hooks have been deprecated. The hooks should be removed, and the APIs set up to return no data. A new DSO should be implemented with the required functionality which users can preload if they need the required features in their binaries.
To be specific this bug is about the removal of: __free_hook __malloc_hook __realloc_hook __memalign_hook
Does this mean that __morecore and __after_morecore_hook remain?
(In reply to Florian Weimer from comment #2) > Does this mean that __morecore and __after_morecore_hook remain? As of right now they do. I will propose the removal of __morecore and __after_morecore_hook in a second patch set.
(In reply to Carlos O'Donell from comment #3) > (In reply to Florian Weimer from comment #2) > > Does this mean that __morecore and __after_morecore_hook remain? > > As of right now they do. > > I will propose the removal of __morecore and __after_morecore_hook in a > second patch set. I take it back. I'm going to roll __morecore, __default_morecore, and __after_morecore_hook removal into this patch too, it's similar enough. I was mistaken that it might have some extra dependencies. There is a bit of a wrinkle in that for brand new arches we must continue to implement mtrace, mcheck, et. al., and to do that we need to keep the hooks (for now) and expose them via a GLIBC_PRIVATE interface to libmalloc-extras.so. Otherwise we have to deprecate and remove all the interfaces that use the hooks, and we need not do that if we can re-implement them a new backend set of MT-safe hooks (like the trace infrastructure we used for mtrace). However, we would likely always keep libmalloc-extras.so to implement the tracing support.
commit 1e5a5866cb9541b5231dba3d86c8a1a35d516de9 Author: Siddhesh Poyarekar <siddhesh@sourceware.org> Date: Thu Jul 22 18:38:12 2021 +0530 Remove malloc hooks [BZ #23328] Make malloc hooks symbols compat-only so that new applications cannot link against them and remove the declarations from the API. Also remove the unused malloc-hooks.h. Finally, mark all symbols in libc_malloc_debug.so as compat so that the library cannot be linked against. Add a note about the deprecation in NEWS. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com> commit 0552fd2c7d4e8a570cb4fe4dfe65e96f6d24b0cd Author: Siddhesh Poyarekar <siddhesh@sourceware.org> Date: Thu Jul 22 18:38:10 2021 +0530 Move malloc_{g,s}et_state to libc_malloc_debug These deprecated functions are only safe to call from __malloc_initialize_hook and as a result, are not useful in the general case. Move the implementations to libc_malloc_debug so that existing binaries that need it will now have to preload the debug DSO to work correctly. This also allows simplification of the core malloc implementation by dropping all the undumping support code that was added to make malloc_set_state work. One known breakage is that of ancient emacs binaries that depend on this. They will now crash when running with this libc. With LD_BIND_NOW=1, it will terminate immediately because of not being able to find malloc_set_state but with lazy binding it will crash in unpredictable ways. It will need a preloaded libc_malloc_debug.so so that its initialization hook is executed to allow its malloc implementation to work properly. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com> commit b5bd5bfe88f496463ec9fab680a8edf64d7c2a42 Author: Siddhesh Poyarekar <siddhesh@sourceware.org> Date: Thu Jul 22 18:38:08 2021 +0530 glibc.malloc.check: Wean away from malloc hooks The malloc-check debugging feature is tightly integrated into glibc malloc, so thanks to an idea from Florian Weimer, much of the malloc implementation has been moved into libc_malloc_debug.so to support malloc-check. Due to this, glibc malloc and malloc-check can no longer work together; they use altogether different (but identical) structures for heap management. This should not make a difference though since the malloc check hook is not disabled anywhere. malloc_set_state does, but it does so early enough that it shouldn't cause any problems. The malloc check tunable is now in the debug DSO and has no effect when the DSO is not preloaded. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com> commit 9dad716d4d2993f50b165747781244bd7c43bc95 Author: Siddhesh Poyarekar <siddhesh@sourceware.org> Date: Thu Jul 22 18:38:06 2021 +0530 mtrace: Wean away from malloc hooks Wean mtrace away from the malloc hooks and move them into the debug DSO. Split the API away from the implementation so that we can add the API to libc.so as well as libc_malloc_debug.so, with the libc implementations being empty. Update localplt data since memalign no longer has any callers after this change. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com> commit cc35896ea3e4532919ec81b17f36299117debe79 Author: Siddhesh Poyarekar <siddhesh@sourceware.org> Date: Thu Jul 22 18:38:04 2021 +0530 Simplify __malloc_initialized Now that mcheck no longer needs to check __malloc_initialized (and no other third party hook can since the symbol is not exported), make the variable boolean and static so that it is used strictly within malloc. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com> commit c142eb253f3814f46527e9b37484041dd85702cf Author: Siddhesh Poyarekar <siddhesh@sourceware.org> Date: Thu Jul 22 18:38:02 2021 +0530 mcheck: Wean away from malloc hooks [BZ #23489] Split the mcheck implementation into the debugging hooks and API so that the API can be replicated in libc and libc_malloc_debug.so. The libc APIs always result in failure. The mcheck implementation has also been moved entirely into libc_malloc_debug.so and with it, all of the hook initialization code can now be moved into the debug library. Now the initialization can be done independently of libc internals. With this patch, libc_malloc_debug.so can no longer be used with older libcs, which is not its goal anyway. tst-vfork3 breaks due to this since it spawns shell scripts, which in turn execute using the system glibc. Move the test to tests-container so that only the built glibc is used. This move also fixes bugs in the mcheck version of memalign and realloc, thus allowing removal of the tests from tests-mcheck exclusion list. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com> commit 2d2d9f2b48a943fa556301db532103d09800da4d Author: Siddhesh Poyarekar <siddhesh@sourceware.org> Date: Thu Jul 22 18:37:59 2021 +0530 Move malloc hooks into a compat DSO Remove all malloc hook uses from core malloc functions and move it into a new library libc_malloc_debug.so. With this, the hooks now no longer have any effect on the core library. libc_malloc_debug.so is a malloc interposer that needs to be preloaded to get hooks functionality back so that the debugging features that depend on the hooks, i.e. malloc-check, mcheck and mtrace work again. Without the preloaded DSO these debugging features will be nops. These features will be ported away from hooks in subsequent patches. Similarly, legacy applications that need hooks functionality need to preload libc_malloc_debug.so. The symbols exported by libc_malloc_debug.so are maintained at exactly the same version as libc.so. Finally, static binaries will no longer be able to use malloc debugging features since they cannot preload the debugging DSO. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com> commit 55a4dd39308951da4b0da84b19e415c2bb451b60 Author: Siddhesh Poyarekar <siddhesh@sourceware.org> Date: Thu Jul 22 18:37:57 2021 +0530 Remove __morecore and __default_morecore Make the __morecore and __default_morecore symbols compat-only and remove their declarations from the API. Also, include morecore.c directly into malloc.c; this should ideally get merged into malloc in a future cleanup. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com> commit 57b07bede12635bd6d6aa0e488824bb510bbeca4 Author: Siddhesh Poyarekar <siddhesh@sourceware.org> Date: Thu Jul 22 18:37:54 2021 +0530 Remove __after_morecore_hook Remove __after_morecore_hook from the API and finalize the symbol so that it can no longer be used in new applications. Old applications using __after_morecore_hook will find that their hook is no longer called. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com>
These hook symbols should also be removed from malloc/Versions . The linked output __free_hook@GLIBC_2.2.5 will be unchanged, but it will do gold/ld.lld a favor. In GNU ld, if an object file defines __free_hook@GLIBC_2.2.5 and __free_hook, and --version-script has `GLIBC_2.2.5 { __free_hook; }`, the output will be the non-default version __free_hook@GLIBC_2.2.5 This is a dark corner of symbol versioning: https://www.airs.com/blog/archives/220 Combining Versions Object files other than malloc/malloc-debug.c are using the unversioned symbol '__free_hook'. Their '__free_hook' references and the __free_hook@GLIBC_2.2.5 definition are combined in a strange/unspecified way. It'd be good to remove non-default version symbols from the version script to fix the issue. --- I report it here because this change is the root cause of `FAIL: malloc/tst-compathooks-on` when I build glibc with ld.lld 13.0.0 https://sourceware.org/pipermail/libc-alpha/2021-July/129450.html
(In reply to Fangrui Song from comment #6) > These hook symbols should also be removed from malloc/Versions . > The linked output __free_hook@GLIBC_2.2.5 will be unchanged, but it will do > gold/ld.lld a favor. > [...] I added a workaround to ld.lld so that foo@v1 and (foo or foo@@v1) can be combined: https://reviews.llvm.org/D107235 Good for ld.lld now. For gold linked libc.so, the dynamic symbol is __free_hook@@GLIBC_2.2.5 because of PR gold/28196 I think using `.symver *, *, remove` in glibc is the way forward. Created PR glibc/28197 Once implemented, it can make all linkers happy. (Probably more to GNU ld which has lots of indirection symbol complexity due to support https://www.airs.com/blog/archives/220 in a contrived way).