[PATCH] malloc: Remove dumped heap support

[Florian Weimer]

* DJ Delorie:

> Sam James <sam@gentoo.org> writes:
>>> malloc_get_state is only used during the Emacs build process,
>>> so we provide a stub implementation only.  Existing Emacs binaries
>>> will not call this stub function, but still reference the symbol.
>>
>> This reassured me, but I was concerned seeing the changes below because
>> of "... contained in historic Emacs executables". My modern Emacs built
>> from trunk does have a .pdmp file installed at least. I don't know if
>> old Emacs ever installed its (non-portable) dump, or if it was ever
>> actually used for anything once installed.
>
> It might be useful to poll the emacs mailing list for their opinions
> too; they might be glad to have some external pressure to force
> 8-year-old emacs binary users to upgrade to something more secure.
>
> (and it's 8-year-old emacs on 0-year-old glibc, not just older distros!)
>
> A quick look at the emacs sources, I see a lot of #ifdef DOUG_LEA_MALLOC
> so removing support might be more complicated than we think.  The
> configure check is simple though:
>
> AC_CACHE_CHECK(
>   [whether malloc is Doug Lea style],
>   [emacs_cv_var_doug_lea_malloc],
>   [emacs_cv_var_doug_lea_malloc=no
>    dnl Hooks do not work with address sanitization.
>    if test "$emacs_cv_sanitize_address" != yes; then
>      AC_LINK_IFELSE(
>        [AC_LANG_PROGRAM(
>           [[#include <malloc.h>
>             static void hook (void) {}]],
>           [[malloc_set_state (malloc_get_state ());
>             __after_morecore_hook = hook;
>             __malloc_initialize_hook = hook;]])],
>        [emacs_cv_var_doug_lea_malloc=yes])
>    fi])
> doug_lea_malloc=$emacs_cv_var_doug_lea_malloc

This configure check already fails with current glibc: malloc_set_state
is no longer part of libc.so.6.  You would have to preload or link
against libc_malloc_debug.so.0, and even that is difficult because it's
a compat symbol.  The question is solely about preserving a way to run
old Emacs binaries.

If really necessary, we could create separate project which provides a
compat malloc.

Thanks,
Florian