malloc: performance improvements and bugfixes

[Steven Munroe]

On Tue, 2016-01-26 at 22:40 +0100, Florian Weimer wrote:
> On 01/26/2016 09:50 PM, Steven Munroe wrote:
> 
> > So why not fix emacs to stop doing this (purely evil behavior).
> > 
> > If they want to persist their internal state from session to session
> > there are better ways. For example: https://sphde.github.io/
> 
> It's complicated, but not just for the technical challenges involved.
> My warning to the Emacs developers that we are going to clean this up on
> the glibc side was not universally well-received.
> 
> Some of the Emacs developers think they have a stop-gap solution for
> future Emacs binaries: They are going to interpose their own malloc.  My
> testing agrees; it should happen automatically once we remove
> long-deprecated symbols from <malloc.h> because that will cause their
> autoconf check to fail, triggering a switch to their built-in malloc.
> (This will happen even before with deprecate and eventually remove
> __malloc_get_state and __malloc_set_state from the API.)
> 
> For existing binaries with newer glibc, I think I found a way to quickly
> rewrite the dumped heap in such a way that we do not have to add any
> special cases to the hot paths in free/realloc.  (We don't even have to
> use Siddhesh's corrupt arena code for that, although that could be an
> option as well.)  The main challenge is whether we can obtain the
> address of the first *allocated* chunk on the main arena.  Once we have
> that, we can do the heap traversal.  I expect maybe a few dozen lines of
> code for this in total, and the constraints on future malloc evolution
> are minimal.  It's much better than maintaining two mallocs inside glibc.
> 
Will this allow the PPC32 ABI to finally use the correct 16-byte
alignment for malloc? Otherwise it is hard to use vector or
_Decimal128...

> I also know one reason why valgrind does not work with dumped Emacs
> binaries, and it's quite straightforward to fix (it could easily be part
> of the traversal to rewrite the heap, but we have to implement it twice,
> once in glibc and once for the Emacs malloc implementation).  Let's hope
> this is the only fix needed to get valgrind going.
> 
> In short, things aren't as bad as we thought they are, and we can
> finally fix bug 6527 after we have implemented the traversal of the
> dumped heap I mentioned above.
> 
> Florian
>