Re: [patch] malloc per-thread cache ready for review

[Carlos O'Donell <carlos at redhat dot com>]

On 02/01/2017 11:44 AM, DJ Delorie wrote:
> 
> Markus Trippelsdorf <markus@trippelsdorf.de> writes:
>> http://locklessinc.com/downloads/lockless_allocator_src.tgz (the best in
>> my testing) or jemalloc.
> 
> Before we go down the "which allocator is best" road... glibc's
> allocator is intended to be a general purpose "reasonably good enough"
> system allocator.  It's easy to find a specific allocator that beats it
> in a specific test, but being a specifically best allocator is not our
> goal here - providing an allocator that can be the default on a
> Linux-based system is.
> 
> Hence, my goal with the per-thread cache is to make it "generally
> better" for overall system performance.
> 
> I am not trying to make it better than every other allocator in every
> case, that's a futile exercise.

Agreed.

The word "best" needs to be qualified by a workload.

If you can provide a workload that performs poorly on glibc malloc and
better on jemalloc for some parameter, then it can be compared more
objectively.

The glibc community broadly agrees that the way to move this forward
is through whole system benchmarking, looking at complete APIs,
gathering traces, and seeing what the workloads are doing and how
to improve the behaviour of certain parameters.

See Red Hat's LPC 2016 presentation and DJ's work on dj/malloc:
https://linuxplumbersconf.org/2016/ocw/system/presentations/3921/original/LPC%202016%20-%20linux%20and%20glibc_%20The%204.5TiB%20malloc%20API%20trace.pdf

Please ask for any help if you are interested in tracing and simulating
the malloc API using the trace/simulation in dj/malloc.

I'm excited to move DJ's tcache work forward since it provides glibc
malloc with an important performance feature already present in tcmalloc
and jemalloc.

-- 
Cheers,
Carlos.