[PATCH v3 0/5] malloc: tcache improvements

[Wilco Dijkstra]

Hi Cupertino,

As I've been looking at various malloc patches, I decided to benchmark this as well.

The results are as follows for taking best of 3 runs on Neoverse V2 vs current trunk:

patches 1-3:

bench-malloc-thread 1:  6.0% slowdown 
bench-malloc-thread 32: 2.6% slowdown 

bench-malloc-simple (ST case): 1.9% - 2.7% slowdown 
bench-malloc-simple (MT case): 1.8% speedup
bench-malloc-simple (thread arena): 0.2% speedup

The slowdowns are due to the changes in pointer swizzling - it's not obvious why
there is a speedup in the multithreaded path (I guess it uses tcache_get/put in a loop
so that the if-statement cannot be optimized out unlike in the fast path), but the main
slowdowns are in the critical tcache fast paths. Note the overhead of curent pointer
swizzling is around 3%/1.6%/0.8% so it almost doubles the overhead in the ST case.

 
patches 1-4:

bench-malloc-thread 1:  34.9% slowdown (!!!)
bench-malloc-thread 32: 32.1% slowdown (!!!)

bench-malloc-simple (ST case): 10.9% - 12.5% slowdown 
bench-malloc-simple (MT case): 8.4% slowdown
bench-malloc-simple (thread arena): 7.4% slowdown
average slowdown 8.9%

As I suspected, the extra code has a significant negative effect on the tcache
fast paths. A way forward might be to first split into small/large block sizes and
only handling small sizes inline in the entry point of malloc and free.

Cheers,
Wilco