RFC: shared-memory synchronization benchmarking in glibc

[Torvald Riegel]

I'd like to get feedback on how we build glibc's synchronization-related
microbenchmarks, in particular regarding two aspects: (1) How to best
emulate real-world workloads and (2) how to best fit them into the
existing microbenchmark suite.

Attached is what I've been playing with, which uses this for rwlock
benchmarking.


=== Workloads

As you can see in the attached patch, I'm using random accesses to both
a thread-shared and a thread-private memory region to simulate work done
between calls to synchronization primitives.  For example, in the rwlock
case, accesses to thread-private data between reader critical sections
and accesses to the shared data within critical sections.  So, "work" is
represented as invoking a Marsaglia-XOR-RNG followed by a memory
accesses for a certain number of times.

This aims at giving the memory system some more stuff to do than if we
were just, for example, spinning until a certain number of CPU cycles
has passed.  Also, it allows one to introduce (cache) misses by varying
the parameters.

Is this a reasonable way of emulating real-world workloads for
concurrent code, from the perspective of your hardware?
Any suggestions for how to improve this?
Any suggestions regarding scenarios that should be covered?
I'm looking for both scenarios that can significantly affect
synchronization performance (eg, stressing the memory system more so
that cache misses are harder to hide performance-wise) as well as
scenarios that are representative of common real-world workloads (eg,
should there be more dependencies such as would arise when navigating
through linked data structures?).
What do you use in your own testing?

I'd also be interested in seeing microbenchmarks that show the
assumptions we make.  For example, it would be nice to show cases in
which the acquire hints on Power result in a performance gain, and where
not (so the rest of the glibc developers get a better idea where to use
the hint and where not to use it).

A first target for improved benchmarking would be lock elision
performance, I believe.


=== Making it fit into the microbenchmark framework

I'd like to hear about ideas and suggestions for how to best integrate
this into the microbenchmark suite.  The difference to many of the
current tests is that it's not sufficient to just run a function in a
tight loop.  We need to look at many more workloads (eg, long vs. short
critical sections, different numbers of threads, ...).

That means we need to collect a lot more data, and present it in a
better way.  In the past (for another project), I've used a simple
mongodb database to store json objects representing benchmark results,
and then ran queries across that fed into visualization (eg, using
gnuplot).  Is this something that somebody else is already working on?

Doing that would mean collecting more information about when/where/... a
benchmark run happened.  Any thoughts on that?

It would also be useful if benchmarks were easier to run manually so
that developers (and users) can experiment more easily with different
workloads.  However, that would mean that we would need to be able to
set benchmark parameters when invoking a benchmark, and not just be able
to produce a benchmark binary that has them baked in.  Any thoughts or
preferences how to best do that?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bench-mt.patch
Type: text/x-patch
Size: 9233 bytes
Desc: not available
URL: <http://sourceware.org/pipermail/libc-alpha/attachments/20170110/8e3c75a9/attachment.bin>