[RFC] Porting string performance tests into benchtests

[Siddhesh Poyarekar <siddhesh at redhat dot com>]

Hi,

I'm looking at porting the string function performance tests
(string/test-*) into benchtests.  They don't fit into the simple
scenario of mean execution times - they are characterized by the way
they scale in relation to other methods.  In memcpy for example, the
comparison is between a simple byte copy, glibc memcpy and
__builtin_memcpy.

I could port these over by simply defining them as 'benchsets', which
would be independent benchmark runs that execute within the framework
and generate a separate output (i.e. not within bench.out) with its
own format.  That way I need to do as less work as possible to port
the functions over and it also adds a new flexibility to the benchmark
suite.  Any thoughts on this?

I also noticed that the string benchmark programs use the hp-timing
macros, that are essentially raw instructions that get the timestamp
from the cpu.  While this is very low overhead compared to the syscall
approach of clock_gettime, it may not necessarily be an accurate
picture of performance compared to CLOCK_PROCESS_CPUTIME_ID since the
former simply a monotonically increasing clock while the latter is an
account of the cpu time consumed by the process.  I'd like to hear
thoughts in favour/against either of these.  As of now I am inclined
towards continuing use of clock_gettime throughout (and hence also for
the string benchmarks once I port them over) due to the advantage I
mentioned above.

Maybe it's not clear cut as that either.  For example, hp-timing seems
to be a better measure for I/O bound functions since we are not just
looking at cpu time consumption there.  Maybe a similar argument could
be made for the string functions too and hence justify having them
stick to hp-timing?

Thanks,
Siddhesh