This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: [PATCH] benchtests/Makefile: Run the string benchmarks four times by default.
- From: OndÅej BÃlka <neleai at seznam dot cz>
- To: Will Newton <will dot newton at linaro dot org>
- Cc: libc-alpha <libc-alpha at sourceware dot org>, Patch Tracking <patches at linaro dot org>
- Date: Thu, 5 Sep 2013 17:03:03 +0200
- Subject: Re: [PATCH] benchtests/Makefile: Run the string benchmarks four times by default.
- Authentication-results: sourceware.org; auth=none
- References: <52274838 dot 7010902 at linaro dot org> <20130904161743 dot GA10358 at domone dot kolej dot mff dot cuni dot cz> <CANu=DmiVWFijri_iMjFGJEWdTWheHbBFOH8XULURRE8pLMkuLA at mail dot gmail dot com> <20130904165211 dot GA14906 at domone dot kolej dot mff dot cuni dot cz> <CANu=Dmgsr3RcN7dRge0QB0tE9GGDuwVYu-TR_vOEOhzdtT4LJw at mail dot gmail dot com>
On Thu, Sep 05, 2013 at 08:51:53AM +0100, Will Newton wrote:
> The intention of my patch - which I may have not made completely clear
> in the commit message - is to improve test stability. What I mean by
> this is that with a physically indexed cache the physical pages
> allocated to the test can have a significant effect on the performance
> at large (e.g. cache size / ways and above) buffer sizes and this will
> cause variation when running the same test multiple times. My aim is
> to average out these differences as it is hard to control for them
> without understanding the details of the cache subsystem of the system
> you are running on.
>
This can be just explained just by having more data. Simply multiplying
iteration count by four would then do same job.
Please run your patch ten times and calculate variance. Compare that to
variance when iteration count is increased 4 times and show if there is
improvement.
> Your test appears to be addressing concerns of test validity by
> running a wider range of buffer alignments, which is an important but
> separate concern IMO.
>
No, your patch will pick src pointer at 4 different physical pages (one
allocated in each run) and calculate average performance.
Mine will pick src pointers in 2000000/4096 = 488 different pages and
calculate average.
As this makes your patch redundant you will add unnecessary complexity
to glibc, there is a very good reason to avoid complexity in a project
like glibc, which is already extremely complex. Some poor soul will
need to maintain all of the complicated solutions long into the future.