This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: [PATCH] randomize benchtests
- From: OndÅej BÃlka <neleai at seznam dot cz>
- To: Torvald Riegel <triegel at redhat dot com>
- Cc: Siddhesh Poyarekar <siddhesh dot poyarekar at gmail dot com>, c at domone dot kolej dot mff dot cuni dot cz, GNU C Library <libc-alpha at sourceware dot org>
- Date: Fri, 17 May 2013 12:44:46 +0200
- Subject: Re: [PATCH] randomize benchtests
- References: <20130422120018 dot GA30323 at domone dot kolej dot mff dot cuni dot cz> <CAAHN_R1pHJLoS3iP7KrQMmA4gPLawvuRWoK8Xy13VPMBbyPk+Q at mail dot gmail dot com> <20130422125625 dot GA30639 at domone dot kolej dot mff dot cuni dot cz> <1368786270 dot 3054 dot 3119 dot camel at triegel dot csb>
On Fri, May 17, 2013 at 12:24:30PM +0200, Torvald Riegel wrote:
> On Mon, 2013-04-22 at 14:56 +0200, OndÅej BÃlka wrote:
> > On Mon, Apr 22, 2013 at 05:44:14PM +0530, Siddhesh Poyarekar wrote:
> > > On 22 April 2013 17:30, OndÅej BÃlka <neleai@seznam.cz> wrote:
> > > > + clock_gettime (CLOCK_PROCESS_CPUTIME_ID, &start);
> > > > + for (k = 0; k < iters; k++)
> > > > + {
> > > > + i = rand_r (&seed)%NUM_SAMPLES;
> > > > + BENCH_FUNC(i);
> > > > + }
> > > > + clock_gettime (CLOCK_PROCESS_CPUTIME_ID, &end);
> > >
> > > This is wrong. The interval also has the time taken to call rand_r.
> > >
> > This is not wrong. You are interested only on differences between
> > implementations and adding same time from rand_r calls does not change
> > that.
>
> But if we should be changing the rand_r implementation in the future
> (e.g., if we might be getting HW support for it on a certain
> architecture), then this will lead to a difference in all our
>From stdlib/rand_r.c:
/* This algorithm is mentioned in the ISO C standard, here extended
for 32 bits. */
Given that we can break applications that depend on rand_r
reproducibility it will not change.
If you want fully specified generator use drand48.
> performance numbers between the prior code revisions and the newer ones.
> Remember that we eventually also want to find performance regressions.
>
> I think that we should make the inner loops as reproducible as possible,
> so this should either be using a custom pseudo RNG, or calibrate it
> against a loop with just a rand_r call, or don't get the random numbers
> in the loop. The latter might not really be an option too, because we'd
> then need to read those from a precomputed random set in memory, which
> might be even more of a distortion.