acos-inputs

[Paul Zimmermann]

       Dear Siddhesh,

> From: Siddhesh Poyarekar <siddhesh@gotplt.org>
> Date: Wed, 11 Aug 2021 21:36:58 +0530
> 
> On 8/11/21 8:55 PM, Paul Zimmermann wrote:
> > Shouldn't the inputs in benchtests/*-inputs (apart from special sections)
> > represent random inputs that could appear in real life?
> 
> IIRC (it's been a while!) I had seeded the inputs lists for math 
> functions with a program that generated inputs sequentially to ensure 
> coverage of the different branches in the algorithms.  The intent then 
> was to model the spread of the performance across various paths in a 
> function into a single mean and see how it moves with code changes since 
> many of these functions didn't really have any performance sensitive 
> real world applications that we could base inputs on.  It's far from 
> ideal, but it was a start that ensured all branches (except the 
> short-circuited ones, e.g. 1.0, 0.0, etc) were treated more or less 
> equally as we didn't know which branches were more interesting.

ok I understand, but in the meantime the slow paths have been removed,
thus the branches are not the same any more.

> Over time the interesting functions did get more useful inputs.  Wilco, 
> Szabolcs and others added some inputs for a subset of functions that 
> were gleaned from either real world workloads or from system benchmarks 
> like SPEC.  The fact that the remaining functions don't have any more 
> inputs is probably an indicator that nobody found any real world 
> applications where these functions were performance hotspots.
> 
> I'm very open to modifying or replacing this initial set if you can 
> think of a better model to evaluate performance of these functions or if 
> you find workloads that have a specific spread of inputs.  I'm not 
> entirely convinced that random inputs would model real world 
> applications better though, could you qualify that claim?

I'm not sure I can, since it would first require to define what are random
inputs. For acos for example, should they be uniformly distributed in [-1,1],
or the exponent should be uniformly distributed?

Maybe we could have two workloads for each function: one which would cover
the different branches of the algorithms (as you did), and one that would
contain randomly-generated inputs?

Best regards,
Paul