Splitting up perf test compilation and data collection

[Doug Evans <dje at google dot com>]

On Thu, Sep 19, 2013 at 3:44 PM, Doug Evans <dje@google.com> wrote:
> Separately,
> We were discussing perf testsuite usage here, and IWBN if there was a mode
> where compilation was separated from perf testing.
> E.g., and this wouldn't be the default of course,
> one could do an initial "make check-perf" that just built the binaries,
> and then a second "make check-perf" that used the prebuilt binaries to
> collect performance data.
> [In between could be various things, like shipping the tests out to
> other machines.]
> I'm just offering this as an idea.  I can imagine implementing this
> in various ways.  Whether we can agree on one ... dunno.
> One thought was to reduce the actual perf collection part of .exp scripts
> to one line that invoked invokes some function passing it the name of
> the python script or some such.
>
> For example,
> We want to be able to run the perf tests in parallel, but we don't want
> test data polluted because, for example, several copies of gcc were also
> running compiling other tests or other tests were running.

[pulled out as a separate thread]

Another thought is to require a convention that perf .exp files are
written like so:

if [doing_perf_compilation] {
    ...
}

if [doing_perf_testing] {
    ...
}

Anyways, these are just thoughts to (try to) advance the discussion.