Re: [PATCH] Add support for multiple data points to perf tests

[Doug Evans <dje at google dot com>]

On Wed, Jul 29, 2015 at 1:45 AM, Yao Qi <qiyaoltc@gmail.com> wrote:
> Doug Evans <xdje42@gmail.com> writes:
>
>> All I've done is rename perftest.log to perftest.sum,
>> and then put more test results in text form in perftest.log.
>
> Oh, I see.
>
>> I *could* put it all in one file, but I like what's in perftest.log today,
>> (though there are a few tweaks I'd like to make),
>> I'm just moving it to perftest.sum.
>> Following the pattern of file naming already in use
>> will be helpful to everyone using the perf testsuite.
>
> Right.  To be clear, with your patch applied, raw data is saved in
> perftest.sum and some basic statistical analysis is output to
> perftest.log.
>
>>
>> If/when we have other forms of output I would expect them
>> to not go to .log or .sum files.
>>
>
> I agree.
>
>>>> I want to record the raw data somewhere, and a bit of statistical analysis
>>>> (standard deviation left for another day), but I also don't want
>>>> it to clutter up the basic report.
>>>> This patch takes a cue from gdb.{sum,log} and does the same thing
>>>> with perftest.{sum,log}.
>>>
>>> I'd like pertest framework only writes raw data down in some formats
>>> (such as text, xml or jason) without any analysis.  Statistical analysis
>>> can be performed in a post-mortem way by other scripts.
>>
>> We can generate a .csv file with the raw data if/when the need arises.
>> [or whatever is easiest for post-processing scripts to read]
>>
>> A human readable form with basic statistical analysis
>> is useful today, and doable in 5 lines of code.
>> Let's add the complexity when there's a demonstrated
>> need for it.
>
> OK, no problem.  I don't have other questions, and your patch looks good
> to me.

Thanks.  Committed.