This is the mail archive of the
gdb-patches@sourceware.org
mailing list for the GDB project.
Re: [RFC] GDB performance testing infrastructure
- From: Yao Qi <yao at codesourcery dot com>
- To: "Agovic, Sanimir" <sanimir dot agovic at intel dot com>
- Cc: "gdb-patches at sourceware dot org" <gdb-patches at sourceware dot org>
- Date: Wed, 28 Aug 2013 11:02:58 +0800
- Subject: Re: [RFC] GDB performance testing infrastructure
- Authentication-results: sourceware.org; auth=none
- References: <520B7F70 dot 6070207 at codesourcery dot com> <0377C58828D86C4588AEEC42FC3B85A71764E92B at IRSMSX105 dot ger dot corp dot intel dot com>
On 08/27/2013 09:49 PM, Agovic, Sanimir wrote:
* Remote debugging. It is slower to read from the remote target, and
> worse, GDB reads the same memory regions in multiple times, or reads
> the consecutive memory by multiple packets.
>
Once gdb and gdbserver share most of the target code, the overhead will be
caused by the serial protocol roundtrips. But this will take a while...
Sanimir, thanks for your comments!
One of the motivations of the performance testing is to measure the
overhead of RSP in some scenarios, and look for the opportunities to
improve it, or add a completely new protocol, which is an extreme case.
Once the infrastructure is ready, we can write some tests to see how
efficient or inefficient RSP is.
> * Tracepoint. Tracepoint is designed to be efficient on collecting
> data in the inferior, so we need performance tests to guarantee that
> tracepoint is still efficient enough. Note that we a test
> `gdb.trace/tspeed.exp', but there are still some rooms to improve.
>
Afaik the tracepoint functionality is quite separated from gdb may be tested
in isolation. Having a generic benchmark framework covering the most parts of
gdb is probably_the_ way to go but I see some room for specialized benchmarks
e.g. for tracepoints with a custom driver. But my knowledge is too vague on
the topic.
Well, it is sort of design trade-off. We need a framework generic
enough to handle most of the testing requirements for different GDB
modules, (such as solib, symbols, backtrace, disassemble, etc), on the
other hand, we want each test is specialized for the corresponding GDB
module, so that we can find more details.
I am inclined to handle testing to _all_ modules under this generic
framework.
> 2. Detect performance regressions. We collected the performance data
> of each micro-benchmark, and we need to detect or identify the
> performance regression by comparing with the previous run. It is
> more powerful to associate it with continuous testing.
>
Something really simple, so simple that one could run it silently with every
make invokation. For a newcomer, it took me some time to get used to make
check e.g. setup, run, and interpret the tests with various settings. Something
simpler would help to run it more often.
Yes, I agree, everything should be simple. I assume that people
running performance testing should be familiar with GDB regular
regression test, like 'make check'. We'll provide 'make check-perf' to
run performance testing ,and it doesn't add extra difficulties on top of
'make check', from user's point of view, IMO.
I like to add the Machine Interface (MI) to the list, but it is quite rudimentary:
$ gdb -interpreter mi -q debugee
[...]
-enable-timings
^done
(gdb)
-break-insert -f main
^done,bkpt={...},time={wallclock="0.00656",user="0.00000",system="0.00000"}
[...]
(gdb)
-exec-step
^running
*running,thread-id="1"
(gdb)
*stopped,[...],time={wallclock="0.19425",user="0.09700",system="0.04200"}
(gdb)
With -enable-timings[1] enabled, every result record has a time triple
appended, even for async[2] ones. If we come up with a full mi parser
one could run tests w/o timings. A mi result is quite json-ish.
Thanks for the input.
(To be honest I do not know how timings are composed of =D)
In addition there are some tools for plotting benchmark results[3].
[1]http://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI-Miscellaneous-Commands.html
[2]https://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI-Async-Records.html
[3]http://speed.pypy.org/
I am using speed to track and show the performance data I got from the
GDB performance tests. It is able to associate the performance data to
the commit, so easy to find which commit causes regressions. However,
my impression is that speed or its dependent packages are not
well-maintained nowadays.
After some search online, I like the chromium performance test and its
plot, personally. It is integrated with buildbot (a customized version).
http://build.chromium.org/f/chromium/perf/dashboard/overview.html
However, as I said in this proposal, let us focus on goal #1 first, get
the framework ready and collect performance data.
--
Yao (éå)