This is the mail archive of the
gdb@sourceware.org
mailing list for the GDB project.
How to properly check for regressions?
- From: Marc Khouzam <marc dot khouzam at ericsson dot com>
- To: "'gdb at sourceware dot org'" <gdb at sourceware dot org>
- Date: Thu, 4 Aug 2011 16:43:55 -0400
- Subject: How to properly check for regressions?
Hi,
I just realized that I might have been checking for regressions wrong!
The doc says
(http://sourceware.org/gdb/current/onlinedocs/gdbint/Testsuite.html#Testsuite)
"When the testsuite is finished, you'll get a summary that looks like this:"
So, I've always looked at that summary, at the end of the console printouts,
and made sure it was the same before and after my patches. (note that
with -j4, that last summary showed thousands of tests)
But now I found the testsuite/gdb.sum file which shows a larger
set of tests in the summary that what is shown in the console.
Doing a diff of that file before and after my patch showed failures
I hadn't seen from the console summary.
What is the right way to make sure my patch does not cause regressions?
And how do I deal with tests that sometimes pass and sometimes fail.
The gdb.sum file does not seem to be constant, event without code changes.
Thanks
Marc