How to properly check for regressions?

Marc Khouzam
Thu Aug 4 20:44:00 GMT 2011


I just realized that I might have been checking for regressions wrong!

The doc says 

"When the testsuite is finished, you'll get a summary that looks like this:"

So, I've always looked at that summary, at the end of the console printouts,
and made sure it was the same before and after my patches. (note that
with -j4, that last summary showed thousands of tests)

But now I found the testsuite/gdb.sum file which shows a larger
set of tests in the summary that what is shown in the console.
Doing a diff of that file before and after my patch showed failures
I hadn't seen from the console summary.

What is the right way to make sure my patch does not cause regressions?

And how do I deal with tests that sometimes pass and sometimes fail.
The gdb.sum file does not seem to be constant, event without code changes.



More information about the Gdb mailing list