Re: [PATCH v2][BZ 14561] Testcase

["Joseph S. Myers" <joseph at codesourcery dot com>]

On Tue, 4 Jun 2013, Ondrej Bilka wrote:

> On Tue, Jun 04, 2013 at 09:08:17AM -0400, Carlos O'Donell wrote:
> > This is a good first pass, but we need to make this easier to debug
> > when it fails and when we switch to PASS/FAIL test we should be able
> > to explain which ordering failed.
> > 
> I plan to add a breakpoint() function to test which counts times it was
> called before failure(or signal). Then we can output something like:
> 
> FAIL, Type following for debugging:

I think the goal with PASS / FAIL information is for each test to have a 
well-defined, fixed (i.e. not depending on the results of the tests) set 
of test assertion names, such that for each assertion it outputs either

PASS: <name>

or

FAIL: <name>

(or other variants such as UNRESOLVED, UNSUPPORTED etc.), resulting in 
both such information for overall tests (the units run from the makefiles) 
and for individual assertions within those tests.  Ideally the <name> 
would be prefixed in some way with the name of the overall test, so

PASS: stdlib/bug-srand-srandom-dependency some name

(for example) in the log for the individual test.

Additionaly information might be printed, but should be separate from the 
basic PASS / FAIL for individual assertions, so that results from multiple 
test runs can be properly compared.

-- 
Joseph S. Myers
joseph@codesourcery.com