[PATCH 0/N] test-suite improvement - PASS/FAIL: initial patch

[Joseph S. Myers]

On Mon, 10 Sep 2012, Andreas Jaeger wrote:

> will it handle running the testsuite with parallel make? In that case the
> output might not be deterministic, so you need to sort it before comparing...

I'm not sure having two parallel runs append to the same file from the 
shell is safe, though it may be in practice.

I'd suggest that you generate a .test-status or similar file for each test 
with its result - and then collect all the data at the end with a separate 
script that produces the summary from all those files in the build tree.  
That also looks like it will work better if someone does

make check
(a test fails because of an old kernel)
make check
(to continue the test run)

rather than producing a summary of only the results from the second part 
of the run.


-- 
Joseph S. Myers
joseph@codesourcery.com