testsuites classification

[Daniel Berlin]

"GNU Team" <gnuteam@kpit.com> writes:

> I have a question regarding gcc testsuites.
> How do I categorize them ?
> After running the gcc 3.0 test suite on  gcc 2.95.3 (sh-coff-gcc)
> I got following results.
>  
> GCC	PASS	      10077	
> 	FAIL	      4927	
> 	XPASS	      2	
> 	XFAIL	      78	
> 	UNRESOLVED	132	
> 	UNSUPPORTED	59	
> 	WARNINGS	23	
> 	ERRORS	3	
> 
> 
> Now, is there any document which classifies these testsuites ?
> ie. which tests must PASS and which may not pass ?
> Or does it mean that compiler has 4500 bugs which must be fixed ?
It means the compiler is failing 4900 tests.
However, this is not an indication of the number of bugs.
For the c-torture tests, for instance, unless the test specifies
otherwise, each test is compiled 8 times (with different flags each
time).
Other than that, all tests that aren't XFAIL, should be passing (in an
ideal world).

> 
> 
> Arati.

-- 
"I'm so hyper...  (Said with a very dull voice.)
"-Steven Wright

------
Want more information?  See the CrossGCC FAQ, http://www.objsw.com/CrossGCC/
Want to unsubscribe? Send a note to crossgcc-unsubscribe@sourceware.cygnus.com