Patch for newlib test assertion naming

[Jeff Johnston]

Joseph S. Myers wrote:
> When integrating dejagnu-based testsuites such as newlib's into an
> automatic regression testing system, the PASS and FAIL lines from the
> ".sum" files from successive runs are extracted and compared.  This
> requires the test assertion names after PASS or FAIL to be unique
> (within the tests for each multilib; successive multilibs repeat the
> same names) and stable so that the differences show only genuine
> changes, not changes because of (for example) the date in a source
> directory name changing.  In addition, it's desirable for a test
> failure not to change the set of test assertion names, so if a
> compilation test fails the following execution test should be marked
> UNRESOLVED rather than disappearing altogether from the test output.
> 
> At present, newlib.sum can have lines such as:
> 
> FAIL: Failed to compile /scratch/gcc/nightly-2005-07-04-csl-arm/src/newlib-mainline/newlib/testsuite/newlib.wctype/twctype.c.
> 
> which has two problems: the test assertion name includes the full
> source directory path, and such a line will only appear if the test
> fails; if it passes, it won't become (and wouldn't make sense to
> become):
> 
> PASS: Failed to compile /scratch/gcc/nightly-2005-07-04-csl-arm/src/newlib-mainline/newlib/testsuite/newlib.wctype/twctype.c.
> 
> Test names such as "newlib.wctype/twctype.c compilation", which can
> appear after either PASS or FAIL, avoid these problems.  This patch
> accordingly changes the newlib testsuite to use such test names, with
> additional information about unexpected results from output comparison
> going in "verbose -log" lines in the log rather than in the FAIL name.
> Tested by running the testsuite for arm-none-eabi.  OK to commit?
> 

Ok.  Patch checked in.

-- Jeff J.

> 2005-07-06  Joseph S. Myers  <joseph@codesourcery.com>
> 
> 	* testsuite/lib/checkoutput.exp (newlib_check_output): Use test
> 	names after PASS and FAIL which do not depend on source directory
> 	name or on whether test passed or failed.
> 	* testsuite/lib/passfail.exp (newlib_pass_fail): Likewise.
> 
> diff -rupN testsuite.orig/lib/checkoutput.exp testsuite/lib/checkoutput.exp
> --- testsuite.orig/lib/checkoutput.exp	2005-07-05 00:10:08.000000000 +0000
> +++ testsuite/lib/checkoutput.exp	2005-07-06 22:39:13.000000000 +0000
> @@ -17,9 +17,11 @@ proc newlib_check_output { srcfile expec
>      set comp_output [newlib_target_compile "$srcfullname" "$test_driver" "executable" ""]
>  
>      if { $comp_output != "" } {
> -	fail "Failed to compile $srcfile.\n"
> +	fail "$subdir/$srcfile compilation"
> +	unresolved "$subdir/$srcfile output"
>  	return
>      }
> +    pass "$subdir/$srcfile compilation"
>  
>      set result [newlib_load $test_driver ""]
>      set status [lindex $result 0]
> @@ -30,11 +32,12 @@ proc newlib_check_output { srcfile expec
>      foreach { expectedval } $expectlist {
>  	set gotval [string trim [lindex $output_lines 0] "\r"]
>  	if { ! [string match $expectedval $gotval] } {
> -	    fail "$srcfile: Expected: $expectedval Got: $gotval "
> +	    verbose -log "$subdir/$srcfile: Expected: $expectedval Got: $gotval "
> +	    fail "$subdir/$srcfile output"
>  	    return
>  	}
>  	set output_lines [lrange $output_lines 1 end]
>      }
>  
> -    pass $srcfile
> +    pass "$subdir/$srcfile output"
>  }
> diff -rupN testsuite.orig/lib/passfail.exp testsuite/lib/passfail.exp
> --- testsuite.orig/lib/passfail.exp	2005-07-05 00:10:08.000000000 +0000
> +++ testsuite/lib/passfail.exp	2005-07-06 22:40:09.000000000 +0000
> @@ -41,10 +41,12 @@ proc newlib_pass_fail { srcfile } {
>      set comp_output [newlib_target_compile "$fullsrcfile" "$test_driver" "executable" ""]
>  
>      if { $comp_output != "" } {
> -	fail "Failed to compile $fullsrcfile.\n"
> +	fail "$subdir/$srcfile compilation"
> +	unresolved "$subdir/$srcfile execution"
>      } else {
> +	pass "$subdir/$srcfile compilation"
>  	set result [newlib_load $test_driver ""]
>  	set status [lindex $result 0]
> -	$status "$fullsrcfile"
> +	$status "$subdir/$srcfile execution"
>      }
>  }
> 
>