[libgo] Improve Solaris 2/SPARC support

[Joseph S. Myers]

On Thu, 31 Mar 2011, Ralf Wildenhues wrote:

> Hello,
> 
> * Rainer Orth wrote on Thu, Mar 31, 2011 at 06:12:31PM CEST:
> > Ian Lance Taylor <iant@google.com> writes:
> > > Argh, no, I am trying to fight against that as long as possible.  We
> > > should be moving away from DejaGNU, not toward it.
> > 
> > Do you have a decent alternative?  I've no idea what happened to QMTest
> > which CodeSourcery tried for the C++ testsuite.
> 
> FWIW, Automake's parallel-tests driver at least provides halfway decent
> test results summary; timeouts and remote control would need to be
> implemented by means of a per-test driver script or program though.
> And of course the summary style is different from the dejagnu one ...

As far as I could see when I last looked at it, Automake's test code lacks 
the basic feature of being able to test installed programs without 
reconfiguring and rebuilding the code.  No-one really uses GCC with long 
series of -B options in a build tree; installed testing is much closer to 
how it's actually used and the aim should be to move towards staged 
installation for build-tree testing: install in a temporary directory 
within the build tree and run programs from there.  The vast bulk of GCC 
users are probably using a version from a distributor who did install in a 
temporary directory at some point in packaging - the case of someone 
wanting to know if a tool is broken before running "make install" to 
replace a previous installation is not the usual one, and "make install" 
itself should just be copying from such a staging directory in the build 
tree.

Basic features for a new test harness for GCC would include, in my view:

* Starting from installed testing as the basic case (run a test-running 
command, such as runtest at present, to run the tests for the tools in the 
PATH), with more complicated commands for build-tree testing meaning 
passing a more complicated configuration file (generated by the build 
process) to the test-running command.  Definitely do not hardcode details 
of build-tree structure in the test tool itself as DejaGnu does.

* Well-defined enumeration of test assertions independent of running the 
tests.  In DejaGnu the test assertions are the things after "PASS:" or 
"FAIL:" - but there can be duplicate names, and the set of names can 
depend on the results of the tests, or on the build directory name, when 
tests or testsuites are buggy; the system should make it inherently 
impossible to have such problems, so that it is always possible to examine 
results and see what tests did not run, for example.  Similarly, test 
assertion names should be as stable as possible to facilitate comparisons 
when the testsuite itself as well as the compilers being testing may have 
changed.

* Structured results so that build commands and output etc. are 
unambiguously associated with a particular test.  (For DejaGnu you need to 
do heuristic processing of .log files.)

* Well-defined interfaces for defining a target board (i.e., how you run 
a test program on the target, get its output, copy inputs to the board, 
etc.), a host board (for testing the tools themselves on a remote host), a 
testsuite.

* Ability to represent expectations for test results just as flexibly as 
the present XFAILs system.  Some test systems seem to be based on a 
presumption that the expected state is no failures at all, and this is not 
realistic for GCC.

-- 
Joseph S. Myers
joseph@codesourcery.com