Re: How to abort a test?

[Simon Marchi <simon dot marchi at ericsson dot com>]

On 16-01-14 11:25 AM, Pedro Alves wrote:
>> The test will be aborted, runtest will output a detailed error, but the test will still
>> pass.  Intuitively, I would think that a test that throws an error should automatically
>> be failed or unresolved, since something unexpected happened.
> 
> IIUC, you wouldn't want pass/fail to even be reached, so I don't understand
> what test would you want to fail.

I think unresolved would be appropriate in that case, since it's not the program under test
failed, but the test, or its setup/environment.

According to the Dejagnu doc, unresolved: "... usually means the test did not execute as
expected, and a human being must go over results to determine if it passed or failed (and
to improve the test case). ".  I think that applies here.

So when there is an uncaught exception coming from the test, Dejagnu could issue a:

  unresolved "Uncaught error from test " # needs a better message

The important part is that the runtest command returns a failure return code, so that
automated builds or scripts consider the run as failed.  To me, a test that ends with
an exception is not a test that ran successfully.

>>
>> The only option I see right now would be to fix the whole return chain and add proper
>> error handling everywhere, to exit early when an error happens.  However, that means
>> changing tens (hundreds?) of callsites through the testsuite, which is why I'm
>> looking for alternative solutions first.
> 
> Test usually return early if running to main fails (if ![runto_main]), so maybe it
> won't be that many places.  Maybe some judiciously placed "return -code return"
> hacks saves you a good chunk.  I don't have any other idea.

Many tests use gdb_run_cmd directly without checking the result:

gdb/testsuite $ grep -rin '^gdb_run_cmd$' gdb.* | wc -l
73

So they would need to be changed:

-gdb_run_cmd
+if ![gdb_run_cmd] {
+    fail "Failed to run"
+}

> For the particular case of gdbserver not being present in the target,
> probably the easiest would be to check that earlier, likely before
> runtest, even.  Not ideal, since the testsuite can mix native and gdbserver
> tests, for instance, but...

And it's a bit hard to check in the case of a remote target, given that it's runtest
that knows how to "compute" the remote hostname, username, expected gdbserver path,
etc, from the --target_board.

Simon