This is the mail archive of the
gdb-patches@sources.redhat.com
mailing list for the GDB project.
Re: [rfa:testsuite} Overhaul sizeof.exp
- From: Michael Elizabeth Chastain <mec at shout dot net>
- To: drow at mvista dot com, fnasser at redhat dot com
- Cc: ac131313 at cygnus dot com, gdb-patches at sources dot redhat dot com
- Date: Wed, 20 Feb 2002 11:17:34 -0600
- Subject: Re: [rfa:testsuite} Overhaul sizeof.exp
Hmmm, let's start by looking at the alternatives. Suppose someone writes
a new test that revels an unexpected bug in gdb (hi Andrew).
[0] Stqtus quo: we reject the test.
[1] FAIL it: we accept the test and let it FAIL.
[2] XFAIL it: we accept the test and mark it with XFAIL.
[3] KFAIL it: we accept the test and mark it with KFAIL.
I really want that test in the test suite. If the test is in the test
suite, it gets run regularly in a lot of configurations. Anyone can
check the status of the test just by reading the latest Sunday Project
errors-warnings-fails report. Someone who tackles the bug has the saves
time when they start and when they finish. And so on.
My preference order is [2] > [1] > [3] > [0]. All but [0] are
acceptable to me.
I understand that [1] has a problem because it looks similar to a
regression. I don't see that in my tables because they distinguish
between "old test that PASSed before and FAILs now" and "new test with
no result before and FAILs now". But I acknowledge that other people
oppose [1].
[3] has a problem because KFAIL does not exist. If the senior maintainers
decide to approve [3] then I can add it to my analysis tables right away,
but someone will have to add it to dejagnu.
I agree with Daniel Jacobowitz; I think [2] is the right thing to do.
It's also practical because we can start today.
@item XFAIL
@kindex XFAIL
@cindex expected failure
@cindex failing test, expected
A test failed, but it was expected to fail. This result indicates no
change in a known bug. If a test fails because the operating system
where the test runs lacks some facility required by the test, the
outcome is @code{UNSUPPORTED} instead.
@item UNSUPPORTED
@kindex UNSUPPORTED
@cindex unsupported test
@cindex test, unsupported
A test depends on a conditionally available feature that does not exist
(in the configured testing environment). For example, you can use this
outcome to report on a test case that does not work on a particular
target because its operating system support does not include a required
subroutine.
@end table
Michael C