[patch/rfc] Remove all setup_xfail's from testsuite/gdb.mi/

[Andrew Cagney]

> Andrew Cagney wrote:
> Fernando,
> 
> What is the resolution of this thread?
> http://sources.redhat.com/ml/gdb-patches/2002-10/msg00508.html
> (you'll probably want to read all of it :-)
> (I'm now going back and emptying my mail box :-()
> 
> Andrew
> 
> 
> Hi Andrew,
> 
> I am just starting to think about this, so bear with me while I think aloud a little bit :-)  There are lots of good points made on that discussion and I don't think this is a clear cut yet or no, lets vote, or anything like that.  I am trying to see if I can collect all good suggestions and propose a course of action.
> 
> It seems that a widespread concern is not to through the baby with the water. On the other hand there is the feeling that we do not act now nothing will ever happen.  And we all agree that many xfails were added incorrectly.
> 
> Independently of the "big plan", one thing is clear.  If you know that a xfail is incorrect (I mean, know for good), then it should be ripped off. But read on...
> 
> Maybe we may not be in full agreement here, but I think it would be nice to enter a bug report even if does not include a full analysis -- that should come eventually.  Something like "something is wrong with the '--xxx-yyy' command output" may eventually cause someone to look into it.  And you can make the xfail a kfail in that case, which is much better.  But I know it is much more work than just removing the xfail and letting the test fail -- you would have to enter 28 bug reports just for the MI tests.  Still sounds like a better approach to me, but there is no consensus on it.

Some things I think worth noting.

In this process, I don't think it is reasonable make it a requirement 
that people fully kfail/analize old badly specified xfails.  Rather I 
think it is only reasonable to ask people to ensure that new tests, or 
changes to existing tests, meet the new requirement (there must be a bug 
report).

An initial bar is established, and then, over time, that bar is raised.

The difference is important.  It avoids the problem of developement 
stalling because of the impossible requirement fixing all the old code 
before the new code can be approved.

As a specific example, the i386 has an apparently low failure rate. 
That rate is badly misleading and the real number of failures is much 
higher :-(  It's just that those failures have been [intentionally] 
camoflaged using xfail.  It would be unfortunate if people, for the 
i386, tried to use that false result (almost zero fails) when initally 
setting the bar.

This is also why I think the xfail's should simply be yanked.  It acts 
as a one time reset of gdb's test results, restoring them to their true 
values.   While this may cause the bar to start out lower than some 
would like,  I think that is far better and far more realistic than 
trying to start with a bar falsely set too high.

Andrew