Re: RFC: Known Failures [Was: RFD: Testsuite cases for inferior function calls]

[Fernando Nasser <fnasser at cygnus dot com>]

"Peter.Schauer" wrote:
> 
> KFAILs would be fine with me, as long as they show up in the testsuite run
> output.
> 

Note that *all* results go to the testsuite run output.  You are referring to the
summary file, perhaps.

I am against adding things to the gdb summary file, as it is meant to a more course
level control.  It must contain only exceptions, things that are not usually there,
which would be the FAILs.

The gdb.log file has everything (even PASSes) and we use grep or perl scripts to
extract things from it when we want more than the summary.  As all the information
is in there, the possibilities are numerous, even using it to feed a database,
using the data from the run in conjunction with a database to generate a report etc.

> I understand the desire of release management to produce a `flawless' release,
> but from a user perspective, I'd really like to know about the known failures,
> without grepping gdb.sum for KFAILs.
> 

It is not a release problem.  It is an engineering problem: we want to be able to
clearly know if a change caused regressions, without having to memorize which is
the usual level of FAILs for each host-x-target combination.

What is the problem with grep?  

grep "^KFAIL" > known-fails

will give you what you want.



-- 
Fernando Nasser
Red Hat Canada Ltd.                     E-Mail:  fnasser@cygnus.com
2323 Yonge Street, Suite #300           Tel:  416-482-2661 ext. 311
Toronto, Ontario   M4P 2C9              Fax:  416-482-6299