Re: A few things we've been up to at Apple lately

[Jason Molenda <jason-swarelist at molenda dot com>]

On Sat, Nov 09, 2002 at 09:06:21AM -0500, Andrew Cagney wrote:

> > diverged from the FSF style; fixing our code to behave correctly;
> > and filing Apple internal bug reports for all the known failures.
> > We're down to 15-20 unfiled fails now, and many of those are known
> > failures in the generic code.
> 
> Do you know about KFAIL?
> (Thanks for the update).

Yeah, I saw some a mention of that as I was setting everything up.
What do you think about accepting KFAIL patch submissions
*-apple-darwin* with references to an internal-only-to-Apple bug
database?  Here's what we've done so far:

For my first cut, I was adding bug report #'s right into the test
case name -- it made it easy to grep for and fix when we close the
bug reports.  Personally, I wanted to see the test failures in my
make check output even though they were known fails so I wouldn't
forget them. :-)  

Klee had a different take on the matter.  (1) he didn't want unnecessary
changes to the testsuite, which inevitably cause merge headaches.  (2)
he didn't want to see known failures that have bugs filed--he wants it to
be easy to see if there are new, unknown problems.  (akin to the KFAIL
behavior)

He added some functionality to dejagnu so that our
dejagnu/baseboards/macosx.exp file contains a list of tests that
are known to fail on our system, e.g.

# 3061801 - gdb doesn't deliver signals to inferior proccess
setup_site_fail "gdb.base/annota1.exp" "*" "send SIGUSR1 (r. 3061801)" "3061801"

# 3049981 - unable to pass structs containing chars to functions called from gdbsetup_site_fail "gdb.base/call-rt-st.exp" "powerpc-apple-darwin*" "print print_three_chars(*three_char) (r. 3049981)" "3049981"

He added a --site command line option to runtest where he can
specify types of setup_site failures to suppress -- xfails, kfails,
or fails, and any testsuite failure that's listed in macosx.exp
will be suppressed in the make check output.

Well anyway, it's an interesting different take on the problem.  If
we could get setup_kfail's added to the FSF sources, I we'd go for
that.  The only minor nit I see with kfail as it's implemented right
now is that it prints "(PRMS: 3061801)" which could potentially confuse
someone into thinking these bug reports are in the gdb PRMS database
somewhere..

J