[2.20] [3/6] Support expected failures in .test-result files

[Joseph S. Myers]

On Thu, 13 Feb 2014, Carlos O'Donell wrote:

> IMO we're going to have to extend this to be more flexible for
> XFAILS, and I think that data will have to be kept outside of
> glibc.

If (as I think we should, but which is a separate matter) we start to use 
XFAILs for architecture-specific cases, I think we should put as much in 
glibc as possible.  That is, where we presently put information about 
conditions for known failures on per-release wiki pages, as much as 
possible of that should go in conditions in glibc for XFAILing tests (or 
skipping relevant parts of them, etc.), and associated comments, so that 
distributors have less work comparing failures with our lists of known 
issues, and generally the out-of-the-box experience with the testsuite is 
as good as possible.

If a failure is expected by more than one distributor, they shouldn't need 
to track it separately; glibc should be automatically marking it expected 
under whatever the relevant conditions are, sharing the work of updating 
the expectations in glibc among the distributors.

(With such an XFAILing practice, as I noted, architecture / distribution 
maintainers should then review XPASSes they see at release time to see if 
any are obsolete.)

(Part of the point of this patch series is to obsolete various 
distribution-specific means of generating lists of failures from a 
testsuite run, replacing them by a single system for generating summaries 
of test results to which all users and distributors can contribute 
improvements.)

-- 
Joseph S. Myers
joseph@codesourcery.com