This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: [2.20] [3/6] Support expected failures in .test-result files
- From: "Joseph S. Myers" <joseph at codesourcery dot com>
- To: Carlos O'Donell <carlos at redhat dot com>
- Cc: Brooks Moses <bmoses at google dot com>, libc-alpha <libc-alpha at sourceware dot org>
- Date: Fri, 14 Feb 2014 13:30:04 +0000
- Subject: Re: [2.20] [3/6] Support expected failures in .test-result files
- Authentication-results: sourceware.org; auth=none
- References: <Pine dot LNX dot 4 dot 64 dot 1401100208000 dot 9412 at digraph dot polyomino dot org dot uk> <Pine dot LNX dot 4 dot 64 dot 1401100211530 dot 9412 at digraph dot polyomino dot org dot uk> <52F95BE2 dot 3070004 at google dot com> <Pine dot LNX dot 4 dot 64 dot 1402102315100 dot 26591 at digraph dot polyomino dot org dot uk> <CAOxa4Kpwqt3D6M9D102KvoN_rq1e8AhFpzdO6RDoC=2-SmM29A at mail dot gmail dot com> <52FD9FBB dot 4030801 at redhat dot com>
On Thu, 13 Feb 2014, Carlos O'Donell wrote:
> IMO we're going to have to extend this to be more flexible for
> XFAILS, and I think that data will have to be kept outside of
> glibc.
If (as I think we should, but which is a separate matter) we start to use
XFAILs for architecture-specific cases, I think we should put as much in
glibc as possible. That is, where we presently put information about
conditions for known failures on per-release wiki pages, as much as
possible of that should go in conditions in glibc for XFAILing tests (or
skipping relevant parts of them, etc.), and associated comments, so that
distributors have less work comparing failures with our lists of known
issues, and generally the out-of-the-box experience with the testsuite is
as good as possible.
If a failure is expected by more than one distributor, they shouldn't need
to track it separately; glibc should be automatically marking it expected
under whatever the relevant conditions are, sharing the work of updating
the expectations in glibc among the distributors.
(With such an XFAILing practice, as I noted, architecture / distribution
maintainers should then review XPASSes they see at release time to see if
any are obsolete.)
(Part of the point of this patch series is to obsolete various
distribution-specific means of generating lists of failures from a
testsuite run, replacing them by a single system for generating summaries
of test results to which all users and distributors can contribute
improvements.)
--
Joseph S. Myers
joseph@codesourcery.com