Re: [2.20] [6/6] Do not terminate default test runs on test failure

[Brooks Moses <bmoses at google dot com>]

On Mon, Feb 10, 2014 at 4:15 PM, Joseph S. Myers
<joseph@codesourcery.com> wrote:
> On Mon, 10 Feb 2014, Brooks Moses wrote:
>> Other than the documentation comments, this seems good to me -- and the
>> overall patch sequence seems a marked improvement over the status quo.  Even
>> on x86_64, we have a handful of local patches that amount to "delete this test
>> because it is broken", and it will be nice to change those to an XFAIL
>> validation step instead.
>
> I don't know what "because it is broken" means, but unless it's something
> about an environment glibc testing isn't intended to support, it would be
> nice to get them sorted out in glibc proper, not in any local patch or
> validation step!

True, and I strongly agree with the sentiment!  However, often the
failures get sorted out in trunk (or in the Linux kernel), and we are
working on an older release branch, and even if fixes get backported
the right answer is often to xfail locally while discussing upstream.

For what it's worth, here are some discussion threads related to tests
we've disabled locally:
http://cygwin.com/ml/libc-alpha/2010-09/msg00030.html
http://cygwin.com/ml/libc-alpha/2012-02/msg00466.html
https://sourceware.org/ml/libc-alpha/2012-10/msg00021.html

Beyond those, we also disabled nptl/tst-robust8 because of flakiness
(I don't know details there), and rt/tst-cpuclock1 and
rt/tst-cputimer{1,3} because of virtual-machine issues in the test
environment, and I think that's all.

- Brooks