relying on testsuite results

[Thiago Jung Bauermann]

Hi Paul,

[ Hijacking thread, changing subject. ]

El lun, 06-04-2009 a las 11:59 -0700, Paul Pluzhnikov escribió:
> On Mon, Feb 9, 2009 at 3:16 PM, Pedro Alves <pedro@codesourcery.com> wrote:
> 
> > I'm not 100% sure if we want to fix this, or if we want to
> > adjust the testsuite instead, and making clear in the docs
> > that the variable can be empty.
> 
> Meanwhile the test continues to fail :-(

I'm sincerely curious about why this bothers you to the point of
remembering this discussion from one month ago and fixing it.

I'd love to rely on testsuite results like this, but unfortunately there
are too many "non-deterministic testcases" (as I call them) and they add
a great deal of noise.

So the most use I get from the testsuite is to run regression tests on
each patch I submit, and tediously eyeball the diff looking to see if
any of the PASS<->FAIL flips actuallly mean something.

Do you have a way out of this except going through each of the
unreliable tests and staring at them long enough to see why they flip
(and that can be tricky)?
-- 
[]'s
Thiago Jung Bauermann
IBM Linux Technology Center