[patch, testsuite] test newlib.string/strcmp-1.c takes too long

[Corinna Vinschen]

On May  2 11:22, Richard Earnshaw wrote:
> On 02/05/12 09:19, Corinna Vinschen wrote:
> > On May  1 17:49, Greta Yorsh wrote:
> >> The test of strcmp in newlib.string testsuite can take too long to run on
> >> simulators, resulting in timeout failures.
> >>
> >> This patch creates a short version of the test, by reducing the length and
> >> the number of strings to compare. This patch also adds a new macro LONG_TEST
> >> to control whether a short or a more comprehensive test should be used.
> >> LONG_TEST is not set by default. 
> > 
> > I'm wondering if that's the right way to solve the problem.  In theory,
> > whatever you set these values to, you could find a simulator which is
> > hitting your timeout values.  Wouldn't it make more sense to raise the
> > timeout values in the testsuite?
> > 
> > 
> > Corinna
> > 
> 
> 
> But to what level?  The test is enormous (over 25 minutes of simulation
> on one run we did), and probably overkill unless you're actively testing
> a rewrite of strcmp.

strcmp is one of the first candidates for a target-specific rewrite in
assembler.  So there's a good chance that you see a rewrite as soon as
a new target gets supported.  Shouldn't we rather opt on the safe side
by default?

> I think defaulting to a cut-down version is a good idea.  It might still
> be necessary to bump the timeout, but the overall cost during normal
> testing doesn't need to be this high.  Don't forget that with multilib
> testing you can end up running that test multiple times.

How long does the cut-down test run in the same scenario as the 25 mins
observed above?


Corinna

-- 
Corinna Vinschen
Cygwin Project Co-Leader
Red Hat