[testsuite, ada] Convert ACATS to DejaGnu (PR testsuite/37703)

[Rainer Orth]

Mark Mitchell <mark@codesourcery.com> writes:

> On 2/1/2011 6:24 PM, Joseph S. Myers wrote:
>
>> For the general issue: a slow test appears to have served its purpose by 
>> showing up a (target-specific) bug in the compiler. 
>
> Indeed.  However, that doesn't justify having lots of slow tests.  What
> it does justify is investigating the reasons behind a slow test before
> turning it off and/or simplifying it.
>
> Here is what I suggest as a policy.
>
> If a test takes longer than 30 seconds to execute (including both
> compilation and execution of the generated program) on ordinary
> workstation hardware, in a native configuration, then the test should be
> investigated.  If the problem is generic (i.e., not specific to a
> particular host or target) and nobody is actively developing a patch to
> solve the problem, then the test should be flagged as "expensive" and
> only when run when the user explicitly requests "expensive" testing.  If
> the slow execution is considered unreasonable then a PR should be filed,
> just as for any other bug.
>
> I realize that "ordinary workstation hardware" is not a well-defined
> term.  But, there's no need to specify this policy with a high degree of
> rigor; when there is a question, we can use our usual processes for
> reaching consensus, and err on the side of leaving the test as
> "inexpensive".
>
> This policy would help to eliminate the small handful of tests that take
> completely disproportionate amounts of time to execute.

If agreement on this policy could be reached, it would be good if it
could be documented somewhere on gcc.gnu.org.  I haven't found a good
place for that, though.

There's just another such set of expensive tests

testsuite/48283		gcc.dg/graphite/block-[3478].c timeouts

and

middle-end/31827	limits-exprparen.c: Pid 2297 received a SIGSEGV for stack growth failure

has excessive stack space requirements.

Thanks.
	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University