Re: [PATCH v2] Improve analysis of racy testcases

[Pedro Alves <palves at redhat dot com>]

On 03/01/2016 05:51 AM, Sergio Durigan Junior wrote:

>   # Note that we must resort to a recursive make invocation here,
> @@ -190,6 +197,26 @@ DO_RUNTEST = \
>   check-single:
>   	$(DO_RUNTEST) $(RUNTESTFLAGS) $(expanded_tests_or_none)
>
> +check-single-racy:

(...)

> +	$(srcdir)/analyze-racy-logs.py \
> +	  `ls racy_outputs/gdb*.sum` > racy.sum; \

This ls invocation needs updating.

> +	sed -n '/=== gdb Summary ===/,$$ p' racy.sum
> +


> diff --git a/gdb/testsuite/README b/gdb/testsuite/README
> index 6b59027..043a8bd 100644
> --- a/gdb/testsuite/README
> +++ b/gdb/testsuite/README
> @@ -50,6 +50,51 @@ to any non-empty value:
>   If you want to use runtest directly instead of using the Makefile, see
>   the description of GDB_PARALLEL below.
>
> +Racy testcases
> +**************
> +
> +Sometimes, new testcases are added to the testsuite that are not
> +entirely deterministic, and can randomly pass or fail.  We call them
> +"racy testcases", and they can be bothersome when one is comparing
> +different testsuite runs.  In order to help identifying them, it is
> +possible to run the tests several times in a row and ask the testsuite
> +machinery to analyze the results.  To do that, you need to specify the
> +RACY_ITER environment variable to make:

Nicer, thanks.  :-)


> +
> +    # Print the header.
> +    print "\t\t=== gdb tests ===\n"

"gdb racy tests" ?

> +
> +    # Print each test.
> +    for line in sorted (racy_tests):
> +        print line
> +
> +    # Print the summary.
> +    print "\n"
> +    print "\t\t=== gdb Summary ===\n"
> +    print "# of racy testcases:\t\t%d" % len (racy_tests)

AFAICS, it's # of racy tests, not # of racy testcases, right?

Feel free to push with the above addressed.

Thanks,
Pedro Alves