This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: Generate .test-result files for ordinary tests
- From: OndÅej BÃlka <neleai at seznam dot cz>
- To: "Joseph S. Myers" <joseph at codesourcery dot com>
- Cc: libc-alpha at sourceware dot org
- Date: Thu, 13 Feb 2014 20:48:23 +0100
- Subject: Re: Generate .test-result files for ordinary tests
- Authentication-results: sourceware.org; auth=none
- References: <Pine dot LNX dot 4 dot 64 dot 1402131848180 dot 11867 at digraph dot polyomino dot org dot uk>
On Thu, Feb 13, 2014 at 06:50:29PM +0000, Joseph S. Myers wrote:
> This patch, an updated version of
> <https://sourceware.org/ml/libc-alpha/2014-01/msg00193.html> now
> proposed for inclusion in glibc, starts the process of generating
> explicit PASS or FAIL status for individual glibc tests. It's based
> on Tomas Dohnalek's patch
> <https://sourceware.org/ml/libc-alpha/2012-10/msg00278.html>, but is
> deliberately more minimal: it doesn't try to cover any tests outside
> of $(tests) / $(xtests) (that's for a later patch), nor does it put
> the result together in an overall summary file (again, a later patch):
> it just generates the .test-result files.
>
> Thus, this patch keeps the overall logic for when a testsuite run
> finishes completely unchanged: a test failing will terminate the run.
> I think we *should* move to a more conventional approach where plain
> "make check" does not terminate for an individual test failure, unless
> e.g. you say "make stop-on-test-failure=y check", but that sort of
> policy change is best done as a separate patch once the infrastructure
> is in place to generate summary files for completed test runs (which
> will entirely consist of PASS and XFAIL lines if the testsuite run
> reaches the point of generating them, until such a policy change is
> made).
>
> Tested x86_64.
>
> Changes relative to the previous version: the way test names are
> computed has changed (in the previous version, the test
> $(objpfx)de/libc.cat in catgets/ would get the name catgets/libc.cat
> when catgets/de/libc.cat seems a more appropriate name for the test),
> and evaluate-test.sh records the original exit status in the
> test-result file as suggested by Brooks.
>
looks ok.