From efc2eef178d9daf6d9e26b6d12f0d1dc205e4521 Mon Sep 17 00:00:00 2001 From: Alexandre Duret-Lutz Date: Sun, 19 Jan 2003 15:36:41 +0000 Subject: [PATCH] * tests/README: New file. --- ChangeLog | 2 + tests/Makefile.in | 2 +- tests/README | 130 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 133 insertions(+), 1 deletion(-) create mode 100644 tests/README diff --git a/ChangeLog b/ChangeLog index ce469c48..8548c4b7 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,7 @@ 2003-01-19 Alexandre Duret-Lutz + * tests/README: New file. + * automake.texi (Conditional Programs): Show a sample Makefile.am. Remind $(EXEEXT) must be appended to configure substitutions. Show how Automake conditionals can be used instead. diff --git a/tests/Makefile.in b/tests/Makefile.in index c69f6e07..5fa62a16 100644 --- a/tests/Makefile.in +++ b/tests/Makefile.in @@ -549,7 +549,7 @@ subdir = tests mkinstalldirs = $(SHELL) $(top_srcdir)/lib/mkinstalldirs CONFIG_CLEAN_FILES = defs DIST_SOURCES = -DIST_COMMON = Makefile.am Makefile.in defs.in +DIST_COMMON = README Makefile.am Makefile.in defs.in all: all-am .SUFFIXES: diff --git a/tests/README b/tests/README new file mode 100644 index 00000000..614342a4 --- /dev/null +++ b/tests/README @@ -0,0 +1,130 @@ + The Automake test suite + + +User interface +============== + + +Running all tests +----------------- + + make check + + +Interpretation +-------------- + + Successes: + PASS - success + XFAIL - expected failure + + Failures: + FAIL - failure + XPASS - unexpected success + + Other: + SKIP - skipped tests (third party tools not available) + + +Getting details from failures +----------------------------- + + Each test is a script. In a non-VPATH build you can run them + directly, they will be verbose. + + Otherwise, you can invoke make as follows, just replace the list + of tests by those you want to check. + + env VERBOSE=x TESTS='first.test second.test ...' make -e check + + +Reporting failures +------------------ + + Send verbose output of failing tests to , + along with the usual version numbers (which Automake, which + Autoconf, which operating system, which make version, which shell, + etc.) + + + +Writing test cases +================== + + +Do +-- + + If you plan to fix a bug, write the test case first. This way you'll + make sure the test catches the bug, and that is succeeds once you have + fixed the bug. + + Add a copyright/license paragraph. + + Explain what the test does. + + Cite the PR number (if any), and the original reporter (if any), so + we can find or ask for information if needed. + + Use `required=...' for required tools. + + Include ./defs (see other tests). + + Use `set -e' to catch failures you might not have thought of. + + ./defs sets a skeleton configure.in. If possible, append to this + file. In some cases you'll have to overwrite it, but this should + be the exception. Note that configure.in registers Makefile.in + but do not output anything by default. If you need ./configure + to create Makefile, append AC_OUTPUT to configure.in. + + Use $ACLOCAL, $AUTOMAKE, $AUTOCONF, $AUTOUPDATE, $AUTOHEADER, + $PERL, $MAKE, $EGREP, and $FGREP, instead of the corresponding + commands. + + Use $sleep when you have to make sure that some file is newer + than another. + + Use `cat' or `grep' to display (part of) files that may be + interesting for debugging, so that when a user send a verbose + output we don't have to ask him for more details. + + It's more important to make sure that a feature works, than + make sure that Automake's output looks correct. It might look + correct and still fails to work. In other words, prefer + running `make' over grepping `Makefile.in' (or do both). + + If you run $AUTOMAKE or $AUTOCONF several times in the same test + and change `configure.in' by the meantime, do + rm -rf autom4te.cache + before the following runs. On fast machines the new `configure.in' + could otherwise have the same timestamp as the old `autom4te.cache'. + + Before commit: make sure the test is executable, add the tests to + TESTS in Makefile.am, add it to XFAIL_TESTS in addition if needed, + write a ChangeLog entry, send the diff to . + + +Do not +------ + + Do not test an Automake error with `$AUTOMAKE && exit 1', or in three + years we'll discover that this test failed for some other bogus reason. + This happened many times. Better use something like + $AUTOMAKE 2>stderr && exit 1 + cat stderr + grep 'expected diagnostic' stderr + (Note this doesn't prevent the test from failing for another + reason, but at least it makes sure the original error is still + here.) + + Do not override Makefile variables using make arguments, as in + $MAKE ANSI2KNR=./ansi2knr U=_ all + this is not portable for recursive targets (targets that + call a sub-make may not pass `ANSI2KNR=./ansi2knr U=_' along). + Use the following instead. + ANSI2KNR=./ansi2knr U=_ $MAKE -e all + + Do not send a test case without signing a copyright disclaimer. + See http://sources.redhat.com/automake/contribute.html or + ask for details. -- 2.43.5