|[ < ]||[ > ]||[ << ]||[ Up ]||[ >> ]||[Top]||[Contents]||[Index]||[ ? ]|
Automake has very flexible support for automated test-suites within a
project distribution, which are discussed more fully in the Automake
manual. I have added a simple shell script based testing facility to
Sic using this support – this kind of testing mechanism is perfectly
adequate for command line projects. The tests themselves simply feed
prescribed input to the uninstalled
sic interpreter and
compare the actual output with what is expected.
Here is one of the test scripts:
The tricky part of this script is the first part which discovers the location of (and loads) ‘$srcdir/defs’. It is a little convoluted because it needs to work if the user has compiled the project in a separate build tree – in which case the ‘defs’ file is in a separate source tree and not in the actual directory in which the test is executed.
The ‘defs’ file allows me to factor out the common definitions from each of the test files so that it can be maintained once in a single file that is read by all of the tests:
Having written a few more test scripts, and made sure that they are
working by running them from the command line, all that remains is to
write a suitable ‘Makefile.am’ so that
automake can run
the test suite automatically.
I have used the ‘testsubdir’ macro to run the tests in their own subdirectory so that the directory containing the actual test scripts is not polluted with lots of fallout files generated by running the tests. For completeness I have used a hook target(26) to remove this subdirectory when the user types:
$ make distclean ... rm -rf testSubDir ...
Adding more tests is accomplished by creating a new test script and
adding it to the list in
noinst_SCRIPTS. Remembering to add the
new ‘tests’ subdirectory to ‘configure.in’ and the top-level
‘Makefile.am’, and reconfiguring the project to propagate the
changes into the various generated files, I can run the whole test suite
from the top directory with:
$ make check
It is often useful run tests in isolation, either when developing new tests, or to examine more closely why a test has failed unexpectedly. Having set this test suite up as I did, individual tests can be executed with:
$ VERBOSE=1 make check TESTS=incomplete.test make check-TESTS make: Entering directory /tmp/sic/tests === Running test incomplete.test 1 2 3 PASS: incomplete.test ================== All 1 tests passed ================== make: Leaving directory /tmp/sic/tests $ ls testSubDir/ err errok in.sic ok out
The ‘testSubDir’ subdirectory now contains the expected and actual output from that particular test for both ‘stdout’ and ‘stderr’, and the input file which generated the actual output. Had the test failed, I would be able to look at these files to decide whether there is a bug in the program or simply a bug in the test script. Being able to examine individual tests like this is invaluable, especially when the test suite becomes very large – because you will, naturally, add tests every time you add features to a project or find and fix a bug.
Another alternative to the pure shell based test mechanism I have presented here is the Autotest facility by François Pinard, as used in Autoconf after release 2.13.
Later in A Complex GNU Autotools Project, the Sic project will be revisited to take advantage of some of the more advanced features of GNU Autotools. But first these advanced features will be discussed in the next several chapters – starting, in the next chapter, with a discussion of how GNU Autotools can help you to make a tarred distribution of your own projects.
|[ < ]||[ > ]||[ << ]||[ Up ]||[ >> ]|
This document was generated by Ben Elliston on May 30, 2015 using texi2html 1.82.