Differences between revisions 26 and 27
Revision 26 as of 2013-04-17 13:19:19
Size: 11219
Editor: PedroAlves
Comment: add hint on how to run a single test
Revision 27 as of 2013-04-17 13:37:38
Size: 11269
Editor: PedroAlves
Comment: clarify with_test_prefix example
Deletions are marked like this. Additions are marked like this.
Line 167: Line 167:
The {{{with_test_prefix}}} function is handy for prefixing test messages with a pass identifier. The {{{with_test_prefix}}} function is one way to address this -- it allows easily prefixing a set of test messages with a unique pass identifier.

This page collects notes on how to write testcases in the GDB testsuite, traps to avoid, etc.

When adding a feature or fixing a bug which is not covered by the testsuite, you should write a new testcase or extend an existing one.

Building the Example Program

The typical template to start with development of a new testcase can be copied from:

gdb/testsuite/gdb.base/template.exp

# Copyright 2013 Free Software Foundation, Inc.

# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

standard_testfile

if { [prepare_for_testing ${testfile}.exp ${testfile} ${srcfile}] } {
    return -1
}

if ![runto_main] {
    untested "could not run to main"
    return -1
}

gdb_test "advance [gdb_get_line_number "next-line"]" "next-line.*" "advance to next-line"

gdb/testsuite/gdb.base/template.c

/* This testcase is part of GDB, the GNU debugger.

   Copyright 2013 Free Software Foundation, Inc.

   This program is free software; you can redistribute it and/or modify
   it under the terms of the GNU General Public License as published by
   the Free Software Foundation; either version 3 of the License, or
   (at your option) any later version.

   This program is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
   GNU General Public License for more details.

   You should have received a copy of the GNU General Public License
   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */

int
main (void)
{
  int i = 0;

  return i; /* next-line */
}

See gdb/testsuite/lib/gdb.exp for the list of routines available and their documentation.

Performing a Test

Testing a command in GDB should be done using either gdb_test, gdb_test_no_output or gdb_test_multiple. Please do not use gdb_expect as the latter does not automatically handle all known error situations (internal errors, timeouts, etc). Known failing new testcases must produce KFAIL (GDB problem) or XFAIL (environment problem).

When matching GDB output in a test, you have to wait for the prompt explicitly. If you ever leave expect in a state where two prompts should be arriving, it won't know which is which; if it sees them separately, you can get out of sync and all tests will fail with unknown output. This means your testcase will have a race condition and may pass or fail intermittently. gdb_test waits for the GDB prompt, gdb_test_multiple doesn't. When using the latter, you have to explicitly put the GDB prompt in your regular expressions.

Also, gdb_test puts an End-of-Line marker at the end of your regular expression.

    gdb_test "break $srcfile:factorial" \
        "Breakpoint.*at.* file .*$srcfile, line.*" \
        "breakpoint function in file"

    gdb_test_no_output "set \$foo=$bp_location11" \
        "set convenience variable \$foo to bp_location11"

    set test "print/x $var"
    gdb_test_multiple "$test" $test {
        -re "\\$\[0-9\]+ = [string_to_regexp $val].*\r\n$gdb_prompt $" {
            pass $test
        }
        -re "\\$\[0-9\]+ = $addr.*\r\n$gdb_prompt $" {
            fail "$test (prints just address)"
        }
        -re "\\$\[0-9\]+ = 0x\[a-f0-9\]+.*\r\n$gdb_prompt $" {
            fail "$test (prints unexpected address)"
        }

If you want to send a command to GDB without increasing the test count, use gdb_test or gdb_test_no_output with an empty message (i.e., pass an empty string as the third argument). Do not use send_gdb since it won't wait for the prompt and thus will make the testcase get out of sync with the GDB output.

Convenient variables defined for use in output matching

Dejagnu defines the following convenient variables (see <dejagnu-prefix>/share/dejagnu/runtest.exp):

  • hex: Matches any hexadecimal number, including the 0x prefix.

  • decimal: Matches a decimal number. It does not handle + or - signs, however, and thus cannot be used to match negative numbers.

The GDB testsuite framework also defines the following convenient variables:

  • octal: Matches an octal number. The number does not need to start with the digit zero.

  • gdb_prompt: Matches the GDB prompt, and the GDB prompt alone. Beware that id does not match the space that usually follows it.

  • inferior_exited_re: Matches the notification printed by GDB when the inferior exited, either normally or not.

  • fullname_syntax: Matches what GDB considers an absolute path. The regexp is not platform-specific, and matches all known forms of absolute paths, including the Windows one, even when testing a Unix debugger on a Unix platform.

  • EXEEXT: The extension used for executables. It looks like this variable is set using the environment variable EXEEXT, or else to the empty string. (FIXME: Is this correct?)

Running the Example Program in GDB

Use either gdb_run_cmd, runto, or runto_main. Do not use gdb_test with either "run" or "start" as the command being sent. This would not work when debugging with gdbserver, for instance. The same applies to gdb_start_cmd.

Note that gdb_run_cmd only sends the command needed to start the execution. It should be followed by a test that verifies the output. For example:

gdb_run_cmd
gdb_test "" ".*Breakpoint.*1.*callee.*14.*" "run"

Similarly to when building the program fails, if you want the testcase to abort, please use return.

Inserting a Breakpoint

It is fine to use gdb_test to insert the breakpoint and match the output in the usual way. In fact, the vast majority of testcase do it this way. However, the gdb.exp library provides a set of routines that can be convenient when a breakpoint needs to be inserted:

  • gdb_get_line_number: This routine allows us to avoid hard-coding the line number where to insert the breakpoint. This is useful when the line number changes because of some adjustment in the code, or even some comment update as we do every year to add the new year in the copyright notices.

  • gdb_breakpoint: This routines simplifies the insertion of the breakpoint and the matching of the output.

gdb_breakpoint ${srcfile}:[gdb_get_line_number "watchpoint-here"]

You should specify the source file, unless you are specifically testing the "break LINE" command. For example, if GDB finds debug information for _start, the default source file may not be what you expect!

Continuing the Execution until Reaching a Breakpoint

It is fine to use gdb_test to continue the execution and match the output in the usual way. However, there is a function provided by gdb.exp which might be useful in simplifying the writing of the test: gdb_continue_to_breakpoint.

gdb_continue_to_breakpoint "Place to set the watchpoint" ".*pattern that should be seen.*"

Implementation note: We should rewrite this routine using gdb_test_multiple.

Restarting GDB

We sometimes need to restart GDB in a testcase. Rather than using the "gdb_exit; gdb_start; gdb_reinitialize_dir; gdb_load" sequence, you can use the clean_restart function.

clean_restart $executable

Dos and Don'ts

Make sure test messages are unique

We sometimes need to test the same operation multiple times in a testcase, so the test results would be the same and duplicated, which should be avoided. After running your new or modified testcase, please check that test messages are unique

$ make check RUNTESTFLAGS="mytest.exp"
$ cat testsuite/gdb.sum | grep "PASS" | sort | uniq -c | sort -n

If you see something like this below in the output, it means messages are not unique. The first column indicates how many times a single line appears, so you want to only see 1 there.

2 PASS: gdb.trace/trace-break.exp: 5 trace ftrace ftrace@0: ftrace after_set_point
2 PASS: gdb.trace/trace-break.exp: 5 trace ftrace trace@1: trace set_point

The with_test_prefix function is one way to address this -- it allows easily prefixing a set of test messages with a unique pass identifier.

Make sure test executables are unique

Each test compilation should produce a separate executable.

It is good practice for each test to have its own source code, but if for some reason a new test reuses the sources of another existing test, the new test shall compile to its own executable. E.g., if newtest.exp wants to reuse oldtest.c, then the new test should make sure to not overwrite the old tests' executable. The simplest way to do this is to use standard_testfile, which sets various globals based on the current .exp file's name.

Similarly, if a single .exp test file builds more than one executable, perhaps to exercise several variants of the same test, then each of the compilations should produce a separate executable. This makes it easier to reproduce test problems by hand. Each such executable, or shared library, should include the .exp file's base name; if the file uses standard_testfile then this is in the global testfile.

Avoid "gdb_suppress_entire_file"

Please avoid using gdb_suppress_entire_file. This function is now deprecated. Use untested/return instead.

"untested" calls

In untested calls, please spell out the reason the test ends up untested, instead of just writing the test name, as with the latter we just end up with the test name duplicated in the gdb.sum output. For example:

untested mytest.exp

results in the not very helpful:

UNTESTED: gdb.base/mytest.exp: mytest.exp

A better untested call would look like:

untested "could not compile test program"

which results in the helpful:

UNTESTED: gdb.base/mytest.exp: could not compile test program

There are many bad examples in the testsuite, but we don't want to add more.

Do not refer directly to "$objdir"

Do not refer directly to $objdir. Instead, use the standard_output_file proc to compute a file name from a base name.

Do not write files into "."

Do not write files into . This is not safe when tests are run in parallel. Again, use standard_output_file to control where output is directed.

None: GDBTestcaseCookbook (last edited 2014-03-29 13:55:19 by StanShebs)

All content (C) 2008 Free Software Foundation. For terms of use, redistribution, and modification, please see the WikiLicense page.