This page collects useful notes on testing GDB, including running the GDB testsuite.
- Caveat emptor
- Running GDB under Valgrind in the testsuite
- RUNTESTFLAGS and parallel (-j) testing
- Passing an option to GDB / Running the whole test suite in a non-default mode
- Passing an option to GCC
- Changing the compiler used to build the testcases
- Running all tests in a testsuite directory
- Running massif
- The Awk Regression Index (ARI)
- Testing gdbserver in a native configuration
- Testing gdbserver in a remote cross-target configuration
- Coverage Testing
Jim Blandy had the following insight at the gdb-patches@ mailing list about analyzing testsuite results:
One unfortunate characteristic of the GDB testsuite is that, even if GDB is horribly broken, many tests will still pass. You can't have much confidence that things are basically working with 356 failures. You should look through gdb.log and work through a bunch of them to get a feel for the state of things.
For native gdb testing, just run "make check". The testsuite supports some parallel execution. This speeds up testing on most modern machines. E.g., run the testsuite with:
$ make check -j8
Running GDB under Valgrind in the testsuite
$ cd testsuite $ runtest --tool gdb GDB="valgrind ../gdb"
$ make check RUNTESTFLAGS='GDB=\"valgrind ../gdb\"'
Suitably augment the Valgrind command line as desired. You may want to increase the testsuite's timeout threshold to avoid failures due to GDB running slower. Edit the file <build-dir>/gdb/testsuite/site.exp and add the line:
set gdb_test_timeout 120
RUNTESTFLAGS and parallel (-j) testing
RUNTESTFLAGS disables parallel testing. To re-enable it, set FORCE_PARALLEL as well. E.g.,
make check RUNTESTFLAGS="--target_board=native-gdbserver" FORCE_PARALLEL="1" -j8
That runs the testsuite against a gdbserver running locally on your machine (the testsuite spawns it for you), in parallel mode.
Passing an option to GDB / Running the whole test suite in a non-default mode
Sometimes it is useful to test running GDB in a specific non-default mode or option. For example, you might want to run the whole test suite forcing breakpoints always-inserted on mode. There's usually a CLI command to toggle that mode on, but no GDB specific command line option or specific test suite knob. Instead, we can make use of the "-ex" GDB command line option, which allows running any command when GDB starts, and couple that with the test suite's GDBFLAGS global, that is available/reserved for the user.
From the command line:
make check RUNTESTFLAGS="GDBFLAGS='-ex set\ breakpoint\ always-inserted\ on'"
Or, from a board file:
set GDBFLAGS "-ex \"set break always-inserted on\""
Passing an option to GCC
Sometimes it is useful to pass a specific option to GCC. For example, you might want to run the test suite using -gdwarf-4:
make check RUNTESTFLAGS="--target_board unix/gdb:debug_flags=-gdwarf-4"
You can also use:
make check RUNTESTFLAGS="CFLAGS_FOR_TARGET='-gdwarf-4' gdb.base/test.exp"
Changing the compiler used to build the testcases
make check RUNTESTFLAGS='--target_board "unix/-m32" CC_FOR_TARGET="/path/to/some/other/gcc"'
You can also specify other compilers using parameters such as CXX_FOR_TARGET, F77_FOR_TARGET and AS_FOR_TARGET.
Running all tests in a testsuite directory
make check RUNTESTFLAGS="--directory=gdb.mi"
If you run massif (the valgrind memory profiler) on GDB, you will quickly see that GDB uses a number of wrapper functions when calling memory allocators. This list of massif options has proven useful to see what is really happening:
--alloc-fn=PyObject_Malloc --alloc-fn=xmalloc --alloc-fn=xrealloc --alloc-fn=xcalloc --alloc-fn=_objalloc_alloc --alloc-fn=objalloc_create --alloc-fn=_obstack_newchunk --alloc-fn=htab_expand --alloc-fn=xstrdup --alloc-fn=xzalloc --alloc-fn=_obstack_begin --alloc-fn=bfd_alloc --alloc-fn=bfd_zalloc
The Awk Regression Index (ARI)
The ARI stands for Awk Regression index and is a script (named gdb_ari.sh) that is run once a day by the machine on which the website is.
The result of this script is transformed into a web page: http://sourceware.org/gdb/current/ari/
Most used files are accessible at the bottom of the web page, in particular gdb_ari.sh is at: http://sourceware.org/gdb/current/ari/gdb_ari.sh
Testing gdbserver in a native configuration
You can use the boards/native-gdbserver.exp board file to configure the testsuite to run natively using gdbserver. To do so, follow the instructions provided in introductory comment of that file.
Testing gdbserver in a remote cross-target configuration
To run gdbserver on a remote machine with a different architecture, first you might need a cross-compiler properly configured in your local machine. Also, you need to build gdb and gdbserver for the specific target architecture. In this case, the GDB you're using for tests should be build with the option "--with-sysroot=/path/to/sysroot" pointing to the sysroot directory where the cross-target libraries are located or "--with-sysroot=remote:" (don't forget the colon at the end) if the libraries can be retrieved from the target machine.
Remember to check if gdb was built with expat support (--with-expat option, enabled by default), otherwise it will not be able to communicate properly with gdbserver. Moreover, you might need also dejagnu version 1.5 (or newer), since GDB testsuite cannot run due to a bug existing on dejagnu's previous versions.
To transfer binary files and run gdbserver remotely, there are alternatives such as scp/ssh or rsh/ftp. The example below uses scp/ssh. In this case, it is highly recommendable to use public keys for ssh authentication and ssh session multiplexing (setting up the directives ControlPath and ControlMaster in $HOME/.ssh/config file) in order to speed up the tests.
The remaining steps follow a similar setup from when you are testing gdbserver locally:
- Create an empty file named site.exp.
- Create a directory named boards in the same location as site.exp.
- Create a file named remote-gdbserver.exp in the boards directory (see below).
- Set the DEJAGNU environment variable to point to the empty site.exp.
- Run the testsuite with make check-gdb RUNTESTFLAGS="--target_board remote-gdbserver".
Remote Board File
# gdbserver running remotely over ssh load_generic_config "gdbserver" process_multilib_options "" # The default compiler for this target. #set_board_info compiler "[find_gcc]" # If gdbserver runs in a cross-target arch, the testsuite should use a cross-compiler set_board_info compiler "/opt/at4.0/bin/powerpc64-linux-gcc" set_board_info c++compiler "/opt/at4.0/bin/powerpc64-linux-g++" set_board_info rsh_prog /usr/bin/ssh set_board_info rcp_prog /usr/bin/scp set_board_info protocol standard set_board_info username user set_board_info hostname remote.example.com # Path to the gdbserver executable on target board. set_board_info gdb_server_prog /home/user/gdbserver-devel/gdbserver # We will be using the standard GDB remote protocol set_board_info gdb_protocol "remote" # Name of the computer whose socket will be used, if required. set_board_info sockethost "remote.example.com:" # Port ID to use for socket connection #set_board_info gdb,socketport "4004" # Use techniques appropriate to a stub set_board_info use_gdb_stub 1 # This gdbserver can only run a process once per session. set_board_info gdb,do_reload_on_run 1 # There's no support for argument-passing (yet). set_board_info noargs 1 # Can't do input (or output) in the current gdbserver. set_board_info gdb,noinferiorio 1 # Can't do hardware watchpoints, in general (it depends on gdbserver support for your target arch) set_board_info gdb,no_hardware_watchpoints 1
The simplest way to look at test coverage of gdb is to rebuild it with CFLAGS='-fprofile-arcs -ftest-coverage', then run the test suite. You can visualize the results easily with lcov.
make mostlyclean make CFLAGS='-g -fprofile-arcs -ftest-coverage' make check lcov -c -d . > lcov.out genhtml --legend -o coverage lcov.out # see also the -p option