The glibc testsuite

The glibc testsuite can be executed by issuing the 'make check' command from the build directory. The testsuite halts whenever it encounters an error. One may continue by simply issuing the 'make check' command again, but it is recommended that testsuite failures be studied seriously and if necessary, reported in bugzilla.

A typical test case writes out a file with a .out extension in the build directory, that contains the output of the test. This file may be inspected in case of a test case failure to determine what the problem. If you report a test case failure in bugzilla, be sure to include the contents of the relevant .out file as well.

To repeat the test that failed, simply remove the .out file associated with that test, since the presence of the file in the build directory (with a more recent timestamp) implies that the test has already been run. Alternatively, one could simply use 'make -k check' to run the entire testsuite without halting for errors and then inspect the output and the relevant .out files for errors.

There's an additional test target: make xcheck. Running make xcheck includes all the tests of make check but adds some additional tests. These additional tests have requirements on the execution environment that are available for normal execution, e.g. network connectivity, but not necessarily for builds as part of build systems.

Developers should use "make xcheck" instead of "make check".

ABI check

The glibc testsuite contains a number of tests to check that the ABI of glibc does not change. It compares symbol names and versions and static variable sizes in generated binaries with those maintained in *.abilist in the source. The test runs as part of 'make check'. You can also run these separately via "make check-abi".

In order to increase the quality of ABI analysis one can run the following additional tests:

Details about make xcheck specific tests

The list below describes the requirements on the execution environment of the tests that are part of only make xcheck:

  1. resolv/tst-leaks2 nss/bug-erange.c posix/bug-ga2.c
    Requires enough network connectivity to do a DNS lookup on www.gnu.org

  2. nptl/tst-setuid1.c and nptl/tst-setuid1-static.c
    Requires user "nobody" in /etc/passwd or similar, and the ability as the current user to setuid to it.

  3. nptl/tst-mutexpp1.c nptl/tst-mutexpp6.c nptl/tst-mutexpp10.c
    Requires enough privileges to set mutex priority ceiling via pthread_mutexattr_setprioceiling

  4. sunrpc/tst-getmyaddr.c
    Requires at least one non-loopback IP address configured on the system.

  5. sunrpc/thrsvc.c
    Requires loopback to be up and operational with IP address 127.0.0.1

  6. iconv/test-iconvconfig
    Requires an existing gconv-modules.cache to be installed under $(inst_gconvdir)

Writing a test case

  /* This is your test case 'main' method.  */
  static int
  do_test (void)
  {
    /* Test goes here.  */
  }
  
  #define TEST_FUNCTION do_test ()
  #include "../test-skeleton.c"

If you don't define TEST_FUNCTION, then your do_test is considered to have the following signature:

int do_test (int argc, char *argv[])

Known testsuite failures

abi-check

You might see a check failure due to a different size for _nl_default_dirname if you build for a different prefix using the --prefix configure option. The size of _nl_default_dirname depends on the prefix and /usr/share/locale is considered the default and hence the value 0x12. If you see such a difference, you should check that the size corresponds to your prefix, i.e. (length of prefix path + 1) to ensure that you haven't really broken abi with your change.

bug-atexit3 and nptl tests

These failures are again seen when a build is configured for a non-standard prefix (i.e. not /usr). This occurs because the built dynamic linker is unable to find libstdc++ and libgcc_s.so, since its default search path is the build directory with a fallback to the ld.so.cache in the prefix directory i.e. $prefix/etc/ld.so.cache. One could work around these failures in one of the following ways:

tst-eintr1

One may see this failure sporadically. This happens because the kernel fails to reap exiting threads fast enough, eventually resulting an EAGAIN when pthread_create is called within the test. This is currently seen as a kernel limitation.

None: Testing/Testsuite (last edited 2013-06-13 16:15:22 by CarlosODonell)