• Testing
• Smoke_test

Smoke test

This document describes how to use the API Sanity Checker tool for smoke testing of glibc releases. The tool generates input data for parameters (reasonable in most, but unfortunately not all, cases) and compose simple test cases for every function in the API through the analysis of declarations in header files. The tool is able to run all generated test cases and detect segfaults (SEGV), aborts (ABRT), all kinds of emitted signals (FPE, BUS, ILL, etc.), non-zero exit codes and program hanging.

WARNING: The tool invokes all functions with random arguments and can damage your system. It's highly recommended to run it on the virtual machine.

Installation

The tool needs G++, Perl 5 and GNU Binutils to be installed on your host.

To install the tool run

$ wget https://github.com/downloads/lvc/api-sanity-checker/api-sanity-checker-1.12.10.tar.gz
$ tar -xzf api-sanity-checker-1.12.10.tar.gz
$ cd api-sanity-checker-1.12.10
$ sudo perl Makefile.pl -install --prefix=/usr

To verify that it is installed correctly and it works on your host run

$ cd tmp/
$ api-sanity-checker -test

Install glibc to a local directory (/home/user/glibc-2.16 for example)

$ wget http://ftp.gnu.org/gnu/glibc/glibc-2.16.0.tar.gz
$ tar -xzf glibc-2.16.0.tar.gz
$ cd glibc-2.16.0
$ mkdir build
$ cd build
$ ../configure --prefix=/home/user/glibc-2.16
$ make
$ make install

Create XML-descriptor file (glibc-descriptor.xml)

<headers>
    {RELPATH}/include/
</headers>

<libs>
    {RELPATH}/lib/
</libs>

<skip_headers>
    bits/
    elf.h
    vm86.h
    regexp.h
    /rpcsvc/
</skip_headers>

<skip_libs>
    gconv/
</skip_libs>

<include_order>
    stdio.h:libio.h
    wchar.h:stdio.h
</include_order>

<gcc_options>
    -D_GNU_SOURCE
    -Wl,--dynamic-linker={RELPATH}/lib/ld-linux.so.2
</gcc_options>

The last option of the descriptor forces test cases to be loaded by the suitable linker.

Usage

Generate tests

To generate, build and execute tests run

$ api-sanity-checker -l glibc -d glibc-descriptor.xml -vnum 2.16 -relpath /home/user/glibc-2.16 -gen -build -run

The /home/user/glibc-2.16 is an example of path to the installed glibc tree (should be absolute to work properly).

View report

The visual HTML report will be generated to

test_results/glibc/2.16/test_results.html
tests/glibc/2.16/view_tests.html

The report should be analyzed by the glibc maintainer to identify real failures and false positives (API misuses).

Current results

This section provides information about current smoke test results for the releases of glibc in GNU/Linux on x86.

Samples

1. Test report for glibc 2.16 in GNU/Linux on x86.