apply gnulib test suite to glibc

[Carlos O'Donell]

On Sat, Apr 21, 2012 at 6:20 PM, Bruno Haible <bruno@clisp.org> wrote:
> Hi,
>
> In his presentation [1] Carlos O'Donell writes:
>  "Current testsuite is very minimal"
>
> GNU gnulib has a growing test suite of POSIX and glibc headers and functions.
> Currently it covers ca. 40 header files and ca. 280 functions. (List below.)
> These tests are known to pass on glibc platforms with 6 different CPU types,
> as well as on major Unix systems (from Mac OS X to Cygwin), modulo platform
> specific bugs on these non-glibc systems.
>
> In several cases this test suite has already uncovered glibc bugs or Linux
> kernel bugs.
>
> The tests are generally written according to the POSIX standard. Many
> tested behaviours are also "common sense". Only few tests could be
> characterized as "hairy".
>
> I can recommend the GNU gnulib test suite to anyone who is about to
> release a libc (or OS with a libc), as a sanity check.
>
> The procedure to run it is:
>  1. On a development machine (a machine with GNU tools), checkout GNU gnulib
>     <https://savannah.gnu.org/projects/gnulib>
>     and run
>     $ ./gnulib-tool --create-testdir --dir=/tmp/testdir \
>                     --with-tests --single-configure \
>                     `./posix-modules`
>  2. Copy the /tmp/testdir directory to the target machine, if it is a
>     different machine.
>  3. In /tmp/testdir, run
>     $ ./configure CPPFLAGS="-Wall"
>     $ make
>     Verify that Gnulib has built no replacement/workaround code
>     (gllib/*.o files) - if so, this indicates problems in the libc.
>     $ make check
>
> What do you think of it? Could it become part of the pre-release testing
> of Glibc to apply this test suite? Would it make sense for some of the
> tests to be copied to Glibc? (The tests are copyright-assigned to the
> FSF already.) I am open to suggestions and collaboration.

I think this would be a great idea.

As Joseph points out the procedure needs some tweaking.

Would you mind creating a glibc wiki page about running the gnulib
testsuite with glibc?

Once we get the procedure down we can link to it from a release process.

Cheers,
Carlos.