Testing against Gnulib's test suite

Contents

  1. Testing against Gnulib's test suite
    1. How to run the test suite
    2. Known problems on Linux
    3. Apparently fixed on Linux as of glibc 2.40
    4. Apparently fixed on Linux as of glibc 2.38
    5. Known problems on Hurd
    6. Investigating new problems

GNU gnulib has a growing test suite of POSIX and glibc headers and functions. As of 2012, it covers ca. 40 header files and ca. 280 functions. These tests are known to pass on glibc platforms with 6 different CPU types, as well as on major Unix systems (from Mac OS X to Cygwin), modulo platform specific bugs on these non-glibc systems.

The tests are generally written according to the POSIX standard. Many tested behaviours are also "common sense". Only few tests could be characterized as "hairy". It is suitable to test glibc against this test suite, as a sanity check.

How to run the test suite

1. On a development machine (a machine with GNU tools, including gperf), as unprivileged user, checkout GNU gnulib https://savannah.gnu.org/projects/gnulib and run 

$ GNULIB_CHECKOUT=`pwd`
$ ${GNULIB_CHECKOUT}/gnulib-tool --create-testdir --dir=/tmp/testdir \
                                 --with-tests --single-configure \
                                 $(${GNULIB_CHECKOUT}/posix-modules)

2. Copy the /tmp/testdir directory to the target machine, if it is a different machine.

3. In /tmp/testdir: For testing a glibc installed in /usr, run 

$ ./configure CPPFLAGS="-Wall"

For testing a glibc built from source with --prefix=$GLIBC_INSTALL_DIR, run 

$ ./configure CC="gcc -Wl,-rpath=$GLIBC_INSTALL_DIR/lib64:$GLIBC_INSTALL_DIR/usr/lib64 -Wl,--dynamic-linker=$GLIBC_INSTALL_DIR/lib64/ld-linux-x86-64.so.2" CPPFLAGS="-nostdinc -I`gcc -print-file-name=include-fixed` -I$GLIBC_INSTALL_DIR/usr/include -I`gcc -print-file-name=include` -Wall"

4. Run 

$ make

Verify that Gnulib has built no replacement/workaround code (gllib/*.o files) - if so, this indicates problems in the libc. 

$ (cd gllib; ls -1 $(${GNULIB_CHECKOUT}/posix-modules | sed -e 's|-posix$||' | sort -u | grep -v 'nonblocking' | sed -e 's|$|.o|') 2>/dev/null )

5. Run 

make check

Known problems on Linux

btowc.o

Observed with Release/2.38, Release/2.39, Release/2.40. 

configure:46783: checking whether btowc is consistent with mbrtowc in the C locale ... no

From documentation: In the C or POSIX locales, this function is not consistent with Gnulib’s mbrtowc and can return WEOF: glibc 2.35, MirOS BSD #10. Thus, expected.

fclose.o

Observed with Release/2.40. 

checking whether fflush works on input streams... no

This is due to

This applies to fclose.o, fflush.o, fseek.o, fseeko.o

fcntl.o

Observed with Release/2.40.

This is due to 

checking whether fcntl understands F_DUPFD_CLOEXEC... needs runtime check

The Linux kernel only added F_DUPFD_CLOEXEC in 2.6.24, so Gnulib always replace it to support the semantics on older kernels that failed with EINVAL. Glibc should do the same.

fflush.o

Observed with Release/2.40.

See fclose.o

fopen.o

Observed with glibc 2.38 and Release/2.40. Needs analysis.

fseek.o

See fclose.o

fseeko.o

See fclose.o

getpayload.o

Observed in Release/2.40. See getpayloadf ...

getpayloadf.o

Observed in Release/2.40. Needs analysis.

From config.log: 

configure:72327: checking whether getpayloadf can be used with libm
configure:72359: gcc -o conftest -g -O2 -Wall  conftest.c  -lm >&5
configure:72359: $? = 0
configure:72370: result: yes
configure:72385: checking whether getpayloadf works
configure:72437: gcc -o conftest -g -O2 -Wall  conftest.c  >&5
/usr/lib/gcc/x86_64-pc-linux-gnu/12/../../../../x86_64-pc-linux-gnu/bin/ld: /tmp/ccaNe9gy.o: in function `main':
/tmp/testdir/conftest.c:572:(.text.startup+0x26): undefined reference to `getpayloadf'
collect2: error: ld returned 1 exit status
configure:72437: $? = 1
configure: program exited with status 1

This looks like a rather stupid new gnulib bug, not a glibc regression.

getpayloadl.o

Observed in Release/2.40. See getpayloadf

ioctl.o

checking for ioctl with POSIX signature... no

Glibc has a wrong parameter type: 'unsigned long' rather than 'int'.

math.o

Needs analysis.

mbrlen.o

This is a known bug:

This affects mbrtowc.o, mbrlen.o, c32rtomb.o, mbrtoc32.o

mbrtoc16.o

mbrtoc32.o

See mbrlen.o

mbrtowc.o

See mbrlen.o

mbsnrtowcs.o

Observed with glibc 2.38 and Release/2.40. Needs analysis.

mbsrtowcs.o

Observed with glibc 2.38 and Release/2.40. Needs analysis.

mbstowcs.o

Seen with glibc 2.38. Needs analysis.

memset_explicit.o

Observed with glibc 2.38 and Release/2.40.

From documentation: This function is missing in glibc. Thus, expected.

mktime.o

Observed with glibc 2.38 and Release/2.40. May be related to (?) 

configure:81341: checking for __mktime_internal
configure:81341: gcc -o conftest -g -O2 -Wall  conftest.c  >&5
/usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../x86_64-pc-linux-gnu/bin/ld: /tmp/ccrjo63V.o: in function `main':
/home/dilfridge/gnulib_testdir/conftest.c:694: undefined reference to `__mktime_internal'
collect2: error: ld returned 1 exit status

Needs analysis.

nanosleep.o

Observed with Release/2.40.

This is due to 

checking for working nanosleep... no (mishandles large arguments)

This function mishandles large arguments when interrupted by a signal on 64-bit Linux platforms. Gnulib has a workaround, Glibc does not.

strerror_r.o

Observed with Release/2.40.

This is due to 

checking for strerror_r with POSIX signature... no

When _GNU_SOURCE is defined, glibc defines strerror_r with a non-POSIX declaration.

sys_socket.o

Observed with Release/2.40. Needs analysis.

time.o

Seen with glibc 2.38 and Release/2.40.

From documentation: not consistent with gettimeofday and timespec_get on some platforms: glibc 2.31 or newer on Linux

unistd.o

Needs analysis

wctype-h.o

Observed with Release/2.40. Needs analysis.

Apparently fixed on Linux as of glibc 2.40

dprintf.o

Observed in Release/2.39. Fixed / gone in Release/2.40. 

checking whether printf supports infinite 'long double' arguments... no

glibc's fprintf produces random output when 'long double' values outside the IEEE range are given. Gnulib developers don't find this is OK and prefer "nan" output.

Related:

This applies to dprintf.o, fprintf.o, printf.o, snprintf.o, sprintf.o, vdprintf.o, vfprintf.o, vprintf.o, vsnprintf.o, vsprintf.o

fprintf.o

See dprintf.o. Fixed / gone in Release/2.40.

printf.o

See dprintf.o

snprintf.o

See dprintf.o

sprintf.o

See dprintf.o

vdprintf.o

See dprintf.o

vfprintf.o

See dprintf.o

vprintf.o

See dprintf.o

vsnprintf.o

See dprintf.o

vsprintf.o

See dprintf.o

wcsstr.o

Seen with glibc 2.38. Fixed / gone with Release/2.40.

From documentation: This function has quadratic instead of linear worst-case complexity on some platforms: glibc 2.37

Apparently fixed on Linux as of glibc 2.38

futimens.o

(fixed) This is due to 

checking whether futimens works... needs runtime check

glob.o

(fixed) This is due to 

checking whether glob lists broken symlinks... no

isfinite.o

(fixed) This is due to 

checking whether isfinite(long double) works... no

linkat.o

(fixed) This is due to 

checking whether linkat(,AT_SYMLINK_FOLLOW) works... need runtime check

regex.o

(fixed) This is due to a bug in a gnulib test case that misses escaping regular expressions in the *.m4 tests.

utimensat.o

(fixed) This is due to 

checking whether utimensat works... needs runtime check

Known problems on Hurd

TODO

Investigating new problems

When an expected .o file is built, or when a check fails, look at

If your conclusion is that it's a bug in Gnulib, please report it at bug-gnulib at gnu dot org.

None: Testing/Gnulib (last edited 2024-07-20 10:24:07 by AndreasHüttel)