Re: better tests for feature test macro permutations?

["Joseph S. Myers" <joseph at codesourcery dot com>]

On Mon, 27 Feb 2012, Roland McGrath wrote:

> between.  So it seems likely we need some tests that attempt including
> various headers under various permutations of feature test macros to ensure
> that they all compile.

The way I think of this is as a test of installed headers; you want the 
headers the user will see to work, without the glibc-internal include/ 
headers being involved, and you want to test all the headers glibc will 
install (which is built up from variables in different directories, so I 
don't think the full list is particularly conveniently available in the 
present makefile system).

I think this also relates to the sys/timerfd.h issue we discussed in 
October.  I agree with 
<http://sourceware.org/ml/libc-alpha/2011-10/msg00050.html> that any 
header #include, with any valid feature test macros (combined with a 
standards mode in the compiler, e.g. -std=gnu99 or -std=c89), should 
either compile cleanly or give a #error.  Separately from that rule for 
*all* installed headers (including the bits/*.h headers, all of which 
#error if you include them directly), each public header defining some 
public API has some subset of configurations that make sense for it, and 
all those configurations should work and not #error.

Testing particular headers with particular feature test macros known to be 
good for those headers is probably easier at present than testing the full 
set of installed headers (for working or #error) - somewhere you'd have a 
list of the valid combinations of header and feature test macros.  You'd 
still get the internal include/ headers used, though.

I take the view that really most of glibc's testsuite ought to work by 
first installing the library in a sysroot directory, then building with 
options that reference that sysroot (unfortunately I'm not sure 
non-sysrooted compilers and linkers can simply use --sysroot= for this, 
which would be the ideal way to do it; some aspects of --sysroot= might 
not work if the compiler/linker wasn't originally configured to use a 
sysroot), rather than long complicated command lines that have little 
resemblance to how people actually use the library.  Only limited parts of 
the testsuite have any real dependence on noninstalled headers or other 
files produced by the glibc build process but not installed; it ought to 
be possible to run most of it on an installed library without needing most 
of the build directory from when that library was built.  But while this 
sort of thing would certainly make it easier to do tests of all installed 
headers, it's also a fairly major project.

-- 
Joseph S. Myers
joseph@codesourcery.com