This is the mail archive of the mailing list for the glibc project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

[PATCH] Add compile testing to glibc test framework.

The goal of this patch is to add compile testing to the glibc
test framework.

The changes allow you to add C source files that the test framework
can compile and verify the compiler exits with a zero exit status for
PASS. While a non-zero exit status by the compiler indicates a FAIL.
The support for UNSUPPORTED is handled in the normal way for other
tests that are conditionally disabled (can't be run and thus can't
exit with code 77). Only C source files are currently supported
because that's all I needed to support today, but it could be extended
to compile any type of source target. I wanted to approach the work

The test is being compiled under the glibc test framework, which
is as close as we currently get to building real user applications.

For example I use the test to compile various glibc/linux header
order combinations e.g.

cat sysdeps/unix/sysv/linux/tst-ipv6-compat1.c
#include <netinet/in.h>
#include <linux/in6.h>

main (void)
  return 0;

git diff sysdeps/unix/sysv/linux/Makefile
diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile
index c57575f..dad556c 100644
--- a/sysdeps/unix/sysv/linux/Makefile
+++ b/sysdeps/unix/sysv/linux/Makefile
@@ -203,3 +203,9 @@ tests += tst-setgetname tst-align-clone tst-getpid1 \
        tst-thread-affinity-pthread tst-thread-affinity-pthread2 \
+ifeq ($(subdir),inet)
+# Add tests for Linux and glibc header compatibility.
+tests-compile += tst-ipv6-compat1 tst-ipv6-compat2 tst-ipv6-compat3
+TEST_SRC_DIR = ../sysdeps/unix/sysv/linux/

Previously these kinds of tests would result in the testsuite failing
to run to completion because compiler errors are treated as harness

I tried a variety of solutions to implement this, but settled on the
present set of two variables: tests-compile and TESTS_SRC_DIR as a way
to avoid needing anything like sysd-rules to find the test sources.
I did not want to unduly slow down build test cycles. The static
pattern rule is pretty concise and optimal.



2015-06-06  Carlos O'Donell  <>

	* Rules [ifeq ($(run-built-tests),no)] (tests): Add $(tests-compile).
	[ifneq ($(run-built-tests),no)] (tests): Add $(tests-compile:%=$(objpfx)%.out).
	(tests): Add $(tests-compile) to merge result list.
	[ifeq ($(build-programs),yes)] (binaries-all-tests): Add $(tests-compile).
	[ifneq "$(strip $(tests-compile))" ""] (tc-OUTPUT_OPTION-file): Define.
	(tc-OUTPUT_OPTION): Define.
	(tc-compile-mkdeps-flags): Define.
	(tc-compile.c): Define.
	($(tests-compile:%=$(objpfx)%.out)): Define static pattern rule.

diff --git a/Rules b/Rules
index 8306d36..e92a9ff 100644
--- a/Rules
+++ b/Rules
@@ -91,10 +91,10 @@ else
 others: $(addprefix $(objpfx),$(extra-objs))
 ifeq ($(run-built-tests),no)
-tests: $(addprefix $(objpfx),$(tests) $(test-srcs)) $(tests-special)
+tests: $(addprefix $(objpfx),$(tests) $(test-srcs)) $(tests-compile) $(tests-special)
 xtests: tests $(xtests-special)
-tests: $(tests:%=$(objpfx)%.out) $(tests-special)
+tests: $(tests:%=$(objpfx)%.out) $(tests-compile:%=$(objpfx)%.out) $(tests-special)
 xtests: tests $(xtests:%=$(objpfx)%.out) $(xtests-special)
@@ -102,7 +102,7 @@ tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special))
 xtests-special-notdir = $(patsubst $(objpfx)%, %, $(xtests-special))
 	$(..)scripts/ -s $(objpfx) $(subdir) \
-	  $(sort $(tests) $(tests-special-notdir:.out=)) \
+	  $(sort $(tests) $(tests-special-notdir:.out=) $(tests-compile)) \
 	  > $(objpfx)subdir-tests.sum
 	$(..)scripts/ -s $(objpfx) $(subdir) \
@@ -111,7 +111,7 @@ xtests:
 ifeq ($(build-programs),yes)
 binaries-all-notests = $(others) $(sysdep-others)
-binaries-all-tests = $(tests) $(xtests) $(test-srcs)
+binaries-all-tests = $(tests) $(xtests) $(test-srcs) $(tests-compile)
 binaries-all = $(binaries-all-notests) $(binaries-all-tests)
 binaries-static-notests = $(others-static)
 binaries-static-tests = $(tests-static) $(xtests-static)
@@ -198,6 +198,45 @@ $(objpfx)%.out: /dev/null $(objpfx)%	# Make it 2nd arg for canned sequence.
 	$(make-test-out) > $@; \
+# tests-compile provides a way for developers to test compiling a singular
+# C source file under the environment of the glibc build.  This is useful
+# when testing system invariants like compatibility between headers files
+# provided by the linux kernel headers (--with-headers).  While it is
+# arguable that such tests, as invariants, should go in configure scripts,
+# that would lead to a large number of unruly tests in
+# Having the compatibility tests as part of the regression testsuite is far
+# more manageable from a maintenance perspective.
+# There are three things you need to do to add a compilation test to glibc:
+# * Create the source file to compile.
+#   - It should compile without error in order for the test to PASS.
+#   - It should compile with error in order for the test to FAIL.
+# * Add the test name to 'tests-compile'
+#   - e.g.
+#     ifeq ($(subdir),inet)
+#     tests-compile += tst-ipv6-compat
+#     ...
+# * Set 'TEST_SRC_DIR' to the relative path to the test sources (no trailing
+#   slash):
+#     ...
+#     TEST_SRC_DIR = ../sysdeps/unix/sysv/linux
+#     endif
+# You will now have an additional test that is compiled during 'make check',
+# with compiler output to $(objpfx)/<test name>.out as you would expect for
+# a normal test.
+ifneq "$(strip $(tests-compile))" ""
+tc-OUTPUT_OPTION-file = $(objpfx)$(notdir $(@:%.out=%.o))
+tc-compile-mkdep-flags = -MD -MP -MF $(tc-OUTPUT_OPTION-file).dt -MT $(tc-OUTPUT_OPTION-file)
+tc-compile.c = $(CC) $(TEST_SRC_DIR)/$(notdir $(@:%.out=%.c)) -c $(CFLAGS) $(CPPFLAGS)
+$(tests-compile:%=$(objpfx)%.out): $(objpfx)%.out : $(TEST_SRC_DIR)/%.c Makefile
+	$(tc-compile.c) $(tc-OUTPUT_OPTION) $(tc-compile-mkdep-flags) \
+	>& $@; $(evaluate-test)
 # tests-unsupported lists tests that we will not try to build at all in
 # this configuration.  Note this runs every time because it does not
 # actually create its target.  The dependency on Makefile is meant to

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]