libctf testing
This page documents various users' standard tests they run before pushing libctf changes, in the hope that others might be able to reuse it. (Feel free to add a new section to document what you do.)
NickAlcock's test matrix
My routine test matrix is largely but not entirely scripted. The scripts mostly make use of local tooling so are absolutely not going to work without a bit of light editing (even if no more than fixing paths, using your own containerizing tools rather than olcontainerize, and using mkdir rather than my strange mktransient mkdir-on-fast-storage tool. I run the following in sequence:
First, the check-binutils script. This uses the pueue queueing tool to run a bunch of configure/make/make check runs in parallel: currently this includes shared, static, enable-targets=all, no-compiler-available, 32-bit, clang sanitizer, and a couple of containerized runs using cross-compilers to PPC and sparc64 (mostly chosen for being big-endian). Without --quick it also runs a couple of valgrinded tests.
These tests are not expected to 100% pass: I do a pueue log afterwards, grep it for 'libctf', and check for signs of compilation warnings, spurious failures etc etc.
Second, if that passes (or sometimes in parallel), I run binutils-intermediate. This is rather a lot less optimized and serial right now, and does rebase-based configure-and-make tests of every commit in the series I'm pushing, in sequence. I don't do make checks: the point is to make sure that all intermediate commits at least compile, to help bisection.
Third, if I've touched things that might affect the linker or BFD, I run a script which does configure/test runs in parallel covering a large subset of all binutils targets (not entirely my work, so not public here yet).
Manual steps
Finally, I run various configure runs on a variety of not-yet-automated platforms. FreeBSD 14:
../configure --enable-shared --disable-static --disable-gdb --disable-gprofng --disable-werror --prefix=/usr/local/ctf LDFLAGS=-Wl,-rpath=/usr/local/ctf/lib CC=/usr/local/ctf/bin/gcc CXX=/usr/local/ctf/bin/c++ CFLAGS="-O2 -g -L/usr/local/lib -L/usr/lib" CXXFLAGS="-L/usr/local/lib -L/usr/lib" && gmake -j 30 --output-sync=line --no-print-directory all-gas all-binutils all-ld && gmake -k -j 30 check-binutils check-ld check-libctf ../configure --disable-shared --disable-werror --disable-gdb --disable-gprofng --prefix=/usr/local/ctf LDFLAGS=-Wl,-rpath=/usr/local/ctf/lib CC=/usr/local/ctf/bin/gcc CXX=/usr/local/ctf/bin/c++ CFLAGS="-O2 -g -L/usr/local/lib -L/usr/lib" CXXFLAGS="-L/usr/local/lib -L/usr/lib" && gmake -j 30 --output-sync=line --no-print-directory all-gas all-binutils all-ld && gmake -k -j 30 check-binutils check-ld check-libctf
A compile-farm sparc64 (gcc102), when that was working:
../configure --prefix=/home/nix/.local --without-python --disable-sim --disable-gprofng --enable-targets=all --enable-shared && make -j 128 --output-sync=line --no-print-directory && make -j 128 -k check-libctf check-binutils check-gas check-ld && LIBCTF_WRITE_FOREIGN_ENDIAN=t make -j 128 -k check-libctf check-ld ../configure --target=x86_64-pc-linux-gnu --without-python --disable-sim --disable-gprofng --enable-shared && make -j 128 --output-sync=line --no-print-directory && make -k check-libctf
A compile-farm PowerPC box, gcc110:
../configure --disable-python --disable-gdb --enable-shared --disable-gprofng --libdir='${exec_prefix}/lib64' && make -j 30 && { make -j 30 check-libctf check-binutils check-gas check-ld; LIBCTF_WRITE_FOREIGN_ENDIAN=t make -j 30 check-libctf check-binutils check-gas check-ld; }
../configure --disable-python --disable-gdb --enable-shared --disable-gprofng --disable-gdbserver --disable-plugin --libdir='${exec_prefix}/lib64' --target=powerpc-unknown-linux-gnu CFLAGS="-O2 -g -m32" && make -j 30 && { make -j 30 check-libctf check-binutils check-gas check-ld; LIBCTF_WRITE_FOREIGN_ENDIAN=t make -j 30 check-libctf check-binutils check-gas check-ld; }A compile-farm RISC-V, cfarm93:
../configure --disable-python --disable-gdb --enable-shared --disable-gprofng && make -j 30 && { make -j 30 check-libctf check-binutils check-gas check-ld; LIBCTF_WRITE_FOREIGN_ENDIAN=t make -j 30 check-libctf check-binutils check-gas check-ld; }A compile-farm Darwin box, gcc104: (limited: non-ELF, so no ld tests)
../configure --prefix=/home/nix/.local --enable-shared --disable-static --enable-targets=all CC=gcc-12 CXX=c++-12 && make -j 6 --output-sync=line --no-print-directory && make -j 6 check-ld check-libctf check-binutils check-gas
A convenient aarch64 box (local to me):
../configure --enable-shared --disable-werror --disable-gprofng && make -j 30 --output-sync=line --no-print-directory && make -j 30 check-binutils check-gas check-ld check-libctf
Cygwin: (limited: non-ELF, so no ld tests)
../configure --disable-werror --enable-shared --enable-static && make -j 8 all-binutils all-ld all-gas && make -k check-ld check-libctf
MinGW64: (limited: non-ELF, so no ld tests)
../configure --enable-shared --prefix=$MINGW_PREFIX/local --disable-werror --disable-sim --disable-gdb --disable-gdbsupport --disable-gdbserver --disable-sim --enable-64-bit-bfd --enable-install-libiberty --without-libiconv-prefix --without-libintl-prefix && make -j 8 && make -j 8 -k check-libsframe check-libctf check-ld check-binutils ../configure --enable-shared --prefix=$MINGW_PREFIX/local --target=arm-none-eabi --disable-werror --disable-sim --disable-gdb --disable-gdbsupport --enable-64-bit-bfd --enable-install-libiberty --without-libiconv-prefix --without-libintl-prefix && make -j 8 && make -j 8 -k check-libctf
DTrace testing
I build a kernel with CTF support (the usual for kernel builds, plus 'make ctf', 'make ctf_install') and run DTrace against it. (This is fairly involved but does test quite a few wrinkles of the generated output, and gives the deduplicator a real workout.)
Build-the-world testing
I have a test Gentoo system with a lot of stuff in it, my latest unpushed binutils as a live ebuild, and -gctf forced into the CFLAGS via an overlay atop eclass/flag-o-matic. After larger libctf changes, I rebuild almost everything by running
time emerge --emptytree --ask --tree --with-bdeps y --complete-graph --exclude "sys-devel/gcc* sys-devel/binutils:2.38 sys-devel/llvm dev-lang/rust sys-devel/autoconf sys-devel/automake sys-devel/libtool virtual/* acct-user/* acct-group/*" @world
overnight (skipping some huge or slow-to-build non-C-based projects), then inspect the resulting output to see if any builds failed or emitted unexpected errors or warnings. (The set of installed software here grows every time I receive a report that something has triggered a libctf bug.)
Cross testing
libctf is unusual in that it emits endian-native output no matter what, and adapts to endianness at read time. There is support (used by the check-binutils script above) for forcing emission in alien endianness, but that too might of course be buggy! So now and then I build some CTF on a big-endian platform and some on a little-endian platform, transfer the CTF over to the other platform, and make sure objdump works. That suffices to test every endian-dependent pathway in the library.
(... more to come ...)