This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Testsuite races
- From: Jeff Law <law at redhat dot com>
- To: libc-alpha <libc-alpha at sourceware dot org>
- Date: Tue, 14 Aug 2012 14:55:05 -0600
- Subject: Testsuite races
As HJ mentioned back in Sep 2011 [1], tests in the intl subdirectory can
intermittently fail because several try to update the same file (it's
more than just tst-codeset/tst-gettext3) which is then read by the
actual code we're testing. If the update happens at just the right time
the test obviously fails.
Needless to say this is quite annoying, particularly for folks that are
new to glibc and thus not familiar with the intermittent failures and
for anything which does automated comparisons of test runs.
It seems to me there's a few ways to deal with this.
1. Disable -j. Obviously this is undesirable, even when just
applied to a particular subdir.
2. Do some file locking in the setup scripts. flock is rather
gross, but it is effective and its use is fairly widespread.
3. Introduce dependencies in the *.out targets so that they
execute serially if there's a file dependency in their setup scripts
4. Do the setup once for all the tests in a new make target. Have
each .out depend on that new make target. Not sure if this is
feasible or not.
I've done #2 & #3 here for testing purposes with success. I really
don't care what solution is selected, just that we pick one and
eliminate the intermittent failures that waste people's time.
Thoughts?
Jeff
[1] http://cygwin.com/ml/libc-alpha/2011-09/msg00088.html