This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Testsuite races



As HJ mentioned back in Sep 2011 [1], tests in the intl subdirectory can intermittently fail because several try to update the same file (it's more than just tst-codeset/tst-gettext3) which is then read by the actual code we're testing. If the update happens at just the right time the test obviously fails.


Needless to say this is quite annoying, particularly for folks that are new to glibc and thus not familiar with the intermittent failures and for anything which does automated comparisons of test runs.

It seems to me there's a few ways to deal with this.

  1. Disable -j.  Obviously this is undesirable, even when just
  applied to a particular subdir.

  2. Do some file locking in the setup scripts.  flock is rather
  gross, but it is effective and its use is fairly widespread.

  3. Introduce dependencies in the *.out targets so that they
  execute serially if there's a file dependency in their setup scripts

  4. Do the setup once for all the tests in a new make target.  Have
  each .out depend on that new make target.  Not sure if this is
  feasible or not.

I've done #2 & #3 here for testing purposes with success. I really don't care what solution is selected, just that we pick one and eliminate the intermittent failures that waste people's time.

Thoughts?

Jeff

[1] http://cygwin.com/ml/libc-alpha/2011-09/msg00088.html


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]