This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
what to do with flaky tests?
- From: Szabolcs Nagy <Szabolcs dot Nagy at arm dot com>
- To: GNU C Library <libc-alpha at sourceware dot org>
- Cc: nd <nd at arm dot com>
- Date: Fri, 3 May 2019 10:06:06 +0000
- Subject: what to do with flaky tests?
currently these tests are flaky on the aarch64 buildbot:
FAIL: malloc/tst-mallocfork2
FAIL: nptl/tst-eintr1
the first one is a timeout: the test can take unbounded time
since there is no guarantee that malloc ever gets interrupted
by the signal (so increasing the timeout limit does not help:
sometimes it finishes immediately sometimes it take several
minutes).
the second one can break make check: kernel side task
accounting is broken and even though no more than 100 threads
are alive at a time the cgroup task limit can be hit (even
if that's >20000 tasks) and then all fork/clone fails so
make check cannot continue and test results are lost.
the server where the buildbot runs is fairly loaded which
contributes to this issue, but that's not under my control
and even on non-loaded system these tests can fail, so
i wonder if it's acceptable to disable flaky tests in some
way (env var?) and then run the buildbots with that.