This is the mail archive of the
mailing list for the glibc project.
Re: How to handle long running tests in nptl.
- From: Carlos O'Donell <carlos at redhat dot com>
- To: Stefan Liebler <stli at linux dot ibm dot com>, GNU C Library <libc-alpha at sourceware dot org>
- Date: Thu, 12 Sep 2019 10:02:15 -0400
- Subject: Re: How to handle long running tests in nptl.
- References: <firstname.lastname@example.org>
On 9/12/19 9:21 AM, Stefan Liebler wrote:
> the nptl tests are currently running in sequence. Some of them are running very long:
> -tst-cond16: 20s
> -tst-cond17: 20s
> -tst-cond18: 20s
> -tst-rwlock-tryrdlock-stall: 20s
> -tst-mutex10: 16s
> -tst-rwlock20: 10s
> -tst-rwlock-trywrlock-stall: 10s
> -tst-rwlock-pwn: 10s
> The listed tests are responsible for over two minutes of runtime of "make check". They all are running a test in a loop for a large amount of iterations or seconds in order to trigger e.g. a race condition.
> How to handle those long running tests with respect of "make check" runtime?
> - reduce runtime by reducing number of iterations or seconds
> - move those tests to "make xcheck"
> - reduce runtime while running "make check" and rerun with unchanged runtime in "make xcheck"
> - change nothing
> - other ideas?
The use of 'make xcheck' is for tests that need specific persmissions
to run, like root.
This issue has come up already with regards to the test-in-container
tests because it takes time to do installed-tree testing because you
have to update the installed tree.
I would like to see a discussion around:
- What do developers want from day-to-day 'make check?'
- Do you want 'make check' to take a constant amount of time?
- Do you want 'make check' to give you good coverage?
- What do developers want to run before checkin?
- Is "before checkin" different from day-to-day testing?
This issue is a topic for GNU Cauldron 2019:
Update the glibc build infrastructure
Discuss getting accurate dependency information into the build system so we can do incremental build and test in as fast a time as possible.
Breaking up tests? Fast. Short. Fixed time.