This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: How to handle long running tests in nptl.
- From: Florian Weimer <fweimer at redhat dot com>
- To: Carlos O'Donell <carlos at redhat dot com>
- Cc: Stefan Liebler <stli at linux dot ibm dot com>, GNU C Library <libc-alpha at sourceware dot org>
- Date: Fri, 13 Dec 2019 12:15:27 +0100
- Subject: Re: How to handle long running tests in nptl.
- References: <0ba7efb0-9e71-1764-bce8-686c8f452285@linux.ibm.com> <b4d07243-4042-d0de-58da-247dc341db99@redhat.com>
* Carlos O'Donell:
> On 9/12/19 9:21 AM, Stefan Liebler wrote:
>> Hi,
>>
>> the nptl tests are currently running in sequence. Some of them are running very long:
>> -tst-cond16: 20s
>> -tst-cond17: 20s
>> -tst-cond18: 20s
>> -tst-rwlock-tryrdlock-stall: 20s
>> -tst-mutex10: 16s
>> -tst-rwlock20: 10s
>> -tst-rwlock-trywrlock-stall: 10s
>> -tst-rwlock-pwn: 10s
>>
>> The listed tests are responsible for over two minutes of runtime of "make check". They all are running a test in a loop for a large amount of iterations or seconds in order to trigger e.g. a race condition.
>>
>> How to handle those long running tests with respect of "make check" runtime?
>> - reduce runtime by reducing number of iterations or seconds
>> - move those tests to "make xcheck"
>> - reduce runtime while running "make check" and rerun with unchanged runtime in "make xcheck"
>> - change nothing
>> - other ideas?
>
> The use of 'make xcheck' is for tests that need specific persmissions
> to run, like root.
It's also used for tests that require many minutes to run or require
special firewall settings, e.g., resolv/tst-resolv-qtypes.
Unfortunately, we do not capture data of legitimate test failures, so
it's hard to tell how valuable such tests are.
Many of the timeout-heavy nptl tests do not actually need a quiet
system, though, so they could be moved to a separate subdirectory that
contains only such tests. Or we could add some markup to the tests and
add a more intelligent test scheduler.
Thanks,
Florian