Re: How to handle long running tests in nptl.

[Carlos O'Donell <carlos at redhat dot com>]

On 9/12/19 9:21 AM, Stefan Liebler wrote:
> Hi,
> 
> the nptl tests are currently running in sequence. Some of them are running very long:
> -tst-cond16: 20s
> -tst-cond17: 20s
> -tst-cond18: 20s
> -tst-rwlock-tryrdlock-stall: 20s
> -tst-mutex10: 16s
> -tst-rwlock20: 10s
> -tst-rwlock-trywrlock-stall: 10s
> -tst-rwlock-pwn: 10s
> 
> The listed tests are responsible for over two minutes of runtime of "make check". They all are running a test in a loop for a large amount of iterations or seconds in order to trigger e.g. a race condition.
> 
> How to handle those long running tests with respect of "make check" runtime?
> - reduce runtime by reducing number of iterations or seconds
> - move those tests to "make xcheck"
> - reduce runtime while running "make check" and rerun with unchanged runtime in "make xcheck"
> - change nothing
> - other ideas?

The use of 'make xcheck' is for tests that need specific persmissions
to run, like root.

This issue has come up already with regards to the test-in-container
tests because it takes time to do installed-tree testing because you
have to update the installed tree.

I would like to see a discussion around:

- What do developers want from day-to-day 'make check?'
  - Do you want 'make check' to take a constant amount of time?
  - Do you want 'make check' to give you good coverage?

- What do developers want to run before checkin?
  - Is "before checkin" different from day-to-day testing?

This issue is a topic for GNU Cauldron 2019:
https://sourceware.org/glibc/wiki/Cauldron2019
~~~
Update the glibc build infrastructure

    Discuss getting accurate dependency information into the build system so we can do incremental build and test in as fast a time as possible.
    Breaking up tests? Fast. Short. Fixed time. 
~~~

-- 
Cheers,
Carlos.