RFC: make fastcheck

Carlos O'Donell carlos@redhat.com
Thu Jul 18 11:01:00 GMT 2019


On 7/18/19 4:13 AM, Florian Weimer wrote:
> * Carlos O'Donell:
> 
>> Florian,
>>
>> What do you think of making a 'fastcheck' that can run all tests which have
>> timeouts less than or equal to the default timeout?
>>
>> My idea here is as follows:
>>
>> - 'make fastcheck' is for developer smoke testing during edit/build/test cycle.
>> - 'make check' if for broader developer testing, and will, if required, update
>>    the chroot for container testing (would have fixed Rafal's observed breakage).
>> - 'make xcheck' runs everything including the expensive tests and tests requiring
>>    explicit permissions.
>>
>> Then you could switch your workflow to using 'fastcheck'?
>>
>> This is one part of fixing the workflow.
>>
>> The other part is actually fixing 'make' to go faster, but DJ and I talked about
>> this recently and DJ has some ideas.
> 
> Build time on my test machine:
> 
> real    1m40.633s
> user    8m19.845s
> sys     2m6.309s
> 
> The test time is:
> 
> real    14m5.543s
> user    29m29.308s
> sys     7m27.912s
> 
> 
> Now apply this patch:
> 
> diff --git a/support/support_test_main.c b/support/support_test_main.c
> index 7e7b9edbb0..f196990a3f 100644
> --- a/support/support_test_main.c
> +++ b/support/support_test_main.c
> @@ -209,6 +209,9 @@ adjust_exit_status (int status)
>   int
>   support_test_main (int argc, char **argv, const struct test_config *config)
>   {
> +  if (config->timeout > DEFAULT_TIMEOUT)
> +    return 0;
> +
>     if (test_main_called)
>       {
>         printf ("error: test_main called for a second time\n");
> 
> The new testing time is:
> 
> real    10m35.001s
> user    25m55.020s
> sys     5m21.004s
> 
> So there is a definite improvement.  But I don't think it doesn't solve
> the core problem because it's still too long.

Sorry, I failed to list a critical part of my own reasoning.

I intended to write that the test-container tests would *not* run in
make fastcheck.

How much does that improve the timing by?

> It also doesn't help with build-many-glibcs.py because that doesn't run
> any tests which could time out.

That's the second part of this problem. I agree it's a problem.

However, testing, even if we fix the build infrastructure, will always
grow in duration, and we need a way to handle this with a "profile" which
allows developers to smoke test their build.

-- 
Cheers,
Carlos.



More information about the Libc-alpha mailing list