RFC: make fastcheck
Carlos O'Donell
carlos@redhat.com
Thu Jul 18 11:01:00 GMT 2019
On 7/18/19 4:13 AM, Florian Weimer wrote:
> * Carlos O'Donell:
>
>> Florian,
>>
>> What do you think of making a 'fastcheck' that can run all tests which have
>> timeouts less than or equal to the default timeout?
>>
>> My idea here is as follows:
>>
>> - 'make fastcheck' is for developer smoke testing during edit/build/test cycle.
>> - 'make check' if for broader developer testing, and will, if required, update
>> the chroot for container testing (would have fixed Rafal's observed breakage).
>> - 'make xcheck' runs everything including the expensive tests and tests requiring
>> explicit permissions.
>>
>> Then you could switch your workflow to using 'fastcheck'?
>>
>> This is one part of fixing the workflow.
>>
>> The other part is actually fixing 'make' to go faster, but DJ and I talked about
>> this recently and DJ has some ideas.
>
> Build time on my test machine:
>
> real 1m40.633s
> user 8m19.845s
> sys 2m6.309s
>
> The test time is:
>
> real 14m5.543s
> user 29m29.308s
> sys 7m27.912s
>
>
> Now apply this patch:
>
> diff --git a/support/support_test_main.c b/support/support_test_main.c
> index 7e7b9edbb0..f196990a3f 100644
> --- a/support/support_test_main.c
> +++ b/support/support_test_main.c
> @@ -209,6 +209,9 @@ adjust_exit_status (int status)
> int
> support_test_main (int argc, char **argv, const struct test_config *config)
> {
> + if (config->timeout > DEFAULT_TIMEOUT)
> + return 0;
> +
> if (test_main_called)
> {
> printf ("error: test_main called for a second time\n");
>
> The new testing time is:
>
> real 10m35.001s
> user 25m55.020s
> sys 5m21.004s
>
> So there is a definite improvement. But I don't think it doesn't solve
> the core problem because it's still too long.
Sorry, I failed to list a critical part of my own reasoning.
I intended to write that the test-container tests would *not* run in
make fastcheck.
How much does that improve the timing by?
> It also doesn't help with build-many-glibcs.py because that doesn't run
> any tests which could time out.
That's the second part of this problem. I agree it's a problem.
However, testing, even if we fix the build infrastructure, will always
grow in duration, and we need a way to handle this with a "profile" which
allows developers to smoke test their build.
--
Cheers,
Carlos.
More information about the Libc-alpha
mailing list