[PATCH 0/5] Improve make and make check parallelism

[Adhemerval Zanella Netto]

On 15/06/26 11:46, Florian Weimer wrote:
> * Adhemerval Zanella Netto:
> 
>> On 15/06/26 11:15, Florian Weimer wrote:
>>> * Adhemerval Zanella:
>>>
>>>> The series does not aim to improve 'make check' with
>>>> run-built-tests=yes: subdirectories such as nptl require their tests
>>>> not to run in parallel.
>>>
>>> Based on my logs, it seems to me that nptl tests now run in parallel
>>> with other directories.  Do others see this as well?
>>>
>>> This somewhat defeats the .NOTPARALLEL directive for nptl (and rt).
>>
>> Yes, but nptl sub-make reads nptl/Makefile, hits .NOTPARALLEL and within
>> that process the nptl tests run one at a time.
>>
>> Making the nptl's tests to additionally not overlap with other subdirs 
>> (restore the old isolation), would need an explicit serialization edge 
>> and that would erase much of the parallel-check speedup.
> 
> Only a subset of the nptl tests would need this anyway.  And we have
> other tests that are CPU burners or that need tons of RAM that should
> not be parallelized, either.
> 
> I think we could have a top-level .NOTPARALLEL wrapper that tests
> everything except nptl/htl and rt (without applying .NOTPARALLEL), then
> nptl/htl, and then rt.  Not sure if there's a simpler approach to get
> the desired semantics.
I am not really fond of these kind of tests, we might get false positive
depending of system loading.  But I also agree that adding this parallel
run does not improve this.

The build of the tests with this approach is highly parallel, so I think we
can split the check in two phases: build (run-built-tests=no) and run
(run-built-tests=yes).  This incurs in two make transversal, but I think 
this is not a performance bottleneck anymore and this will make keeping 
current .NOTPARALLEL semantics easier to define in the Makefile.

I will send a newer version, I found an issue when make subfolder/tests
is used (it now runs all tests instead only the one in the subfolder).