libctf and libsframe testing

[Nick Alcock]

On 8 Aug 2025, Jan Beulich uttered the following:

> On 08.08.2025 09:45, Indu Bhagat wrote:
>> On 8/7/25 8:04 AM, Nick Alcock wrote:
>>> On 7 Aug 2025, Indu Bhagat outgrape:
>>>> On 8/5/25 12:27 PM, Nick Alcock wrote:
>>>>> On 5 Aug 2025, Jan Beulich stated:
>>>>>
>>>>>> On 05.08.2025 13:43, Nick Alcock wrote:
>>>>>>> On 1 Aug 2025, Jan Beulich stated:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> just came to notice that the testsuite runs are all building binaries for the
>>>>>>>> host, not the target.
>>>>>>>
>>>>>>> Well... the libctf testsuite should be building binaries for both,
>>>>>>> actually. libctf itself is a host library, with no target-dependent
>>>>>>> components (it leans on bfd for those), so we do want to test it on the
>>>>>>> host: that's where it'sll run.
>>>>>>>
>>>>>>> ... *however*. The binaries the testsuite builds fall into two
>>>>>>> categories:
>>>>>>>
>>>>>>>    - "lookup binaries", which are executed on the host: some just do their
>>>>>>>      own thing using libctf API calls, but others (e.g. in libctf-lookup/)
>>>>>>>      are used to inspect...
>>>>>>>    - ... object files and linked binaries which contain CTF. These are
>>>>>>>      built with the target compiler, or should be.
>>>>>>
>>>>>> Provided there is one, that is. I surely don't have some 150 cross compilers
>>>>>> installed.
>>>>> Yeah, this is skipped if the target compiler is not available.
>>>>>
>>>>>>> The compilation of both these binaries happens in
>>>>>>> libctf/testsuite/lib/ctf-lib.exp:run_lookup_test(), if you're
>>>>>>> interested. They are driven by *.lk files: any *.lk files which do not
>>>>>>> specify a source: are building lookup binaries only, on the host.
>>>>>>>
>>>>>>>>                          While this may be sensible when testing an individual
>>>>>>>> target, (afaict) it's doing all the same over and over when doing bulk tests.
>>>>>>>> I couldn't spot any means (e.g. an env var) to override this behavior, i.e.
>>>>>>>> optionally suppress such redundant testing. If there is one, could you please
>>>>>>>> point me in the right direction? If there is none, can something be added?
>>>>>>>
>>>>>>> ... sure: we could at least not rerun those tests that don't use source:
>>>>>>> at all. I wonder what the least horrible approach would be... maybe a
>>>>>>> configure flag to ask run_lookup_test to quietly do nothing if host !=
>>>>>>> target and source: is not in use?
>>>>>>
>>>>>> Well, that's precisely what I don't know, or else I might have preferred to
>>>>>> post a patch.
>>>>> I guess a configure flag to libctf/configure.ac might do it. I'll see if
>>>>> I can whip one up this week, before the summer holiday.
>>>>
>>>> As for libsframe, which is also a host library, all tests so far need to be run for the host.
>>>>
>>>> Skipping the libsframe testsuite for host != target when doing bulk tests is fine.  I too think that it makes sense to add a
>>>> configure option (--disable-lisbframe-tests?) so they are skipped only when the usecase is doing bulk tests (and host != target).
>>>
>>> If multiple libs need this sort of thing, maybe we want a
>>> --disable-host-tests option which disables such tests for both of them
>>> when host != target?
>> 
>> Thats a better option name, but still lacks the context that the option 
>> disables the tests only when cross builds are detected.
>> 
>> Perhaps --disable-host-tests-cross-build ? but then apart from libctf 
>> and libsframe, are there other tests that could be disabled and are 
>> expected to be tied to --disable-host-tests-cross-build ?
>
> Do we need to restrict this to cross builds? People just shouldn't use
> this option when they care about the testing being done.

I thought the point of the option was to avoid likely (but not always)
redundant test runs when doing multiple crosses. When the build is not a
cross build, you're probably interested in the output of host tests, so
running these tests is not redundant.

(I think a better option name might be --disable-cross-host-tests though.)

-- 
NULL && (void)