libctf and libsframe testing

[Indu Bhagat]

On 8/7/25 8:04 AM, Nick Alcock wrote:
> On 7 Aug 2025, Indu Bhagat outgrape:
>> On 8/5/25 12:27 PM, Nick Alcock wrote:
>>> On 5 Aug 2025, Jan Beulich stated:
>>>
>>>> On 05.08.2025 13:43, Nick Alcock wrote:
>>>>> On 1 Aug 2025, Jan Beulich stated:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> just came to notice that the testsuite runs are all building binaries for the
>>>>>> host, not the target.
>>>>>
>>>>> Well... the libctf testsuite should be building binaries for both,
>>>>> actually. libctf itself is a host library, with no target-dependent
>>>>> components (it leans on bfd for those), so we do want to test it on the
>>>>> host: that's where it'sll run.
>>>>>
>>>>> ... *however*. The binaries the testsuite builds fall into two
>>>>> categories:
>>>>>
>>>>>    - "lookup binaries", which are executed on the host: some just do their
>>>>>      own thing using libctf API calls, but others (e.g. in libctf-lookup/)
>>>>>      are used to inspect...
>>>>>    - ... object files and linked binaries which contain CTF. These are
>>>>>      built with the target compiler, or should be.
>>>>
>>>> Provided there is one, that is. I surely don't have some 150 cross compilers
>>>> installed.
>>> Yeah, this is skipped if the target compiler is not available.
>>>
>>>>> The compilation of both these binaries happens in
>>>>> libctf/testsuite/lib/ctf-lib.exp:run_lookup_test(), if you're
>>>>> interested. They are driven by *.lk files: any *.lk files which do not
>>>>> specify a source: are building lookup binaries only, on the host.
>>>>>
>>>>>>                          While this may be sensible when testing an individual
>>>>>> target, (afaict) it's doing all the same over and over when doing bulk tests.
>>>>>> I couldn't spot any means (e.g. an env var) to override this behavior, i.e.
>>>>>> optionally suppress such redundant testing. If there is one, could you please
>>>>>> point me in the right direction? If there is none, can something be added?
>>>>>
>>>>> ... sure: we could at least not rerun those tests that don't use source:
>>>>> at all. I wonder what the least horrible approach would be... maybe a
>>>>> configure flag to ask run_lookup_test to quietly do nothing if host !=
>>>>> target and source: is not in use?
>>>>
>>>> Well, that's precisely what I don't know, or else I might have preferred to
>>>> post a patch.
>>> I guess a configure flag to libctf/configure.ac might do it. I'll see if
>>> I can whip one up this week, before the summer holiday.
>>
>> As for libsframe, which is also a host library, all tests so far need to be run for the host.
>>
>> Skipping the libsframe testsuite for host != target when doing bulk tests is fine.  I too think that it makes sense to add a
>> configure option (--disable-lisbframe-tests?) so they are skipped only when the usecase is doing bulk tests (and host != target).
> 
> If multiple libs need this sort of thing, maybe we want a
> --disable-host-tests option which disables such tests for both of them
> when host != target?
> 

Thats a better option name, but still lacks the context that the option 
disables the tests only when cross builds are detected.

Perhaps --disable-host-tests-cross-build ? but then apart from libctf 
and libsframe, are there other tests that could be disabled and are 
expected to be tied to --disable-host-tests-cross-build ?