libctf and libsframe testing

[Jan Beulich]

On 08.08.2025 09:45, Indu Bhagat wrote:
> On 8/7/25 8:04 AM, Nick Alcock wrote:
>> On 7 Aug 2025, Indu Bhagat outgrape:
>>> On 8/5/25 12:27 PM, Nick Alcock wrote:
>>>> On 5 Aug 2025, Jan Beulich stated:
>>>>
>>>>> On 05.08.2025 13:43, Nick Alcock wrote:
>>>>>> On 1 Aug 2025, Jan Beulich stated:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> just came to notice that the testsuite runs are all building binaries for the
>>>>>>> host, not the target.
>>>>>>
>>>>>> Well... the libctf testsuite should be building binaries for both,
>>>>>> actually. libctf itself is a host library, with no target-dependent
>>>>>> components (it leans on bfd for those), so we do want to test it on the
>>>>>> host: that's where it'sll run.
>>>>>>
>>>>>> ... *however*. The binaries the testsuite builds fall into two
>>>>>> categories:
>>>>>>
>>>>>>    - "lookup binaries", which are executed on the host: some just do their
>>>>>>      own thing using libctf API calls, but others (e.g. in libctf-lookup/)
>>>>>>      are used to inspect...
>>>>>>    - ... object files and linked binaries which contain CTF. These are
>>>>>>      built with the target compiler, or should be.
>>>>>
>>>>> Provided there is one, that is. I surely don't have some 150 cross compilers
>>>>> installed.
>>>> Yeah, this is skipped if the target compiler is not available.
>>>>
>>>>>> The compilation of both these binaries happens in
>>>>>> libctf/testsuite/lib/ctf-lib.exp:run_lookup_test(), if you're
>>>>>> interested. They are driven by *.lk files: any *.lk files which do not
>>>>>> specify a source: are building lookup binaries only, on the host.
>>>>>>
>>>>>>>                          While this may be sensible when testing an individual
>>>>>>> target, (afaict) it's doing all the same over and over when doing bulk tests.
>>>>>>> I couldn't spot any means (e.g. an env var) to override this behavior, i.e.
>>>>>>> optionally suppress such redundant testing. If there is one, could you please
>>>>>>> point me in the right direction? If there is none, can something be added?
>>>>>>
>>>>>> ... sure: we could at least not rerun those tests that don't use source:
>>>>>> at all. I wonder what the least horrible approach would be... maybe a
>>>>>> configure flag to ask run_lookup_test to quietly do nothing if host !=
>>>>>> target and source: is not in use?
>>>>>
>>>>> Well, that's precisely what I don't know, or else I might have preferred to
>>>>> post a patch.
>>>> I guess a configure flag to libctf/configure.ac might do it. I'll see if
>>>> I can whip one up this week, before the summer holiday.
>>>
>>> As for libsframe, which is also a host library, all tests so far need to be run for the host.
>>>
>>> Skipping the libsframe testsuite for host != target when doing bulk tests is fine.  I too think that it makes sense to add a
>>> configure option (--disable-lisbframe-tests?) so they are skipped only when the usecase is doing bulk tests (and host != target).
>>
>> If multiple libs need this sort of thing, maybe we want a
>> --disable-host-tests option which disables such tests for both of them
>> when host != target?
> 
> Thats a better option name, but still lacks the context that the option 
> disables the tests only when cross builds are detected.
> 
> Perhaps --disable-host-tests-cross-build ? but then apart from libctf 
> and libsframe, are there other tests that could be disabled and are 
> expected to be tied to --disable-host-tests-cross-build ?

Do we need to restrict this to cross builds? People just shouldn't use
this option when they care about the testing being done.

Jan