Re: RFC: Unique testnames

[Pedro Alves <palves at redhat dot com>]

On 09/13/2018 03:29 PM, Jeff Law wrote:

> So to give wider background to the community.  It's the .sum file
> comparison tool provided by GCC (contrib/compare_tests) that doesn't
> handle this case gracefully.
> 
> If the tests have different PASS/FAIL states, the tool will report that
> there were regressions/errors in the new testrun.  Uniqueness of the
> testnames avoids this problem.
> 
> The irony is that the Codesourcery guys were raising the uniqueness of
> testnames issues for GCC eons ago (in the QMtest discussions).  I
> largely ignored it at the time since I've always done my comparisons by
> hand (thankfully others took on the task of addressing this issue for
> GCC).  Of course this stuff matters with automatic regression testers :-)
> 

We're pretty anal about this in GDB too, for the same reasons.  :-)  See:

 https://sourceware.org/gdb/wiki/GDBTestcaseCookbook#Make_sure_test_messages_are_unique

Note GDB avoids using " (foo)" at end of names as a way to make test
names unique.  Not sure whether that's relevant for binutils:

  https://sourceware.org/gdb/wiki/GDBTestcaseCookbook#Do_not_use_.22tail_parentheses.22_on_test_messages

Thanks,
Pedro Alves