[PATCH 00/43] aarch64: Add missing test coverage, plus some fixes

[Richard Earnshaw (lists)]

On 21/04/2025 22:13, Alice Carlotti wrote:
> This series addresses signifcant gaps in test coverage for aarch64 assembly and
> disassembly.  There are also two functional fixes included in this series:
> 
> Patch 01/43 disallows some incorrect SVE addressing modes that were being
> misparsed as a different valid addressing mode.
> 
> Patch 38/43 (with regenerated content in 39/43) fixes disassembly of the dgh
> instruction.
> 
> The patch I used to assess test coverage will be attached in a separate reply,
> but is not suitable for use upstream.  Prior to this series, approximately 10%
> of the entries in aarch64_opcode_table were not tested by the gas testsuite.
> More specifically, 347 out of 3560 entries were not used in assembly and 414
> out of 3560 entres were not used in disassembly.  After this series, all 3534
> remaining entries are used in assembly, while just 64 entries are unused in
> disassembly.  These 64 entries are all marked as pseudo instructions or as
> having aliases (F_PSEUDO or F_HAS_ALIAS), so the gaps are clearly accounted
> for.
> 
> I have made virtually no attempt to improve the quality of tests for
> instructions that are already tested, and where I have added to existing files
> I have matched the existing test quality.  For new test files, however, I have
> implemented tests that have much better coverage of the operand space
> (including ensuring that all variable bits in encodings take both possible
> values) - this is the same quality of (positive) test coverage that I would use
> if adding new instructions.
> 
> I have not investigated the coverage of error tests (or lack thereof) for
> several reasons.  It is harder to assess coverage of invalid inputs, and
> accurate errors are also less important than accurate assembly/disassembly of
> valid code (though both are still important).  It is also likely that changes
> that break existing error handling would also break valid assembly or
> disassembly, or would already be picked up by an existing test.
> 
> 
> The series consists of the following:
> 
> Patches 01-03 address issues with the AARCH64_OPND_SVE_ADDR_R operand.  I
> identified these issues originally when working on nearby code, but the
> duplicate opcode table entries also show up as a gap in disassembly test
> coverage.
> 
> Patches 04-06 improve the regexes used in three existing assembly tests, to
> facilitate later modifications to these test files.
> 
> Patches 07-10 add instructions to four existing test files, in cases where the
> previously untested instructions are part of a natural encoding group that was
> already mostly covered by an existing test file.
> 
> Patch 11 adjusts tests for the fmops instruction, which was already being
> assembled, but only within an error test.
> 
> Patches 12-33 add new tests to cover the rest of the testsuite gaps.  Some of
> these tests include instructions that were already covered by an existing test;
> I have explicitly mentioned all such cases in the commit messages.
> 
> Patches 34-43 add missing F_PSEUDO, F_ALIAS and F_HAS_ALIAS annotations to
> appropriate instructions.  This fixes dgh disassembly (with patch 40 cleaning
> up the dgh tests).
> 
> 
> Is this series ok for master?

Many thanks for this, it's a major improvement in our testing coverage.  You've detailed how you checked the coverage, but could you say a bit about how you've validated the additional tests for correctness?  The issue in patch 1 is evidence enough that bugs can slip in if we just assume that the output of the tools is correct.

I've minor issues with patches 5, 29 and 30, but the rest are all ok and those 3 are also OK with the suggested issues addressed.

R.