| Summary: | binutils/testsuite/lib/binutils-common.exp: Support free-form shell commands and check patterns | ||
|---|---|---|---|
| Product: | binutils | Reporter: | Fangrui Song <i> |
| Component: | binutils | Assignee: | Not yet assigned to anyone <unassigned> |
| Status: | UNCONFIRMED --- | ||
| Severity: | normal | ||
| Priority: | P2 | ||
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Host: | Target: | ||
| Build: | Last reconfirmed: | ||
|
Description
Fangrui Song
2021-11-18 03:04:24 UTC
From https://maskray.me/blog/2021-08-08-toolchain-testing#split-file In Aug 2020, I added `split-file` to llvm-project which allows you to place multiple extra files in one file. It'd be nice if binutils has something similar. Use case A (organizing input of different formats (e.g. linker script+assembly) in one file). ```text # RUN: split-file %s %t # RUN: llvm-mc %t/asm -o %t.o # RUN: ld.lld -T %t/lds %t.o -o %t This is sometimes better than the %S/Inputs/ approach because the user can see the auxiliary files immediately and don't have to open another file. # asm ... # lds ... ``` Use case B (for utilities which don't have built-in input splitting feature): ```text // RUN: split-file %s %t // RUN: llc < %t/1.ll | FileCheck %s --check-prefix=CASE1 // RUN: llc < %t/2.ll | FileCheck %s --check-prefix=CASE2 Combing tests prudently can improve readability. For example, when testing parsing errors if the recovery mechanism isn't possible, grouping the tests in one file can more readily see test coverage/strategy. //--- 1.ll ... //--- 2.ll ... ``` (In reply to Fangrui Song from comment #0) > 3) Can we allow free-form shell commands instead of specialized directives? > > It is mythical what can and what can't be used in `#foo:`. > OK, a user can figure out this with trial and error. "Users" are here programmers, which are expected to be able to find the "myth" in binutils-common.exp and read the head comment, maybe find a flaw or extend it. The method to accomplish this isn't really different to most other programming-related problems. The difference is mostly that the testsuite for the tools isn't as documented as the tools themselves, just as can be expected. > If we allow free-form shell commands and use line prefixes to differentiate > the two output streams (objdump -d output and ld -Map output): > > ## Test local ifunc are dumped in the link map. > #RUN: ld: -shared -Map tmpdir/ifunc-1-x86.map --hash-style=sysv %s -o %t > #RUN: objdump -dw %t | check # by default, CHECK is the prefix > #RUN: cat tmpdir/ifunc-1-x86.map | check MAP > > #CHECK: [ \t0-9a-f]+:[ \t0-9a-f]+call[ > \t0-9a-fq]+<\*ABS\*(\+0x[0-9a-f]+|)@plt> > #CHECK-N: if there is a next line > #CHECK-N: likewise. > #CHECK: skip arbitrary lines > #CHECK-N: if there is a next line > > #MAP: Local IFUNC function ... Maybe this suggestion can fit run_dump_test but with the splitting-thing it evolves into a whole different kind of beast. That sounds like this should be a different "proc" (with a different suffix). Something reading from a descriptive file in a manner *similar* to run_dump_test may be useful. I just don't believe bolting stuff onto run_dump_test is the preferred method. People needing new features for new tests not simply matched by run_dump_test, or need to run random shell commands, usually write inline dejagnu-tcl in an existing or new .exp, usually for the purpose of a single test or array of similar tests. There's no myth or magic here - at least not of a form different to the kind of programming performed on the actual tools! |